1. Introduction
The loss of limbs, often resulting from trauma, illness, or congenital conditions, significantly impacts an individual’s mobility and quality of life [
1,
2]. Advances in prosthetic technology have significantly improved the functional capabilities and comfort of amputees, facilitating greater independence and participation in daily activities [
3].
Among these innovations, the utilization of electroencephalography (EEG) signals to control prosthetic devices represents the leading edge of innovative research [
4,
5]. EEG, a non-invasive technique that records electrical activity along the scalp, provides a direct window into the brain’s activity, offering insights into a user’s intentions [
6]. Harnessing these signals, research has been dedicated to design innovative systems able to measure neural activities and convert them into prosthetic devices’ movements, enabling individuals with limb impairments to interact with the world in unprecedented ways [
7]. EEG-based approaches have overcome the traditional methods based on residual muscle activity gathered from surface electromyography or external switches, which are often characterized by poor movement accuracy and a restricted range of motion [
8,
9]. Several studies have shown a significant incidence of issues that amputees face when using conventional control methods, with up to 70% reporting difficulties, such as comfort and stability [
10]. Instead, EEG-based prosthetic control systems are directly fed with signals associated with motor intentions. Through signal processing algorithms and machine learning techniques, these systems decode neural patterns, discerning commands for various movements such as walking, running, or even fine tasks associated with the upper limbs [
11]. This not only enhances the naturalness and fluidity of prosthetic movements but also empowers users with greater autonomy and functionality in their daily life. Moreover, the versatility of EEG-based prosthetic control extends beyond mere movement execution.
Thus, it is clear how the EEG signal identification and classification of lower-limb activities has gained significant attraction during the last two decades [
12,
13]. Thus, several studies have been published to understand the feasibility of classifying gait-related parameters through the analysis of EEG signals. Among others, Liu et al. [
14] explored the use of BCI to decode lower-limb movement intention from EEG signals, aiming to promote motor recovery and brain plasticity. The study focused on continuous classification and asynchronous detection of movement-related cortical potentials during self-initiated ankle plantar flexion tasks. In comparison to current online detection techniques, the suggested framework showed a greater true-positive rate, fewer false positives, and comparable latencies [
14]. Chai et al. [
15] attempted to identify gait-related movements in subjects walking with and without an exoskeleton by analyzing the EEG signals. By using features related to mu- and beta-frequency bands, an average classification accuracy of 74% was obtained for the testing sets [
15]. A comparison among different neural network algorithms fed with EEG-based features was proposed in [
16], revealing the support vector machine model as the best one, achieving accuracy greater than 98% in the recognition of gait phases. A similar approach was proposed by Bodda et al. for the identification of the stance and swing phase utilizing the analysis of EEG activity [
17]. The intention of walking movement has been discriminated by using a support vector machine with RBF kernel, achieving an overall accuracy of 73% when the algorithm was fed with features extracted from EEG signals gathered from six healthy subjects and two amputees [
18]. To automatically identify unstable gait patterns, Soangra and colleagues [
19] applied both machine learning and deep learning algorithms to implement a BCI able to prevent falls, finding the recurrent network as the best solution with an accuracy greater than 80% [
19]. A support vector machine classifier combined with a directed acyclic graph was validated to move a prosthetic leg according to specific gait trajectories [
20]. By testing the method on three healthy subjects, it was observed that all of them were able to smoothly walk on floors and stairs. It is clear that all the previously reported articles are focused on specific parameters of the gait, such as gait phases or stability, rather than the discrimination of different walking activities. In addition, no studies have been published to understand the performance of EEG-based algorithms when walking on irregular terrains; in fact, all the papers only evaluated the performance in controlled environments.
Despite the plethora of studies and the advantages of using EEG for developing control systems of prostheses, challenges persist in guaranteeing the widespread adoption of EEG-controlled prosthetics, ranging from signal variability to the need to test the methodology in uncontrolled environments over irregular terrains. To the best of the authors’ knowledge, the comparison among different machine learning algorithms to identify the best method for the identification of different locomotion tasks in different walking terrains outdoors is still lacking, since all the studies focused on level walking in a laboratory. Therefore, the contribution of this study is to fill this gap by comparing the performance of different machine learning algorithms, particularly random forest and k-nearest neighbors, in classifying locomotion activities based on EEG data. The findings of this study are expected to advance the field by providing insights into the most effective algorithms for implementing EEG-based prosthetic control systems.
3. Results and Discussion
An example of the graph obtained by the application of infomax ICA and ADJUST is reported in
Figure 5, where it is possible to see the statistical approach revealing the probability that each IC belongs to brain-related activity.
For the sake of completeness,
Table 5 reports the number of brain-related features for each activity extracted by the two tested ICA’s algorithms.
Moving to the machine learning algorithms, the results of the performance analysis are depicted in
Table 6.
In the case of the random forest (RF) algorithm, the ICs dataset demonstrates an overall accuracy of 0.73, which is lower than the threshold for the optimum classifier [
33]. Within this dataset, R is notably high for indices W and DR, achieving values of 0.96 and 0.92, respectively. However, R is lower for DS and AS, with values of 0.25 and 0.47, whereas the AR index exhibits a moderate R of 0.61. The precision values vary, ranging from 0.70 for W to 0.89 for AR, with DS achieving the highest P at 0.79. Finally, the F1-score is highest for DR at 0.87 and lowest for DS at 0.38. When analyzing the adj_ICs dataset with the RF algorithm, there is a marked improvement in overall accuracy, which rises to 0.93, falling in the optimal range [
33]. Recall remains very high across all indices, particularly for W (0.99) and DR (0.98), while DS, despite having the lowest R among the indices, still records a substantial 0.80, which is in line with the above-mentioned threshold. P is consistently high across all indices, with values exceeding 0.90. Correspondingly, the F1-score also shows high values, particularly for DR (0.98) and W (0.95). Conversely, the k-nearest neighbors (kNN) algorithm exhibits a lower overall accuracy of 0.64 on the ICs dataset. For this dataset, the R is highest for W (0.84) and DR (0.83), but it is quite low for DS (0.24). Precision for kNN varies moderately, with values ranging from 0.53 for AS to 0.79 for DS. The F1-score is highest for DR at 0.76 and lowest for DS at 0.34. The performance of the kNN algorithm improves when applied to the adj_ICs dataset, achieving an overall accuracy of 0.80. R is highest for DR (0.99) and W (0.91) but remains low for DS (0.27). P is, instead, highest for DS at 0.93 and lowest for DR at 0.67. The F1-score ranges from 0.42 for DS to 0.87 for W.
By comparing the results, the random forest algorithm consistently outperforms kNN in terms of overall accuracy for both datasets. By considering all the metrics together, it is clear that only the RF algorithm on the adj_ICs dataset revealed itself as optimum for all the indices, whereas the kNN algorithm implemented on the adj_ICs dataset does not meet all the performance criteria, although it is characterized by an optimal overall accuracy. In fact, even if high accuracy indicates that the classifier is generally making correct predictions, a lower value of precision does not allow us to exclude a large number of false positives [
34]. Considering the final aim of the classifier, which is to be implemented into a control system for prostheses, it is clear how a classifier with a high rate of false positives can severely affect the user experience by causing safety risks, increased cognitive load, and reduced performance [
35]. Moreover, the RF algorithm must be preferred to kNN for other theoretical reasons. One of the primary benefits of RF is its robustness and ability to handle a wide variety of data types and structures [
31]. Unlike kNN, which relies on the notion of distance metrics and can struggle with high-dimensional data, RF constructs multiple decision trees and merges their results to improve accuracy and control overfitting. This ensemble approach allows RF to manage complex datasets, as EEG data, more effectively and produce more reliable predictions. Another significant advantage of RF is its robustness to noise and overfitting, which is fundamental in the case of the EEG dataset. This is why each tree in the forest is built from a random subset of the data, which helps to mitigate the impact of anomalies and noise, leading to more generalized and stable predictions [
31]. Speed and scalability are other areas where RF outperforms kNN. Once a random forest model is trained, making predictions is relatively fast because it involves threshold-based classification rather than computing distances to every training example, as required by kNN. This efficiency makes random forest more suitable for large-scale applications where rapid predictions are necessary, for example, in the real-time control of prostheses [
36]. Concerning the comparison between the two ICA algorithms, it is clear that the use of the ADJUST algorithm for the extraction of the independent components allows continuous increase of the performance. This finding is in line with the literature, where it has been proved that ADJUST provides a more robust and reliable method for artifact removal [
29]. As a consequence, it has been already proven that by removing artifacts and enhancing the signal-to-noise ratio, data are better suited for training and testing supervised learning models [
37]. Finally, the achieved performance of the best-performing algorithm is comparable with the one reported in the literature for the discrimination of other gait-related activities, such as the risk of falls [
19], gait phases, [
15,
16] and gait initiation [
18], even if the results have been obtained with different machine learning algorithms. This finding could suggest the development of a control system based on different machine learning algorithms, each tailored to a specific task.
By focusing on the specific errors made by each classification algorithm, in RF based on the ICs dataset, RF struggles notably with detecting ascending stairs and descending stairs movements. This is reflected in its low recall values for these activities, indicating that the algorithm often fails to identify these movements when they occur. This low recall suggests a high rate of false negatives, where actual instances of these movements are missed by the model. The precision of RF across the various movements also highlights its classification errors; for ascending stairs and descending stairs movements, the algorithm shows moderate-to-low precision, meaning that while it identifies some instances correctly, it also misclassifies a significant number of non-relevant instances as these movements. This is evident from the precision scores, where ascending stairs and descending stairs movements have lower values compared to other activities. The adjustment of ICs improves both recall and precision across the board, reducing the occurrence of these false negatives and false positives and resulting in a more balanced performance. kNN, instead, exhibits a different error profile. With the ICs dataset, kNN’s recall is particularly low for ascending stairs and descending stairs movements, similar to that of RF, which suggests that the algorithm frequently misses these movements. This indicates that kNN has a high false-negative rate, where true instances of these movements are not detected. Additionally, kNN shows lower precision for ascending stairs and descending stairs movements, implying that when the algorithm does predict these movements, it often does so incorrectly, leading to a high rate of false positives. With the adjusted ICs dataset, kNN shows improvements in recall for most movements, especially for ascending stairs and descending ramps. This suggests that the adjustments help the algorithm better identify these activities, though challenges remain. Precision also improves, but not uniformly across all movements. Despite these improvements, kNN still exhibits variability in its performance, reflecting a continued presence of classification errors in certain movements. These error patterns highlight the need for further refinement in both algorithms to achieve more reliable and accurate movement detection.
By summarizing, we can affirm that the RF fed with EEG-based features extracted through the ADJUST algorithm for the independent component analysis could be a suitable solution for implementing a control system for the real-time movement of prosthetic limbs also in uncontrolled environments, such as the daily activities involving locomotion on irregular terrains.
Limitations
Although the results are promising, the study presents some limitations that must be addressed to fully validate and generalize the findings. To build on this work, it is crucial to conduct additional research that encompasses a larger and more diverse sample. Expanding the pool of enrolled participants to include individuals with different demographic characteristics will enhance the representativeness of the results and ensure that the findings are broadly applicable. Furthermore, it is important to apply the algorithm by including participants who are actual users of prosthetic devices. This practical testing will provide valuable insights into the algorithm’s effectiveness and usability in everyday scenarios, including various distractions and obstacles and also considering possible optimization for the computational time required by the signal process. Moreover, further machine and deep learning algorithms will be compared. By addressing these aspects, future research can offer a more comprehensive understanding and confirm the algorithm’s practical utility across different contexts and populations.
Finally, due to the spread of flexible electronic sensors in several applications for biological healthy data collection [
38,
39,
40,
41], their applicability for EEG acquisition could be tested.