*2.1. Participants*

A total of 50 volunteers were recruited for this experiment. Among them, 29 were male volunteers and 21 were female volunteers. The ratio of male to female was approximately 3:2. All volunteers were right-handed, in good health, and had no history of mental illness or cardiovascular and cerebrovascular diseases. In addition, all volunteers participated in the experiment for the first time and only once. Before the experiment, all volunteers were informed of the experiment details and signed the informed consent form of the experiment.

#### *2.2. Experiment Paradigm*

In order to increase the controllability and practicability of upper-limb auxiliary equipment, four common upper-limb movement paradigms in daily life designed in daily life: Lifting up, putting down, pulling back and pushing forward. During the experiment, the near-infrared data of the subjects in different motion states were collected to provide a supervised learning data set for subsequent research on motion intention recognition. The experimental process is shown in Figure 1. Tasks 1, 2, 3, and 4 in the figure represent the four action paradigms of lifting up, putting down, pulling back, and pushing forward, respectively.

During the experiment, the subjects should keep their scalp clean and keep their hair dry before the experiment. It should be noted that the subjects need to rest for about 60 s before the start of the experiment. The rest time between lifting and lowering actions was 10 s, and the rest time between other actions was about 50 s. The start of each task and the rest time were all controlled by the subjects autonomously without any prompts from the outside world. The specific paradigm of the experiment is shown in Figure 2. Figure 2a–d represents the four action paradigms of lifting up, putting down, pulling back, and pushing forward, respectively.

*Electronics* **2021**, *10*, x FOR PEER REVIEW 3 of 10

**Figure 1.** The diagram of the experiment. **Figure 1.** The diagram of the experiment. pushing forward, respectively.

#### pushing forward. *2.3. Cortical Regions*

pushing forward.

*2.3. Cortical Regions*  The FOIRE-3000 near-infrared acquisition system of Shimadzu Corporation was used as a brain signal acquisition device to record the changes in the concentration of oxygen, The FOIRE-3000 near-infrared acquisition system of Shimadzu Corporation was used as a brain signal acquisition device to record the changes in the concentration of oxygen, deoxygenation, and total oxygen hemoglobin in the experiment [10]. Because this experiment emphasized the spontaneous movement intention of the subjects, task execution and rest time were controlled by the subjects spontaneously, we chose to use continuous mode to measure the cerebral hemoglobin information during the experiment. The equipment sampling time was 130 ms.

**Figure 2.** The experimental diagram of upper limb movement. (**a**) Lifting up, (**b**) putting down, (**c**) pulling back, and (**d**) *2.3. Cortical Regions*  The FOIRE-3000 near-infrared acquisition system of Shimadzu Corporation was used as a brain signal acquisition device to record the changes in the concentration of oxygen, This study used the internationally recognized 10–20 system as the positioning standard to locate brain functional areas. The experiment designed a 4 × 4 headgear layout to detect the above key functional areas. There were a total of 24 effective test channels. The overall headgear layout is shown in Figure 3. In the layout, the Cz point was used as a reference point for the relative position, and the distance between Cz and the center of the 23rd channel was 1.5 cm. The entire layout can cover four brain functional areas: PFC, SMA, PMC (PMC = PMCL + PMCR), PMCR, and M1 brain area function [11].

sampling time was 130 ms.

**Figure 3.** The schematic diagram of 4 × 4 headgear layout. **Figure 3.** The schematic diagram of 4 × 4 headgear layout.

#### **3. Data Analysis 3. Data Analysis**

#### *3.1. Data Preprocessing 3.1. Data Preprocessing*

The interference components in the near-infrared signal mainly include baseline drift, physiological interference, and high-frequency noise. Mathematical morphology has a very strict data theoretical basis, and research shows that this method achieved good results in nonlinear signal processing. Corrosion calculation and expansion calculation were the basic calculation methods of this method. Based on the corrosion expansion calculation, two different morphological calculations were used for the combination of opening and closing. Among them, the open operation used the first expansion and then the corrosion operation method to eliminate the peak of the signal to filter the peak noise above the signal, and the closed operation used the first erosion and then the expansion operation to fill the signal trough to smooth or suppress the signal valley noise. We combined the on-off filter to eliminate both positive and negative impulse noises in the signal to avoid unidirectional deviation of the filtered signal [12,13]. The interference components in the near-infrared signal mainly include baseline drift, physiological interference, and high-frequency noise. Mathematical morphology has a very strict data theoretical basis, and research shows that this method achieved good results in nonlinear signal processing. Corrosion calculation and expansion calculation were the basic calculation methods of this method. Based on the corrosion expansion calculation, two different morphological calculations were used for the combination of opening and closing. Among them, the open operation used the first expansion and then the corrosion operation method to eliminate the peak of the signal to filter the peak noise above the signal, and the closed operation used the first erosion and then the expansion operation to fill the signal trough to smooth or suppress the signal valley noise. We combined the on-off filter to eliminate both positive and negative impulse noises in the signal to avoid unidirectional deviation of the filtered signal [12,13].

deoxygenation, and total oxygen hemoglobin in the experiment [10]. Because this experiment emphasized the spontaneous movement intention of the subjects, task execution and rest time were controlled by the subjects spontaneously, we chose to use continuous mode to measure the cerebral hemoglobin information during the experiment. The equipment

This study used the internationally recognized 10–20 system as the positioning standard to locate brain functional areas. The experiment designed a 4 × 4 headgear layout to detect the above key functional areas. There were a total of 24 effective test channels. The overall headgear layout is shown in Figure 3. In the layout, the Cz point was used as a reference point for the relative position, and the distance between Cz and the center of the 23rd channel was 1.5 cm. The entire layout can cover four brain functional areas: PFC,

SMA, PMC (PMC = PMCL + PMCR), PMCR, and M1 brain area function [11].

## *3.2. Data Preprocessing*

#### *3.2. Data Preprocessing*  3.2.1. Action Initiation Intent Feature Extraction

3.2.1. Action Initiation Intent Feature Extraction In order to accurately and quickly detect the time point of the subject's transition from resting to exercise, the selected feature has the ability to detect sudden changes and singular signals. Near-infrared signals are non-stationary and nonlinear. Among a variety of feature categories based on biological information to distinguish resting/exercise status, Teager–Kaiser energy operator and slope value were used in many initial detection studies of biological signals [14,15]. By calculating the Teager–Kaiser energy operator at each In order to accurately and quickly detect the time point of the subject's transition from resting to exercise, the selected feature has the ability to detect sudden changes and singular signals. Near-infrared signals are non-stationary and nonlinear. Among a variety of feature categories based on biological information to distinguish resting/exercise status, Teager–Kaiser energy operator and slope value were used in many initial detection studies of biological signals [14,15]. By calculating the Teager–Kaiser energy operator at each time point and the blood oxygen slope characteristics at each time point, the instantaneous changes attributed to the signal at each time point were obtained, and the waveform changes of the measured signal were tracked in real time.

#### 3.2.2. Motion State Feature Extraction

In the exercise state classification task, the goal was to recognize the exercise intention before the real action. Therefore, it was necessary to extract the data of the real pre-exercise time period for analysis. The training time period selection method is shown in Figure 4. Took the starting point of the real action as the reference point (0), took the signal from −3 s to 0 s as the training time period, and applied the sliding time window method. The length of the window was set to 20. The overlapping length of two adjacent sliding windows was three sampling points. Then, we calculated the features within the time window to obtain the feature data set.

changes of the measured signal were tracked in real time.

3.2.2. Motion State Feature Extraction

time point and the blood oxygen slope characteristics at each time point, the instantaneous changes attributed to the signal at each time point were obtained, and the waveform

In the exercise state classification task, the goal was to recognize the exercise intention before the real action. Therefore, it was necessary to extract the data of the real pre-exercise time period for analysis. The training time period selection method is shown in Figure 4. Took the starting point of the real action as the reference point (0), took the signal from -3 s to 0 s as the training time period, and applied the sliding time window method. The length of the window was set to 20. The overlapping length of two adjacent sliding windows was three sampling points. Then, we calculated the features within the time window

**Figure 4.** The schematic diagram of 4 × 4 headgear layout.

#### **Figure 4.** The schematic diagram of 4\*4 headgear layout. *3.3. Feature Selection*

3.3.1. Action Initiation Intention Feature Selection

to obtain the feature data set.

*3.3. Feature Selection*  3.3.1. Action Initiation Intention Feature Selection During the data collection process, because the subjects were at rest time and the length of time in the task was different, there were also differences in the number of samples in the rest segment and the number of samples in the task segment in the collection of data. The construction of the action initiation intention recognition model faced the problem of sample imbalance between different categories. Compared with the traditional RF feature selection algorithm, this paper proposes an improved feature selection method based on RF. For the original feature set, the algorithm first set the final number of retained features, then used the sequential backward search method to remove the features with During the data collection process, because the subjects were at rest time and the length of time in the task was different, there were also differences in the number of samples in the rest segment and the number of samples in the task segment in the collection of data. The construction of the action initiation intention recognition model faced the problem of sample imbalance between different categories. Compared with the traditional RF feature selection algorithm, this paper proposes an improved feature selection method based on RF. For the original feature set, the algorithm first set the final number of retained features, then used the sequential backward search method to remove the features with low importance score from the feature set, until the number of features was the specified number, the algorithm stopped, and the reserved feature subset was taken as the final feature set [14,15].

#### low importance score from the feature set, until the number of features was the specified 3.3.2. Movement State Feature Selection

number, the algorithm stopped, and the reserved feature subset was taken as the final feature set [14,15]. 3.3.2. Movement State Feature Selection Facing the high dimensionality of features in multi-classification tasks, this paper proposes a combined feature selection method based on ReliefF algorithm and genetic algorithm. In order to eliminate irrelevant features and redundant features in the original feature space, the ReliefF algorithm was first applied to evaluate the feature importance Facing the high dimensionality of features in multi-classification tasks, this paper proposes a combined feature selection method based on ReliefF algorithm and genetic algorithm. In order to eliminate irrelevant features and redundant features in the original feature space, the ReliefF algorithm was first applied to evaluate the feature importance based on the distance measurement between sample features, and features with higher feature importance were retained as a feature subset [16,17]. On this basis, genetic algorithm was applied to find the optimal feature combination under the current feature subset. The flow of the feature selection method is shown in Figure 5. *Electronics* **2021**, *10*, x FOR PEER REVIEW 6 of 10

#### **Figure 5.** Feature selection flowchart. **4. Results**

#### **4. Results**  *4.1. Data Preprocessing Results*

*4.1. Data Preprocessing Results*  In order to obtain the ideal morphological filtering effect, the signal-to-noise ratio and mean square error were used as evaluation indicators to conduct experiments on filters with different structural elements, different amplitudes, and different widths. We selected the parameter combination with the best comprehensive performance to construct the morphology learn filters. Table 1 shows the maximum signal-to-noise ratio ௫ correct. **Figure 5.** Feature selection flowchart. In order to obtain the ideal morphological filtering effect, the signal-to-noise ratio and mean square error were used as evaluation indicators to conduct experiments on filters with different structural elements, different amplitudes, and different widths. We selected the parameter combination with the best comprehensive performance to construct the morphology learn filters. Table 1 shows the maximum signal-to-noise ratio *SNRmax* the minimum filtering error *MSEmin* and the corresponding combination of amplitude and width of the near-infrared signal after filtering by different structural elements.

the minimum filtering error and the corresponding combination of amplitude and width of the near-infrared signal after filtering by different structural elements.

The initial intention detection task of this action divided the data samples of 50 subjects participating in the experiment into a data set, of which the experimental data of 40 subjects were used as the cross-validation set, the total number of samples was 65,354, and the experimental data of the remaining 10 subjects were used as a test set, the total number of samples was 16,392. Figure 6 shows the test results of the participants' initiation of action during the experiment. The red dashed line represented the start time of the action, and the green dashed line marked the end of the task. In the example in the figure, "0" and "1" were used to indicate the rest state and task state, respectively. If it was judged as "0" in the rest period, it was regarded as correct, and when it was judged as "1", it was regarded as a misjudgment, it was judged in the task segment. If it was "1", it was deemed

*4.2. The Results and Analysis of Initial Intention Detection* 

4.2.1. The Results of Initial Intention of the Action

**Structural Element Amplitude Width** 

**Table 1.** The filtering effect of various structural elements on the near-infrared signal.

Cosine 0.8 70 11.9419 0.000219995

semicircle 1 70 13.0082 0.000197746 Straight line 1 50 14.9635 0.000162624


**Table 1.** The filtering effect of various structural elements on the near-infrared signal.

#### *4.2. The Results and Analysis of Initial Intention Detection*

#### 4.2.1. The Results of Initial Intention of the Action

The initial intention detection task of this action divided the data samples of 50 subjects participating in the experiment into a data set, of which the experimental data of 40 subjects were used as the cross-validation set, the total number of samples was 65,354, and the experimental data of the remaining 10 subjects were used as a test set, the total number of samples was 16,392. Figure 6 shows the test results of the participants' initiation of action during the experiment. The red dashed line represented the start time of the action, and the green dashed line marked the end of the task. In the example in the figure, "0" and "1" were used to indicate the rest state and task state, respectively. If it was judged as "0" in the rest period, it was regarded as correct, and when it was judged as "1", it was regarded as a misjudgment, it was judged in the task segment. If it was "1", it was deemed correct. *Electronics* **2021**, *10*, x FOR PEER REVIEW 7 of 10

**Figure 6.** Schematic diagram of the judgment result of the initial intention of the action. **Figure 6.** Schematic diagram of the judgment result of the initial intention of the action.

In order to explore the impact of different feature dimensions and different probability thresholds on the classification results, we used 40, 50, 60, 70, 80, and 90 as the feature dimensions, and applied the "threshold shift method" with 0.3, 0.4, 0.5, 0.6, and 0.7, respectively, as probability threshold to carry out the experiment. We used RF and gradient boosting tree (GBDT) classification algorithm to detect the initial intention of the action. It can be seen that when the feature dimension was 60 and the threshold was 0.4, both classification algorithms maintain their optimal performance. The optimal recognition results of the RF and GBDT are shown in Tables 2 and 3. In order to explore the impact of different feature dimensions and different probability thresholds on the classification results, we used 40, 50, 60, 70, 80, and 90 as the feature dimensions, and applied the "threshold shift method" with 0.3, 0.4, 0.5, 0.6, and 0.7, respectively, as probability threshold to carry out the experiment. We used RF and gradient boosting tree (GBDT) classification algorithm to detect the initial intention of the action. It can be seen that when the feature dimension was 60 and the threshold was 0.4, both classification algorithms maintain their optimal performance. The optimal recognition results of the RF and GBDT are shown in Tables 2 and 3.

**Table 2.** Optimal results of the RF algorithm. By comparing the comprehensive indicators of the two classifiers of RF and GBDT, the RF is finally selected as the final classification algorithm.

By comparing the comprehensive indicators of the two classifiers of RF and GBDT,

The distribution of the brain areas of each feature and the feature weight proportion of the blood oxygen type were sorted out. The results show that among the three types of characteristics of oxygen, deoxygenation, and total oxygen, the dominant characteristic channels were channels 1, 4, and 5, and the important characteristic channels were 6, 12, 13, 19, 20 channels. The remaining channels are generally active channels, which are help-

In this classification task, the 50 test data samples participating in the experiment were divided into data sets. Among them, the experimental data of 40 subjects were used

the RF is finally selected as the final classification algorithm.

*4.3. The Results and Analysis of Exercise Status Recognition* 

4.3.1. Recognition Results of Motion State

4.2.2. Feature Analysis

ful to the recognition task.

**Accuracy Rate False Judgment Rate Discrimination Delay (s) Comprehensive Index** 

**Table 3.** Optimal results of GBDT algorithm.

**Accuracy Rate False Judgment Rate Discrimination Delay (s) Comprehensive Index** 

Train 94.4% 1.1% −0.267 1.200 Test 92.5% 2.1% −0.067 0.971


**Table 2.** Optimal results of the RF algorithm.

**Table 3.** Optimal results of GBDT algorithm.


#### 4.2.2. Feature Analysis

The distribution of the brain areas of each feature and the feature weight proportion of the blood oxygen type were sorted out. The results show that among the three types of characteristics of oxygen, deoxygenation, and total oxygen, the dominant characteristic channels were channels 1, 4, and 5, and the important characteristic channels were 6, 12, 13, 19, 20 channels. The remaining channels are generally active channels, which are helpful to the recognition task.

#### *4.3. The Results and Analysis of Exercise Status Recognition*

#### 4.3.1. Recognition Results of Motion State

In this classification task, the 50 test data samples participating in the experiment were divided into data sets. Among them, the experimental data of 40 subjects were used as the cross-validation set, the total number of samples was 160, and the experimental data of the remaining 10 subjects were used as the test set. The total number of samples was 40.

By comparing the final convergence results of the genetic algorithms of different classification algorithms, the final iterative results of the four classification algorithms are shown in Table 4. The final classification algorithm was sorted from highest to lowest score as SVM > LR > LDA > NB. Therefore, for the classification task of this data set, SVM was selected as the classifier of the final results [18]. The SVM algorithm's recognition results for all subject samples are shown in Table 5.

**Table 4.** The best evaluation indicators of different classifiers.



#### 4.3.2. Feature Analysis

The distribution of the brain regions of each feature and the weight ratio of the blood oxygen type were classified and statistics [19]. In the characteristics of oxygen content, the dominant characteristic channels are channel 1, 4, 5, and 12, and the important characteristic channels are channel 6, 11, 13 and 15, among which channel 2, 21, 23, and 24 have no relevant characteristics, and other channels are generally active. Among the deoxygenation characteristics, the dominant characteristic channels are channel 1, 4, 5 and 6, and the important characteristic channels are channel 3, 7, 11, 12, 15, and 19. Channel 24 has no relevant characteristics, and other channels are generally active. Among the total oxygen characteristics, the dominant characteristic channels are channel 1, 4, 5, 6, 9, 12, and 13, and the important characteristic channels are channel 7, 8, 10, 17, 19, and 20. Channel 22, 23, and 24 have no relevant characteristics, while other channels are generally active.

## **5. Discussion**

This paper proposed a set of upper limb rehabilitation training robot systems based on user spontaneous movement fNIRS-BCI. Four upper-limb movement paradigms that are highly related to daily life were designed. The combined filtering method of morphological filtering and Butterworth band-pass filtering was used to preprocess the signal. In terms of the recognition algorithm for detecting the initial intention of upper limb movements, GBDT and RF were selected for classification experiments. Finally, RF classifiers with better comprehensive indicators were selected as the final classification algorithm. The best offline recognition rate was 94.4% (151/160), the false positive rate was 1.1% (733/65354). The ReliefF algorithm based on distance measurement and the genetic algorithm proposed in the genetic theory were used to select features. In terms of upper limb motion state recognition algorithms, LR, SVM, NB, and LDA were selected for experiments, and Kappa coefficient was used as the classification index to evaluate the performance of the classifier. Finally, SVM classification got the best performance. The Kappa coefficient of the classification result was 0.792, and the four-class recognition accuracy rate was 84.4%.

This study has many advantages: First of all, in the process of the experiment, the corresponding tasks were automatically controlled by the subjects, during which there was no task hint and no external stimulation. Four upper-limb movement paradigms that are highly related to daily life were designed. At present, numerous research subjects need to carry out experiments according to stimulus cues (mostly visual or auditory stimulus). Kus et al. instructed the subjects to imagine left/right/feet movements according to the prompts displayed on the screen, and tried to classify different imaginary movements [20]. Tengfei Ma et al. instructed the subjects to perform left/right hand finger tapping task guided by a voice prompt [21]. Furthermore, in this paper, the research on upper limb movement intention was based on the blood oxygen signal before the real action task. According to the brain signal before the actual action, the motor intention was judged, which provided an important practical basis for the realization of real-time control application based on brain computer interface technology. Now the majority of research on motor imagery is based on the extraction of real motion signals for experiments, but this cannot guarantee great real-time performance. For example, Trakoolwilaiwan et al. extracted the time samples of the subjects during the resting and left and right hand movement tasks for training and recognition [22,23]. Finally, in this paper, a variety of algorithms (including ensemble learning) were selected for experiments and achieved high accuracy. In the aspect of the initial intention recognition algorithm of upper limb movement, GBDT and RF are selected for classification experiments. RF classifier with better comprehensive indicators was selected as the final classification algorithm. The best offline recognition rate was 94.4%(151/160). In the aspect of upper limb motion state recognition algorithms, LR, SVM, NB, and LDA were selected for the experiment. SVM got the highest recognition accuracy in four classification recognition (84.4%). Now, the recognition accuracy of multi-classification tasks on motor imagery is relatively low in a lot of research. Keum Shik Hong et al. simultaneously obtained fNIRS signals of mental arithmetic (MA), right hand motor imagery (RI) and left hand motor imagery (LI) from prefrontal cortex and primary motor cortex. Multiclass linear discriminant analysis was utilized to classify MA vs. RI vs. LI with an average classification accuracy of 75.6% across the ten subjects, for a 2–7 s time window during the 10 s task period [22,24]. Wang Wenle et al. collected fNIRS signals of 16 subjects' brain motor areas during the actual and imaginary movements of six types of sign language tasks. Finally, they used AdaBoost.M1, SVM, LDA, HMM, NB, and KNN algorithms to recognize the fNIRS signals. Finally, LDA achieves the highest average

classification accuracy (78.70% ± 1.78%) [23,24]. The recognition accuracy in our paper is higher than 75.6% and 78.70% ± 1.78%.

Although this study has the above advantages, there are still some areas that need improvement. For example, the cascade structure adopted by the current system will cause errors to occur between different classification tasks, during which a problem in a certain recognition link will affect the final overall recognition rate. Therefore, in future research, it is expected that other real-time algorithm frameworks can be used to make up for this deficiency, such as parallel structure and hybrid structure.

**Author Contributions:** Conceptualization, funding acquisition, project administration, supervision, methodology, C.L.; data analysis, experimental verification, writing, Y.X.; data preprocessing, methodology, L.H.; data curation, Y.Z.; investigation, S.K.; supervision, formal analysis, L.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by National Natural Science Foundation of China (61673286), and National key research and development program of China (U1713218).

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Electronics* Editorial Office E-mail: electronics@mdpi.com www.mdpi.com/journal/electronics

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34

www.mdpi.com

ISBN 978-3-0365-5671-0