1. Introduction
As of 2016, Japan’s elderly population, people older than 65 years old, was 34.61 million, accounting for 27.3% of the whole population [
1]. This report underscores the severity of Japan’s rapidly aging society. According to Ministry of Health, Labor, and Welfare statistics, the primary reason for care nursing is stroke, which accounts for 21.5% of all cases requiring care [
2]. Cerebral stroke causes intellectual disability and language disorders, together with hemiplegia of every body part including the legs. Early rehabilitation is necessary to treat hemiplegia. Training to improve range of motion and muscle strength is typically conducted using a continuous passive motion (CPM) device [
3,
4]. However, such efforts are ineffective after the chronic phase more than six months after cerebral infarction.
Among the promising techniques is a brain–computer interface (BCI) by which electroencephalogram (EEG) signals (i.e., surface electrical brain activity measured on multiple electrodes placed on the scalp) are typically used to operate the rehabilitation device. Such a device provides visual stimulation that evokes brain activity of event-related potentials (ERP) [
5,
6] or steady-state visually evoked potentials (SSVEP), which correspond to the temporal frequency of the stimulus [
7,
8].
One field for BCI application is physical rehabilitation. Recently, so-called repetitive facilitative exercise (Kawahira method) [
9,
10] has drawn attention in the rehabilitation field. Stimulation is given to the neural system (brain) by observing an affected limb with motor intention (imagery), while the therapist moves the limb. This stimulation can be applied to injured muscles or tendons. Such treatment might facilitate the development of a novel neural circuit responsible for intentional movement by reconstructing novel circuitry and improving motor ability. This “neurorehabilitation” is expected to be beneficial for patients with chronic difficulties.
A fundamental human behavior is walking, and the ability to walk supports the independent life of an elderly person. Thus, ankle joint rehabilitation is emphasized, because it is indispensable in forming a prosperous and vigorous society with elderly people living longer, maintaining and improving their activities of daily life (ADLs) and quality of life (QOL) and maintaining their ability to walk independently. Rehabilitation related to the ankle is of interest because (1) although several studies have addressed lower limb issues such as ankle control and improvement [
4,
11], fewer studies have examined BCI, except as related to the upper limbs; and (2) generally speaking, gait is a universal behavior involving the ankle, which is an important structure supporting body weight. Consequently, constructing ankle rehabilitation devices is a crucially important issue related to BCI.
For actual rehabilitation purposes, the time allocated to feature extraction should be short, possibly with completion in hours. Existing learning algorithms [
5,
6,
7,
8] usually take several weeks to months for machine learning calculations to extract characteristic EEG patterns from a specific frequency range.
Therefore, the goal of this study is to construct a neurorehabilitation system to extract motor intentions rapidly. We developed two methods: (1) heuristic BCI using fuzzy template matching (FTM), because the process quickly and simply extracts characteristic EEG patterns, and (2) ankle rehabilitation devices (ARDs) that change the load automatically according to the degree of contracture. Two elements constitute the ankle neurorehabilitation system (ANS).
This report describes the development of an ANS prototype: (1) an overview of the heuristic BCI system, (2) design and control of the ARD, and (3) an overview of the entire system comprising these two components. Finally, we present results of actual experiments conducted while operating the system.
2. Heuristic BCI Using L-FTM
In this chapter, we describe fuzzy template matching based on simplified fuzzy reasoning, which detects motion intention from measured EEG signals.
2.1. Measurement of EEG Signal
We used a portable, multichannel, biopotential measurement system (Active Two, Biosemi, Amsterdam, Netherlands) to record the EEG signals. A participant wears a head cap, with Ag-AgCl active electrodes connected to an amplifier with which EEG signals from the brain are measured from the scalp. The head cap with 16-pin active electrodes was set according to the standard international 10–20 system. The reference common-mode-sense (CMS) electrode and the driven-right-leg (DRL) electrode were located in the parietal region. The task was to use imagery of the right ankle at 30° dorsiflexion. The EEG signals during the task and during a resting period were detected, then sent to the amplifier, where they were A/D converted and sent through a USB receiver (via optic fiber) to a PC, where data were processed using LabVIEW 2015 software (National Instruments Corp., Austin, TX, USA). The sampling frequency was 2048 Hz. For signal processing, fast Fourier transform (FFT) was applied to the EEG signal detected from each electrode. The sum of the power values in the range of the selected frequency band was obtained and used as the input value. For the two frequency bands selected as input, we used α-band (8–13 Hz) and β-band (14–50 Hz); this is a wider range definition than the conventional one.
2.2. Heuristic BCI Using L-FTM
The heuristic BCI we developed for detection of motor intention uses learning-type fuzzy template matching (L-FTM) [
12] based on simplified fuzzy reasoning. The flow of the heuristic BCI process is presented in
Figure 1.
2.2.1. Fuzzy Template Matching (FTM)
This study uses FTM [
12], a method of template matching based on simplified fuzzy reasoning. Each fuzzy template was constructed with two fuzzy labels of “high” and “low” in an antecedent clause of a fuzzy rule. As shown in
Figure 2, the 2
16 rule is constructed from 16 inputs based on the number of EEG measurement channels (8 electrodes, 2 frequencies), and 2 inputs based on the number of fuzzy labels. For instance, a template can be composed of inputs having features of different types such as “EEG power of Oz measurement site in the α wave frequency band” and “amplitude of the surface electromyography (EMG) signal of the lower right limb.” This approach, which reduces the computational complexity of the heuristic BCI, can classify EEG signals quickly into several categories linked respectively to specific cognitive states.
Each membership function in a rule (template) is constructed by calculating the power of each channel (microV2) and finding maximum and minimum values. The domain of membership function was ascertained from the maximum and minimum values of the power. The shape of the membership function is a symmetrical triangle, where the peak turns to 1 located at the middle of the domain and the bottom turns to zero [
13,
14].
The EEG power was input to the membership function of the defined fuzzy label of the
jth input of the
ith fuzzy rule. The multiplied outputs of all membership functions of the
ith fuzzy rule,
Gj, are obtained as the compatibility degree of the
ith fuzzy rule,
µi, which is expressed as Equation (1):
Output value
Z was calculated as the weighted average of the consequent values of all rules. Here,
Zi is the consequent value of the
ith rule, which is described as Equation (2):
The output value corresponds to the relative compatibility degree during motor imagery.
2.2.2. Pruning Fuzzy Rules
Fuzzy rule sets were automatically generated from EEG feature patterns from combinations of input and fuzzy labels such as “high” and “low.” Rules with a high degree of compatibility for the EEG during both “task” and “nontask” status were expected to be included. Such fuzzy rules compatible with both states reduce the identification accuracy. To avoid this adverse effect, we implemented “pruning” [
15].
The pruning process deletes inadequate fuzzy rules with a high degree of compatibility for both states. We retained only rules with a high degree of compatibility for either a task state or a nontask state. The maximum value of the compatibility degree of the
ith fuzzy rule during the task situation was calculated as
Ob(i), which is described as Equation (3):
where
t is the time of input data,
te is the last time, and
Ot(i;t) and
On(i;t) are the degree of compatibility of the
ith rule at time t during the task and nontask status, respectively. Then, the difference between the total value of each degree of compatibility is calculated. Its absolute value is
Os(i), as presented in Equation (4):
If the value of
Os(i) normalized with
Os(i) is lower than the adequately set threshold, then the
ith rule is judged to have a high degree of compatibility in both states and is deleted. If the difference is large, then the
ith rule is judged to have a high degree of compatibility only for task or nontask status. The rule was retained as shown in Equation (5):
Using only retained subsets of templates (rules), the learning process was undertaken with input values of those stored for pruning. The consequent values of the subset of rules (templates) were set by learning processes. The new template set increases the accuracy of identification of the two states, as shown in
Figure 3.
Consequent values were set up by the learning process [
15,
16]. Using the steepest descent method,
Z is approximated to the target value of the teacher signal
T in Equation (6).
Zi’ is the consequent value before updating, which differs from
Zi. Here,
is a learning coefficient, as shown below:
The teacher signal is allowed to be any value if the values can be linked to several distinct states. Initially, 0 was set as the resting state of the EEG and five as a label for the EEG during foot movement. In the learning process, consequent values of rules with a high degree of compatibility for a certain state are modified to the teacher signal corresponding to the state, thereby producing effective rules (templates) for recognition of the specific status. Recognized EEG features during a cognitive task are then extracted from the rule set consisting of the combination of labels (a sort of searching process).
The output signal is binary sorted through a thresholding function. The threshold is set according to the level of the power of the task and the resting state (Figure 10). When the output value of FTM surpasses the threshold, a trigger is generated, which is transmitted to the ankle rehabilitation device (ARD).
4. Ankle Neurorehabilitation System (ANS)
4.1. System Setup
Figure 9 shows the ankle neurorehabilitation system. This system consists of three subsystems: EEG measurement system, heuristic BCI system of learning fuzzy template matching (L-FTM), and the ARD. First, the EEG signals were detected, then they were signal-processed and passed to the L-FTM algorithm. The motor intention was extracted by the algorithm output signals, which were passed to the actual ARD, where the rehabilitation training was conducted.
The task was to use motor imagery of the right ankle at 30°. EEG signals produced during the task and a resting period were detected. The EEG signals were sent to the amplifier, where A/D conversion was done, then through a USB receiver (via optical fiber) to a PC for data processing. The sampling frequency was 2048 Hz. Then, in the learning process of the heuristic BCI, the values found using MATLAB 2018 software (Mathworks Inc., Natick, MA, USA) were applied to elucidate the task and nontask status. Here, the learning coefficient was set to 0.9, and the teacher signal was set to five in the task condition and 0 in the nontask condition. When the heuristic BCI-processed EEG signal surpassed the threshold, a triggering signal was generated and sent to a data acquisition device (USB-6000, National Instruments Corp., Austin, TX, USA). It was also sent by the trigger sending device (sampling frequency 2048 Hz; MARQ, Kissei Comtec Co. Ltd., Matsumoto, Japan) to the trigger receiver. This receiver was connected to another PC, where LabVIEW 2015 software processes controlled the triggering signals. Here, the triggering signal sent commands to where the rehabilitation device operates the pull up by 30° for a duration of 20 s. The thresholds, set by direct visualization, were initially set lower to detect motor imagery signals, but were later refined to exclude weaker L-FTM signals of resting EEG. The rehabilitation device is activated by air pressure generated by a compressed air supply through a compressor (YC-4, Yaezaki-Kuatsu Co., Ltd., Tokyo, Japan) to the electropneumatic regulator (RTR-200-1, Koganei Corp., Tokyo, Japan). Control signals are sent through an I/O board (MF634, Humusoft Ltd., Praha, Czech). The rotation angle is measured using a potentiometer (SVO1, Murata Manufacturing Co. Ltd., Kyoto, Japan). Data from the potentiometer are sent through the I/O board to the PC.
4.2. Experimental Procedure
Five volunteers (22–24 years old, all healthy male students) participated in the experiment. The experiment followed the Kwansei Gakuin University regulations of ethics for the Protection of Human Subjects of Medical Research, which was approved by the campus committee. Informed consent was obtained from each participant. The study protocol conforms to the Declaration of Helsinki. Each participant was seated on a chair in a relaxed position with the right leg on the footrest with the ankle bent in a natural position. The left leg remained on the floor. EEG signals were detected using the Active Two system with eight electrodes (F3, F4, C3, Cz, C4, P3, Pz, and P4) based on the conventional 10–20 method. Only alpha (8–12 Hz) and beta (13–50 Hz) bandwidths were used for the heuristic BCI using L-FTM detection.
4.3. Experimental Protocol
Using the system explained above, we first verified that the signal of motor imagery of ankle uplift movement was indeed detected, which in turn operated the machinery. The experiment was conducted in three stages: (1) develop and practice motor imagery for the participant, (2) tune up the system to determine the parameters of motor imagery EEG, and (3) test motor imagery EEG to use the device to activate machinery.
For motor imagery development, voluntary upward movement of the right ankle was done first (voluntary task: 30 trials). (I) Motor imagery of the voluntary movement immediately followed (imagery task: 30 trials). The machine produced autonomous ankle movements simulating human rehabilitation behavior (autonomous movement task: 30 trials). Each trial was initiated by a trigger LED and each consisted of 4 s periods. EEG was done during this session. (II) The second process of system tune-up consisted of two substeps during which (a) EEG signals of the motor imagery were detected (2 min) (for comparison, a baseline EEG signal was measured during the resting period (2 min)), and (b) the heuristic BCI system using the L-FTM algorithm was applied to ascertain parameters to discriminate the two states of motor imagery and resting EEG. The FTM calculation took 60 s. (III) Finally, as the third step, the heuristic BCI system using L-FTM logic was tested by detecting the real motor imagery EEG. A trigger signal was generated as the output FTM signal. As soon as the system detected the motor imagery EEG, the triggering signal was presumed to be generated, which was to be transferred to the robot system. To verify this point, two tasks took place: participants were asked to (1) employ motor imagery when the LED light was illuminated, and (2) rest in order to test the resting EEG signal.
To analyze the brain activity tendency, independent component analysis was carried out using EEGLAB [
20]. The EEG signal was first notch-filtered (60 ± 0.5 Hz and 120 ± 0.5 Hz bandwidth) to exclude artifacts from the electrical current, followed by band-pass filtering with frequencies of 0.04–200 Hz (finite impulse response (FIR) filtering). Special care was taken to minimize the notch filter artifacts when determining optimal bandwidth.
5. Results
Figure 10 shows the heuristic BCI outputs and the system thresholds, triggering position, and rotation angle. When the output value of the heuristic BCI surpassed the threshold, a trigger was generated and the rehabilitation device was operated. It is noteworthy that the system was programmed such that the rotation angle was kept constant for 20 s once the trigger was generated. Extra triggers during this period were ignored. After the 20 s period, another trigger can operate the system.
The motor imagery effect on the EEG was compared with the control condition in which the machine moves the leg autonomously with the heuristic BCI algorithm [
15,
16]. Each column in
Figure 10 shows independent component analysis (ICA) [
21,
22], where the relative activation area is highlighted in red as compared with a less activated area in blue and an average one in green. ICA results indicate that the overall EEG power was stronger during the test period, when the ankle rehabilitation system using heuristic BCI was used during motor imagery (
Figure 11a) compared to the period with autonomous movement (
Figure 11b). Results demonstrate that the activity around Cz was more conspicuous for movement with motor imagery. This result shows that the brain signals of the EEG increased with motor imagery, which was used to operate the rehabilitation system.
The additional average value of brain activity of the Cz channel is shown at rest, during voluntary movement, during passive movement, and during neurorehabilitation in the bar graph of
Figure 12. Two-sided
t-test for each task compared to rest showed a significant difference of more than 5% in all tasks.
6. Conclusions
For this study, we developed an integrated neurorehabilitation system incorporating the human brain and a machine through a computer processing algorithm of the heuristic BCI using L-FTM. An important benefit of the system is that it allows detection of motor intention extremely quickly, within an hour, and requires no training of participants.
- -
The heuristic BCI was developed to identify EEG characteristics based on the motion intention (image) and to confirm that the trigger signal can be output.
The most peculiar aspect of the heuristic BCI is its robustness in classifying the two states of EEG brain signals: the task state and the nontask state of brain activity. Its fuzzy characteristic is where the algorithm does not yield the perfect answer, but instead gives a rather crude (quick and dirty) answer within a short time. Here, frequencies of two kinds, alpha and beta bandwidths, were chosen, which markedly limits the inquiry search space, and therefore yielded a significantly short search time. An instant answer of 65,536 rules was obtained almost on time.
- -
Switching the control of position and force is realized automatically according to the patient’s degree of joint contracture.
This result shows that for patients with contracture, rehabilitation training can be performed on their own without the need for presetting by therapists.
- -
Using the system, activation of brain activity near the Cz channel was observed. The rehabilitation effect was observed.
This result indicates that the motor area is activated, and it is considered that rehabilitation strongly connecting the motor intention and the rehabilitation motion is performed.
As the next stage of research, we will evaluate the effectiveness of this system by many brain-damaged patients undergoing rehabilitation. In addition, we will review the frequency band, measurement channels, etc., so that the heuristic BCI can be further optimized and shortened.