1. Introduction
Surface electromyography (sEMG) has been well recognized as an effective tool to generate control commands for devices and human-assisting manipulators (such as exoskeletons and prostheses). In human–computer interaction controls, the most important feature of bioelectrical signals is that they can directly reflect the user’s intentions and muscle activities. The interactive device must be adapted to the articulation and consistent with the operator’s movement. In recent years, the sEMG proportional control method has been gradually developed on the basis of pattern recognition. This method requires that the joint motion angle is continuously recognized, and the quality of the human–computer interaction is largely limited by the accuracy of recognition [
1]. In previous studies, many scholars have discussed classification algorithms and tried various methods to improve the control accuracy of sEMG [
2]. However, there are some interrelated factors that can have an important impact on the control process because of differences in tasks or operators. These factors are rarely noticed by people, such as the direction of movement, concentration of attention or proprioception.
Most daily autonomous movements depend on a continuous monitoring of muscle contraction based on proprioceptive and exteroceptive sensory feedback [
3]. This feedback allows continuous readjustments and corrections of muscle contraction according to the purpose of the action, and is transmitted to the central nervous system to ensure the precise position of the limb. First of all, vision is an important channel to feel the outside, and motion control has obvious visual reliance. Vision can guide the attention of the individual. Therefore, the study of visual feedback can focus on an analysis of the effects of motion control from an external and internal focus of attention. In addition, assigning a certain level of attention or ontological attention can increase the stability of motion [
4]. Some studies have shown that the focus of attention has an important impact on action execution [
5]. Internal focus of attention focuses on one’s action itself, and external focus of attention pays attention to the efficiency of action. Vance et al. [
6] demonstrated for the first time that the sEMG activity was different under internal and external focus of attention conditions, and their study showed that surface electromyographic activity decreased when subjects used an external focus of attention. It also showed that changes in attention may make interactive control more natural and more efficient in motion mode. In addition, proprioception represents some neurosensory receptors, which do not have visual feedback, including limb location and motor sensation, power sensation, and balance sensation [
7]. Zhang and Luo [
8] used closed eye tests to show that the effects of proprioception and visual feedback are different in the stability of human motion. Therefore, the influence of proprioception is an important factor that needs to be tested, in order to reduce the fluctuation of muscle force, maintain the stability of exercise and improve the accuracy of sEMG control of upper limbs. In previous studies, most authors have discussed how attention and proprioception maintain posture [
9] or enhance motor skills learning [
10]. These studies focused on the outcome of motion, and their methods were mostly to discuss the characteristics of muscles during sustained isometric contraction. Few scholars have studied the motion process, and have not discussed how to improve the efficiency of human–computer interactions and the accuracy of continuous proportional control by selecting these factors. In continuous motion control, since the mode of action between muscles changes momentarily, it is necessary to ensure the coordination and stability of muscles throughout a motion.
The purpose of this study is to find out the effects of visual focus of attention and proprioception on the prediction of elbow joint angle by the sEMG signal through experiments and analysis. From the root point of view, this is a problem that solves the ideal joint angle output from the perspective of the relationship between psychological factors and motion control. In fact, whether brain activity and muscle activity are functionally related has been noticed by many scholars. Conway et al. [
11] found that there is a positive correlation between the EMG signal and the activity of the brain motor cortex 15–30 Hz by coherent analysis, which proved that muscle activity is controlled from the brain (β rhythm). Brown et al. [
12] found coherence in the low gamma band (>30 Hz) between the motor cortex and muscle. At the same time, changes in the brain’s motor cortex during visual information processing are also around the 20 Hz band [
13]. Therefore, people can adjust their motions through visual information. In this paper, the coherence between the electroencephalogram (EEG) and the sEMG signal was used to analyze the influence of three factors on the recognition accuracy. Through coherence analysis, the functional association between sEMG and EEG recorded during muscle contraction was discovered, and the different responses of the brain to motion were further investigated. Studying the influence of three psychological factors on the performance of the action not only provides a new perspective on the automatic control of the motion system, but also has important significance for the improvement of the action. Our study set three psychological factors experimentally in two directions. The features of the sEMG signal were extracted to predict the elbow joint angle, and the accuracy of the prediction was examined and analyzed. To our knowledge, to date no study has recognized the impact of these psychological factors on continuous motion control. The methods and results presented here can be used as a basis for optimizing the sEMG control method, so that an interactive device can be controlled in accordance with the human’s intention.
3. Experiment Protocol
3.1. Experiment Apparatus
The sEMG signals during experiments were collected, amplified and transmitted by a ten-channel digital sEMG system (FlexComp Infiniti System, Thought Technology Ltd., Montreal, QC, Canada). The sensors (MyoScan sensor), which connected with electrodes, could record the sEMG signals up to 1600 micro-volts (µV) and an active range from 20 to 500 Hz. An electrogoniometer was used for measuring the elbow rotate joint angles. It has a voltage range of ±2 V, which correspond to a complete 360° range of motion. All data were collected at 1024 Hz. The raw sEMG signals and the elbow angle measurements were transferred to a personal computer through a serial interface (USB) by setting the system to offline mode. EEG data were collected using the ActiveTwo64 channel EEG system from BioSemi, the Netherlands. According to the position defined by the international 10–20 standard electrode placement method, the data of C3 and P3 channels were collected, and the sampling frequency was also 1024 Hz.
When the signal intensity of the target muscle is relatively low, crosstalk is more likely to occur between the signals of other muscles. This also directly affects the contraction process of the muscles. Therefore, in many studies, loads with a constant weight were used to enhance the output power of the target muscle in order to improve the interaction efficiency. For example, Tang et al. [
24] compared the effects of several loads on elbow joint angle recognition. For upper limb joints, if the load is held by hand, it is difficult to ensure constant force during a motion. Therefore, the experimental equipment was set to have a constant resistance, allowing the subject to perform isotonic motion throughout the experiment. The self-made wearable device is shown in
Figure 2 to ensure that the elbow joint maintains constant resistance during the experiment. On the day before the experiment, the mean value of the maximum torque of the elbow joint of each subject was tested. We set the constant resistance to 20% of the maximum torque, which is the resistance value that is easy to maintain normal operation and not easy to fatigue. For identification of the elbow angle, the four muscles of arm were used as target muscles, including brachioradialis (BR), anconeus (AN), biceps brachii (BB) and triceps brachii (TB). By adjusting these muscle forces, the elbow angle could be arbitrarily controlled [
25]. The sEMG signal of the target muscle was used to predict the angle of the elbow joint from 0° to 90°.
3.2. Experiment Procedure
Eight (4 male and 4 female) healthy adult subjects without any previous history of elbow injury volunteered for this study. The mean (±standard deviation) age, height and mass of the subjects were 26.2 (±3.6) years, 170.2 (±9.1) cm and 65.8 (±13.4) kg, respectively. All subjects were right-hand dominant, and had good health during the experiment. Within 24 h before the experiment, the subjects did not carry out any vigorous exercise. All subjects had previously participated in similar motor control studies. Prior to beginning the experimental session, subjects received a written description of the project and signed informed consent forms. Subjects were then asked to perform a series of familiarization trials. Because the EEG signal needs to be acquired at the same time, the subject was restricted from head movement from preparation to exercise, avoiding swallowing and eye blinking, and performing slow eye movement and facial muscle movement.
Proprioception is the feeling that moving organs, such as muscles, tendons, and joints, appear on their own during exercise or at rest. For example, a person can sense the position of various parts of the body when closing his eyes. Therefore, our experiment required that the subject felt the movement angle of the elbow joint with subjective feeling under the proprioceptive condition, and there was no visual feedback. The subjects were asked to close their eyes slightly and focus on the motion of the elbow joint. The experimental setup of the internal and external attention conditions required significant visual feedback. In the external focus of attention experiment, the subjects were asked to focus on a display screen (1280 × 800 px) with a front distance of 1 m. The sEMG signal and joint angle waveforms, which were the effect of motion, were displayed on the screen. In the internal focus of attention experiment, the subjects were asked to visually notice their elbow joint motions. According to different psychological factors and motion directions, data from six experiments were collected, as shown in
Table 1. The three psychological factors included external visual focus of attention (EVA), internal visual focus of attention (IVA), and proprioceptive no visual feedback (NVF). The two motion directions included a horizontal direction (HD) and a vertical direction (VD). In all experiments, the subject performed flexion and extension of the right elbow joint. The angle of the elbow joint was the target variable. All experimental exercise times were 5 s. According to the instructions of the experimenter, the subjects performed flexion and extension of the elbow joint as evenly as possible. Subjects were asked to move their elbow from 0° to 90° and then return to the initial position, as shown in
Figure 3. In the horizontal direction experiment, the subjects performed elbow rotation while maintaining the shoulder joint at 90°. The starting point of motion was the horizontal extension of the subject’s arm. In the vertical direction experiment, the initial posture was that the upper arm was perpendicular to the ground. A minimum of 2 min of rest was given between each motion in order to mitigate the accumulation of fatigue. Several non-standard questions were placed in each experiment to ensure the validity of the experiment. The questions centered around the attention of the subject and the performance of the sensory variables. After confirming that the subjects were not affected by other non-predetermined variables, we marked five valid data points for subsequent processing in each subject’s effective experiment.
4. Results
The sEMG signals from the four muscles were successfully measured and used for controlling the elbow.
Figure 4 shows the predicted error curve of a subject. It can be seen that the model predictive angle curve performs a good regression on the actual motion angle curve, and it is feasible to use the sEMG signal as the control signal of interactive devices. The regression curve shows that there is still some error in the local point during the motion, especially at the starting point and the inflection point of flexion and extension. This may be because the muscle response at these points is delayed or a large noise is generated. It can be seen from the deviation of the regression curve that the recognition error of motion angle under the internal focus of attention condition is the lowest.
Figure 5 shows the predicted error curve. The curves and shadows indicate which time period errors are greater in the time series of the entire motion process.
Table 2 shows the average RMSE of all subjects under different conditions. It can be seen that the recognition results under the three psychological factors all show a consistent trend in the two directions of motion. In comparison, the recognition results in the proprioceptive condition experiment are preferably in the two directions of motion. Among them, the highest recognition accuracy is the proprioception experiment in the vertical direction (the RMSE mean is 6.8061°). The mean RMSE in the external focus of attention condition experiment was 1.81° and 1.34° lower than the internal focus of attention condition experiment, which means that the recognition accuracy is higher. The lowest recognition accuracy is the experiment of the internal focus of attention condition in the horizontal direction (the RMSE mean is 11.734°). Comparing the elbow motion in both directions, the angle estimate in the vertical direction is generally better than the horizontal direction, and the RMSE value is maintained between 6–10°. The recognition model has a good predictive effect. The RMSE value in the horizontal direction is between 8–13°. This also shows that the stability of the elbow joint in the vertical direction is higher than the horizontal direction. The two-way analysis of variance showed that both psychological factors and direction of motion had significant effects on RMSE values. This statistical test was normally distributed at the 95% confidence level. Among them, the main effect of psychological factors was significant, F(2, 84) = 96.87,
p < 0.001. The main effect of the direction of motion was significant, F(1, 24) = 280.48,
p < 0.001. However, there was no significant interaction between the two factors, F(2, 84) = 1.75,
p > 0.05.
From the average values of the coherence coefficients of all the experiments in
Table 3, there was no coherence between the sEMG and EEG signals under the proprioceptive condition, and coherence was observed under both the internal and external conditions. The largest value of the coherence coefficient is the C3 channel of EEG and the biceps channel of sEMG (0.2285) in the proprioception experiment in vertical motion. The smallest is the P3 channel in the proprioception experiment and the biceps channel (0.0919) in horizontal motion. The coefficient values between the biceps and triceps were compared, and the difference is not obvious because they participate in exercise as active and antagonist muscles throughout the flexion and extension of the elbow joint.
Figure 6 plots the EEG–sEMG coherence waveforms for three factors in various bands of the C3 and biceps channels in vertical motion. It can be observed that the frequency bands in which the coefficient values increase significantly are concentrated at 16–25 Hz. It has coherence in the local frequency band above 25 Hz. Similar results were observed in horizontal exercise, indicating that the EEG signal produced a certain degree of change in the attention experiment with visual feedback.
5. Discussion
This paper showed that the accuracy of angle prediction can be affected by some psychological factors. The basis for this is that the elbow joint angle can be estimated by the sEMG signal during continuous motion using a classification or regression algorithm. For the elbow joint motion control based on the sEMG signal, at present, the classification recognition error using the motion type or the motion direction is relatively high. However, the features of the sEMG signal are converted to joint angle values through the regression method and then directly applied to the exoskeleton actuator, which is more able to ensure continuous matching of human–machine motion. This also effectively compensates for the inflexibility of classification recognition. Although there is still a large error in angular regression, we have found that these errors can be appropriately reduced under some psychological conditions. The results of this paper can be used as a basis for more optimal sEMG control.
Many people intuitively believe that visual feedback is necessary to produce a constant level of force and believe that providing more frequent visual feedback can contribute to a reduction in motion variation. However, many results also show that the maintenance of exercise stability can be promoted by proprioceptive information [
26]. As in our experiments, subjects in the blink of an eye can better move the elbow joint from 0° to 90° based on visual cues. It is also possible to achieve this requirement by relying on proprioception when closing the eyes. From the experimental results, the accuracy of elbow joint angle recognition under proprioceptive conditions is higher than that of visual feedback. The wider force fluctuations in the presence of a continuous visual feedback were explained by continuous sensorial information, which possibly induced the subject to a greater number of adjustments and corrections of the force output [
27]. This also leads to a reduction in the degree of automatic movement. Hamilton et al. [
28] showed that the amplitude of force fluctuation, an index of force stability, is also inversely related to the level of effort and to the number of the recruited motor units. The results of Slifkin et al. [
29] also show that the complexity of the force variability (signal noise) is lower without visual feedback. More importantly, there are instances where the visual and proprioceptive senses “overlap”, informing the brain about common conditions of the body and its environment [
30]. However, discrepancies may also arise due to the inherent differences in the filtering characteristics of the two kinds of sensing information and the difference in the coordinate reference will cause the information provided to the brain to be inconsistent, so that the mapped joint angle produces more complex nonlinear calculations [
31]. The visual and proprioceptive sensory organs encode position information in different coordinate frames, so there will always be “disagreement” between visual and proprioceptive feedback of limb position and movement. Proprioceptors inherently encode muscle length and rate of length change, while vision encodes object location in egocentric coordinates. Although in many cases, the sEMG control requires visual feedback to help maintain the accuracy of the output target as much as possible. However, the main concern in our research is not the accuracy of the motion but the accuracy of the angle recognition. This is a study of the stability of motion and the adjustment of compensation actions, reflecting the functional coordination mode of the relevant muscles. Slifkin et al. [
29] concluded that different types of visual feedback are inversely related to the stability of muscle force. The proprioception mainly enhances the more natural dynamic transformation between motions.
For the two attention conditions under visual feedback, it can be found that the selection of attention factors can lead to different action effects, and the accuracy of elbow joint recognition can be improved when the external visual focus of attention is adopted. This shows that the external visual focus of attention leads to better motion stability than the internal visual focus of attention. Some studies have shown that the external visual focus of attention can enhance the balance of exercise [
32]. Wulf et al. [
33] considered that subjects’ motions had higher frequency adjustments in an external focus of attention. High-frequency motion adjustment is seen as a more automatic control and is a reflection mode based on a more rapid and detailed motion response. Cheng [
34] argued that the automated process does not require effort, but rather, it uses a more efficient information processing strategy and therefore does not require a limited processing capacity. In other words, external visual focus of attention reduces the amount of attention required for action operations, reducing the cost of the task at the cognitive level. Our experimental results also show that external visual focus of attention was more conducive to the execution of elbow joint motion, making it more fluent and regular. In contrast, the limitations of internal visual focus of attention not only come from the subject’s movements themselves, but also the interaction of other muscles, even the entire motor system. The internal visual focus of attention is given to larger sEMG activity, which may add noise to the motion system, and the effectiveness of the motion is limited and hindered, making the motion output unstable.
The coherence calculation results show that the coherence coefficient value is higher under the attention condition, which may have two reasons. First, in the case of increased attention, the degree of excitation of the corresponding brain region neurons increases. The increase in coherence is due to an increased connection between the parietal lobe and the motor zone. The central nervous system accomplishes the stretching of the upper limbs by controlling the synergy of the muscles. Another reason may be that the motion control mode is changed as visual information is added. Cooperation of multiple sensory systems is required for flexion and extension of the elbow joint, and the specific information exchange between the visual system and the motion system is maintained during the motion. Under the proprioceptive condition, there was almost no coherence between the two signals, and no crosstalk occurs between them. In combination with recognition accuracy, the automatic characteristics of motion and the flexibility of motion were not limited under the proprioceptive condition.
6. Conclusions
Although there is still a large error in the accuracy of the regression of the elbow joint angle, we believe that pattern recognition based sEMG proportional control will likely be implemented and become more stable in the near future. The results of this study provide an important perspective on how to improve human–machine interaction performance by choosing proprioception and two visual focuses of attention. We have experimentally proved that using proprioception or adding external visual focus of attention conditions in motion, the recognition accuracy of the joint angle was improved, and the operator’s stability and balance of motion control were enhanced. Future work will include the following two aspects. First, as the attention to the sensory signal is accompanied by a neuropsychological processing process, the EEG signals can be integrated to improve the control accuracy. Secondly, through the verification of this paper, it was found that the use of traditional pattern recognition algorithms, including BPNN, SVM and RF, have certain limitations in the accuracy of sEMG-based angle regression. In order to reduce the recognition error, we hope to study the sEMG array data from more channels of biosignal acquisition devices, and deep learning algorithms will be used for processing. Finally, interactive device research that can support people’s daily activities is also our future work.