Myoelectric Control for Upper Limb Prostheses

Abstract

State-of-the-art high-end prostheses are electro-mechanically able to provide a great variety of movements. Nevertheless, in order to functionally replace a human limb, it is essential that each movement is properly controlled. This is the goal of prosthesis control, which has become a growing research field in the last decades, with the ultimate goal of reproducing biological limb control. Therefore, exploration and development of prosthesis control are crucial to improve many aspects of an amputee’s life. Nowadays, a large divergence between academia and industry has become evident in commercial systems. Although several studies propose more natural control systems with promising results, basic one degree of freedom (DoF), a control switching system is the most widely used option in industry because of simplicity, robustness and inertia. A few classification controlled prostheses have emerged in the last years but they are still a low percentage of the used ones. One of the factors that generate this situation is the lack of robustness of more advanced control algorithms in daily life activities outside of laboratory conditions. Because of this, research has shifted towards more functional prosthesis control. This work reviews the most recent literature in upper limb prosthetic control. It covers commonly used variants of possible biological inputs, its processing and translation to actual control, mostly focusing on electromyograms as well as the problems it will have to overcome in near future.

1. Introduction

In the USA alone, each year around 158,000 persons undergo amputations [1]. In 2005, 1.6 million persons were living in the USA with the loss of a limb, with a 3.6 million prediction by the year 2050 [2]. For amputees, the use of artificial limbs (prostheses) is vital for their quality of life. Unfortunately, rejection and non-functional use has been traditionally high, specially in upper-limb amputation cases [3,4] and satisfaction with prostheses was reported limited [5].

Upper limb prosthesis control has been a growing research topic in the last decades. Different studies focused on developing robust prosthesis control to help the thousands of people that suffer a major upper-limb loss. Trying to predict the user’s intent from available biological signals has been a challenging problem. Electromyography (EMG) is the most important input source for upper limb prosthesis control. Numerous methods have been tested in the last 15 years presenting different approaches to this problem and leading to a diversified literature. The performance of the different methods strongly depends on the environment and experimental setup, proving the nonexistence of a unique solution for prosthesis control.

Despite the great number of different attempts, not many of the recent, complex and powerful proposals have emerged as a daily functional option. Some companies have developed prostheses using sophisticated classification controllers, but the difference between academia and industry is still one of the most intriguing situations. As a consequence, clinical usability has become a key issue in the research community. The need for more intuitive, natural and robust controls is the main topic in many recent papers [6,7,8,9]. Prostheses have to improve a lot to naturally replace a human limb. The prosthesis rejection decreased to a 20% in the last years [3,10]. This rejection percentage goes up to a 35% for body-powered devices in general [11].

Diverse signal sources were considered for a better estimation of the user’s intent. Non-invasive and invasive methods have been compared in order to establish a preference depending on the user’s condition. Contingent to the physiology of the patient, there are differences in the availability of muscles and signals. For the data acquisition process (see Figure 1), surface EMG signals are the most common ones. They are used to obtain the features in the time or frequency domain that are the basis for the prosthesis control algorithm. Depending on how data is collected and processed, offline and online methods can be used, i.e., learning can be done with prerecorded signals or in real time. The opportunity for the user to adapt during an online experimentation and in real life makes these versions more realistic and accurate.

Another fundamental challenge is the learning model, the second step of the control loop in Figure 1. Commercial systems generally work with a hard-coded control using two channels. One option is that with a co-contraction, the user switches between functions. Furthermore, contractions in different channels generate outputs in the selected function. The other one is where the slope in each channel signal determines the function used. The prosthesis does not have any learning process. Here, the user adjusts the parameters to the best fitting, but the machine is already programmed to work in a specific way. As opposed, machine learning models have to be trained to learn. They know what the goal is, typically, an error cost function to be minimized, and try to learn from the input data. During training, the machine finds the most suitable transformation for the input data to estimate the best output according to the goal set and the model parameters. This is known as machine adaptation; the machine adapts itself to improve its performance using the input information. After training, it is tested in unknown conditions to evaluate its performance. Classification models, which assign the current data to specific movements (a finite length alphabet of allowed movements) were the most common models in past research. Seeking for a more natural and smooth control, researchers are recently implementing regression models with high performance. This allows a more continuous control of the model’s output.

To complete the closed loop structure of Figure 1, the last element is the feedback channel. The most common feedback is the visual channel, when the user is observing the prosthesis reactions or graphical representations of the model output in virtual tasks. The importance of feedback has been proven as critical in the learning process [12], optimizing the performance if the user is able to predict the prosthesis’ behaviour and to interact with the system. At the beginning, studies were focused only in the machine adaptation process. But recently, researchers found out that the human adaptation is as important and should be studied with the same effort. Human adaptation is the process in which a human learns from the machine’s actions and changes his behavior to achieve the best results [13]. The great impact of the concept of human adaptation in prosthesis control motivated the researchers to further investigate it. Due to this, co-adaptation emerged [14]. The application of this concept, understood as the simultaneous adaptation of machine and user, has shown promising results. Both learners are able to adapt to the other’s response. This helps the control to be more dynamic and accurate in real time, avoiding non-optimal solutions. Another interesting concept developed is transfer learning, which allows previous knowledge from a different task to be used to perform a new task with high accuracy and within a shorter training period [15].

Algorithms achieved peak performance in controlled environments, but adapting these to realistic environments and daily life situations exhibited issues in robustness [16,17]. Many factors, such as changes in arm position [18], small electrode shifts [19], skin conditions [12], mechanical load due to the weight of the prosthesis [20] or time between algorithm training and application [21] can impact the reliability and contribute to the limited usage of more sophisticated prosthesis control methods in final users. Trying to deal with this, a new research area focused on clinical usability has been initiated. Studying i.e., sensor shifting effects or long-term use, helps to improve the prosthesis’ performance in the real world. Furthermore, testing in real prostheses has become more regular, being aware of the need to analyze the theoretical progress in real environments [22,23].

This work presents an overview of literature related to upper limb prosthesis control. We seek to clarify the different methods that have been used by research laboratories around the world. This review made it possible to identify the large number of alternatives used for prosthesis control in the literature. These were analyzed in depth, in order to present their advantages and disadvantages. In the next sections, we review in detail, each of the blocks in Figure 1 and in the last section, we discuss the future challenges in myoelectric upper limb prosthetic control.

2. Data Acquisition

2.1. Input Source

Decoding information transmitted from the brain to the muscles is a complicate task. The access to it can be performed invasively or noninvasively and can be seen as a straight forward procedure, but its decoding, interpretation and usage as control input for a prosthesis are more challenging. In principle, an expectation of a human’s movement intention can usually be extracted from different stages of the transmission.

It is possible to access signals directly from the brain, using i.e., electroencephalography (EEG) [24,25,26,27]. Ganguly et al. [28] were able to generate a stable cortical map for prosthetic function. The problem is to create a stable neural interface that remains unchanged over time. Due to the plasticity of cortical circuits, neural representations for natural movements have been proven unstable, but with long-time use of prosthetic control, a stable map can be created. This representation can persist over time even with the addition of other cortical maps. Finding a stable EEG feature is key to a prosthesis brain control. Galan et al. [29] controlled a wheelchair after users were able to perform stable EEG features that maximize separability between tasks. Nevertheless, the data acquisition process, as well as its necessary hardware are still not suitable for daily use.

Another option is to access it directly from activated muscles using EMGs. Nowadays, these are the most commonly used signals for upper limb prosthesis control and they have been utilized since the 1940s [30,31,32]. EMG measures electrical potentials generated in a muscle during its contraction representing neuromuscular activities. Therefore, they contain information about the neural signal sent to attempt a specific movement. Because of its easy access and the available information, EMGs are the first option for prosthesis control. They can be recorded placing non invasive superficial electrodes on the skin of the stump or other active muscles. Working with EMGs as inputs for user-intent estimation models has trained systems with excellent performance [6,7,16,33,34,35,36]. A deeper study on EMGs was developed by Sartori et al. [37], in which they generated a biomimetic model-based decoder that synthesizes the dynamics of the musculo-skeletal system controlled by the residual EMG measured. Additionally, it is possible to extract the central nervous system (CNS) “force functions” of muscular synergies from EMG recordings, which represent the Degrees of Freedom (DoFs) control signals that models have to estimate.

In addition, if the availability of active muscles is limited, Targeted Muscle Re-innervation (TMR) [38,39,40,41,42], is an effective procedure to generate meaningful inputs and to treat phantom limb pain [43]. With TMR, residual nerves are surgically transferred to alternative muscles where surface EMGs can be recorded. After successful reinnervations, the new muscles contract on motor commands sent to the lost limb. Thus new hot spots for intuitive EMG driven prostheses are created [44]. The experiments on patients who underwent TMR surgery prove that the nerve activity is amplified to an acceptable amplitude level. Using this it is possible to develop a control with the generated EMGs as good as the systems with biologically natural EMGs. This situation is more frequent in patients with a high level of amputation like transhumeral or shoulder-disarticulation amputations were some nerves are no longer bio-mechanically useful [44]. Moreover, for these individuals, the need to create additional signal sources is particularly large, as a high number of functions need to be replaced while only few natural sources are available. The TMR surgery helps to overcome this paradox and offers these patients a functional use of prostheses.

As the EMG signals are the most suitable and the most common in the literature, we will focus on them from now on.

2.2. Data Amount: Number of Channels and Sampling Frequency

Larger amounts of acquired data contain more information to be used as an input for a prosthesis. Nevertheless, this additional information might not always be valuable. At a certain point, increasing the number of channels in a limited space will provide a lot of redundant, and therefore, useless information. This occurs to be a problem if it increases the computational cost needed to process the data in a reasonable amount of time, consequently decreasing the overall efficiency and performance of the system. Therefore, before performing acquisition, it is necessary to establish the number of channels to record from and, therewith, the amount of data to collect. Some studies use a high-density data acquisition system reaching up to 192 channels of EMG signals [13,45,46,47], but this amount of data is not necessary for high performance. Young et al. [45] tested the effect of the number of channels during EMG data acquisition suggesting that having more than six channels did not reduce the estimation error, therewith, performance did not improve. Hahne et al. [47] performed a similar study and showed that fewer channels were more optimal, due to the lower amounts of training data required. The increase of performance was not significant regarding the computational cost implied and most of the extra information obtained was redundant. The state-of-the-art methods are typically based on four to twelve channels using EMGs [34,48,49,50,51,52]. Within this range, models reach their peak performance with a high efficiency during the data acquisition process.

Since the bandwidth of EMG signals is around 400–500 Hz [53], a typical sampling frequency is 1000 Hz (Nyquist–Shannon sampling theorem) [54,55,56]. Other studies use a lower sampling frequency with similar results. For a comparison of sampling frequencies and performance see [57]. In some commercial devices, the sampling frequency is reduced to 200 Hz in order to capture enough signal information at a lower cost [58].

2.3. Data Segmentation: Sample Size for Feature Extraction

Once the input source has been digitized and the number of channels have been properly set, the EMG signal must be processed in order to use it afterwards to feed the system. Therefore, the next phase consists of segmentation, where input signal is windowed for feature extraction purposes: from all the data, only a specific amount (simple size defined by the window size) will be used to extract the information at each time step as it is sketched in Figure 2. The window is continuously moving accepting new samples. Farrell et al. [59] proposed a maximum delay of 300 ms to avoid unacceptable delays in real-time operations. The optimal window length was between 100 and 125 ms. Finding the trade-off between accuracy and time response is clue to window sizing. Nielsen et al. [60] establish that the system decreases performance with windows smaller than 100 ms.

The second point in data segmentation is the windowing technique. The shape of the window is the rectangular one. With respect to the displacement, there are two options: adjacent windowing (disjoint segments) and overlapped windowing (windows slide over each other, smoothing the feature vector). While variance can also be reduced in non overlapping windowing by using greater windows, they are slower and introduce delays. These delays exacerbate the user experience in real-time operations. Overlapping will reduce these delays, not having to wait for a time set by the window size to generate a new output. At the same time, overlapping uses enough data to not generate high-variance outputs. Phinyonmark et al. [61] tested different overlapping options, suggesting that overlapping does not improve the accuracy of the methods but helps reducing the delay using larger window sizes.

2.4. Feature Extraction

As previously stated, feeding the system directly with myoelectric signals is unpractical due to the randomness and non-stationarity of the inputs. Some recent studies have been working with full EMG signals using a Convolutional Neural Network (CNN) to extract the main information [15], but the common workflow consists of mapping the signals into smaller dimensions, increasing the information density. This method is called feature extraction and resides on the condensation of the relevant information, drafted in Figure 2. This process is critical for the success of any model.

There are three different categories of features for feature extraction: time domain (TD), frequency domain (FD) and time-scale domain (TSD). Oskoei et al. [48] did a deep theoretical study on the different categories and features that can be extracted from EMGs. Time domain features often investigate amplitude and related features of the EMGs, while frequency domain features are more focused on the power spectrum parameters. The use of wavelets falls into time-scale domain features. Time domain features are the most common in myoelectric controls due to their simplicity and since they are computed rapidly. The root mean square (RMS) is an example of TD features that works better with high level contractions following a Gaussian Model. Others, like mean absolute value (MAV), work better with low level contractions (or fatigue effects) using a Laplacian model. Phinyomark et al. [58] studied 26 different, individual features and eight sets of multiple features. With lower sampling frequencies, that can reduce power-consumption and computational costs in a clinical application, the selection of the feature has a critical effect. The effect of dropping the sampling frequency was unavoidable, but signal amplitude or power features incurred less reductions. Features like EMG amplitude estimators, e.g., integrated absolute value (IAV), mean absolute value (MAV), root mean square (RMS), and waveform length (WL)) and power features, e.g., difference absolute mean value (DAMV), difference absolute standard deviation value (DASDV) and mean value of the square root (MSR)) obtained good results.

3. Learning

Extracted features are the input to the learning system. Features will be used to train the model for estimating the user’s intent. It will map the features to an output of various degrees of freedom. The majority of commercial devices are non-learning systems that control one DoF at a time. An activation pattern allows the user to switch between the functions available. While using other patterns, the user can control the output of the prosthesis in the activated function. Which DoF is controlled will depend on the function. Looking for a more realistic and natural control of the output, researchers developed more complex algorithms.

There are two agents that participate in this learning process: the machine and the human. Focusing on the machine learning system first, the models are mostly based on two fundamental approaches: classification and regression. Depending on the final application we want it for, we will use one or the other.

3.1. Classification

A classifier is designed to identify patterns in data and to categorize them. Classifiers recognize patterns in the data generated during the training phase and assign a particular input to the corresponding target motion class during the application phase. The concept can thus be applied to recognizing and separating EMG signals and to relate them to the intention of the user. Several linear and non-linear approaches were investigated in the last decades. Early attempts used a linear approach based on time series parameters that was able to correctly separate classes [62]. Artificial Neural Networks, which are mathematically modelled networks inspired by biological neurons, added the ability to learn the distinction between different conditions in patterns and therewith, the linear and nonlinear relationships directly from the data being classified. Kelly et al. [63] showed that a discrete Hopfield model is capable of generating the same time series parameters as those produced by the conventional sequential least-squares-algorithm with higher computational efficiency. Furthermore, the model could distinguish between four separate arm functions using a two-layer perceptron, although at still high computational costs.

Heretofore, many classifiers have been explored, such as Linear Discriminant Analysis (LDA) [33,64], Gaussian mixture models [65,66], Support Vector Machines (SVM) [48,67,68], Hidden Markov Models (HMM) [69], K-Nearest Neighbors (KNN) [70], Multi-Layer Perceptrons (MLP) [63,71], Quadratic Discriminant Analysis (QDA) [16] and Hyper-Dimensional Computing (HDC) [72].

Currently, it is widely accepted that a simple time-domain feature-set, as proposed by Hudgins [73] in combination with an LDA classifier is sufficient and presents a good balance between classification accuracy and computational usage, as well as robustness towards some non-stationarities [19].

Due to the usage of multiple channels for signal acquisition, the extracted feature vector dimension can become large. In order to overcome problems with dimensionality, feature-reduction (FR) or feature selection (FS) are commonly performed. Using Principal Component Analysis as a method for FR decreases computational costs by projecting the high dimensional feature set into a relatively low dimensional space, still preserving the linearity [74]. FS is achieved with methods such as Sequential Forward Selection [74,75], Genetic Algorithms [76,77], Kohonen’s Self-organizing Map [78] and Particle Swarm Optimization [79]. Moreover, Common Spatial Patterns (CSP), a method generally employed to overcome binary classification problems of EEG signals, has been shown to also improve performance and robustness against noise in EMG pattern recognition [80].

In fact, not many classification methods reached clinical usage, in part due to missing reliability outside of laboratory conditions [12,81,82,83]. All classifiers need thorough training to identify the intention of the user and the high performance levels obtained with the applied techniques often drop when natural variations in the EMG patterns and noise-sources, typical for real-world conditions, are introduced [21,84,85]. These non-stationarities [86,87] may be caused by changed electrode impedances due to sweat or dry skin [12], altered armpositions [18,88], mechanical loads due to the weight of the prosthesis [20,89], small shifts of electrode positioning [19,90] or variations in the user’s contractions. Furthermore, a classifier provides only an estimation about the executed movement but not the level of contraction that is needed to control the velocity or grip force of a prosthesis. To obtain a proportional control, which is clinically important, the discrete signals of the classifier output are combined with a force-estimate [66], achieved by averaging of the amplitude of all EMG channels [12].

3.2. Regression

Regression models do not classify input signals in a discrete set of classes, but approximate continuous multivariate outputs. Classifiers have the disadvantage that only a finite number of pre-trained patterns can be learned. Regressors do not have that handicap and give the user more freedom. A continuous mapping of the output allows a complete control and lots of combinations of values. The user can perform any movement generated by the controlled DoFs activation even if it has not been trained.

Classification has other limitations. The need of re-training for the non-stationary EMGs is one of them. As mentioned before, small changes in the EMG signals, e.g., fatigue, electrode shifting or sweat, are not well handled by classifiers. The user is more able to adapt to these effects in regression [91]. Small changes in the input signal can generate small variations in the prediction, which could lead to a missclassification. A small variation in the prediction on a regressor is better handled due to its continuity. There are no abrupt changes as class-boundaries, so the user can directly react and better compensate for potential errors in the estimation.

In summary, regression models include the control for all DoFs simultaneously, independent and proportional, generating a smoother and more natural behavior of the prosthesis [35]. Therefore, a large amount of different motions could be used for prosthesis control. But for now, only two to three DoFs can be reliably controlled [14,46,47,92,93,94,95,96,97,98]. Regression allows the user to skip the separate and sequential control of different DoFs that classification proposes.

Hahne et al. [47] compared different linear and non linear regression techniques for two DoFs control. These techniques include linear regression (LR), mixture of linear experts (ME), multilayer-perceptron, and kernel ridge regression (KRR). Results have shown that KRR outperformed the other regressors. But with a basic linearization in the feature space, simpler regressors as ME or LR were able to perform as well as KRR, showing that simple linear models with the correct features are perfectly suitable for prosthesis control, increasing the efficiency of the model. It was also shown in the study how regressors were able to generalize for DoF combinations, where no training data was provided, proving their robustness against unknown situations for the model.

This more natural control was taken to a realistic manipulation scenario by Strazzulla et al. [99]. They were able to control two robotic arms that had ten independent DoFs between both, using a linear regressor for each DoF. With them they controlled the torque and force for each of the motors. The learning models were based on incremental ridge regression with random Fourier features. They achieved a completion rate of 95% of single-handed tasks and 84% of bimanual tasks.

Another very common regression approach is the support vector machine regression-based (rSVM). Ameri et al. [15] compared this regressor to a new regression convolutional network (rCNN). Regressors showed advantage over previous CNN classification studies facing independent simultaneous control of motions. Furthermore, the ability of the rCNN to extract underlying motor information in the EMG with no need for feature selection was presented as an advantage over rSVM and an option to solve robustness issues.

In regression control, there have been two strategies. One can either control the position [14,91] or the velocity [15,97,100] of movement. Those two approaches were hard coded and fix. Recently, Igual et al. [96] presented an adaptive auto-regressive filter algorithm based on adaptive infinite impulse response filtering theory. This linear method is a generalized algorithm that includes both control protocols and is adjustable to more options. This was the first adaptive strategy to directly learn proportional velocity control. The machine is who learns one protocol or the other, depending on the user’s actions. The studies showed a clear trend to velocity control. This option seems to be more natural to the user and has a number of benefits in practice [101] such as less overall effort and no limitations on range of motion.

3.3. Feedback

Biological myoelectric control does not only consist of the brain activating a muscle in order to control a limb. A fundamental feature of our motor apparatus is a highly efficient sensory feedback system. In a human hand, this feedback carries information about its position, the pressure it is applying on something or even the stretch level of its tendons. Without sensory feedback, a prosthesis will keep being a simple tool with no possibilities of completely replace a missing limb.

Therefore, after generating the data and learning a prediction model, feedback is the last element of the state-of-the-art closed loop structure. Feedback increases the efficiency of the learning process. It is shown that, providing feedback to the user, the movement can be intuitively corrected by him. Problems appearing in the model learning process as estimation errors and poor robustness in changing conditions can be avoided with an appropriate feedback. This also will help to avoid local minimum solutions and give the user the ability to interact with the system when the output is not the desired.

There have been different approaches through time. One way of generating feedback and replacing the affected limb was bilateral mirror training [102]. The user is asked to perform the desired movement with both arms. Nielsen et al. [60] used this to generate data to control the force and torque of the affected limb. Executing the task with the complete limb helps the user to reproduce the movement on the other side while EMG generation is not as easy. In the same study, they also proved that by generating both signals at the same time models in one limb are also suitable for the other limb. They recorded data of the force and torque in the healthy limb and generated a model to control those outputs with the EMGs. Then, they applied that model to the other limb so force and torque could be controlled with its own EMGs. Ameri et al. [94,103] used the same training procedure to also help the user and to mirror the recorded data of the intact limb to the phantom limb.

This feedback method is natural but it is not functional for bilateral amputees, where there is no intact limb to measure from. So, Ameri et al. [95] proposed a virtual visual feedback. With this approach, the system generates a visual representation of the users’ performance. It is essential that the users understand the visual feedback’s meaning. The higher the relationship between the visual representation and the realistic movement, the easier the comprehension for the participants. Virtually showing the users’ output has been the most common feedback used by researchers. Users can control cursors as in Figure 3a [14,24,93,94,95,96,104] or even virtual prostheses. Powell et al. [105] developed a virtual prosthesis that executed the system’s output for the user’s intent. Then, the user was able to compare that to the desired output and try to correct it if necessary.

The next level of visual feedback came with Virtual Reality (VR) (Figure 3b) [13]. This allows the user to train in a realistic environment where the patient can use the prosthesis as it is going to be used in real life. The user sees how the prosthesis will react to the inputs and can adjust the behaviour in a more realistic way, which helps to the user’s acceptance and adaptation.

Another class of feedback is based on mechanical communication. This looks for generating some kind of non visual stimulation in the user. Peerdman et al. [49] developed a study among prosthesis users to find out which were their needs. The need for a proper environmental feedback was one of the main request that users had in common. They found that an implementation of a proper environmental feedback system that helps them control the prosthesis and interact with its surroundings, is extremely important.

On the one side, non-invasive methods, as vibrotactors (Figure 3c), are easily introduced into the prosthesis [106,107]. On the other side, invasive electrodes directly innervating the nerves might simulate human sensory feedback to a much higher degree. Markovic et. al. [108] showed that providing non-invasive feedback in form of vibrations on the remaining stump, subjects were able to scale the force of their prostheses better.

3.4. Human Adaptation

The machine is not the only learner in the control system. Humans have to learn how to use the system in order to generate more stable and consistent signals. Practice makes the human a better user, therefore, everything that helps the human to adapt to the system will increase the system’s performance [109]. Ison et al. [13] proved that these learning skills positively influence the system. Human learning is consistent with the stages of typical motor skill learning for new tasks. First, it requires gathering a lot of information. Then, with repetition, the user starts to understand the task and gets used to it. Finally, the task becomes autonomous. The effects of this human adaptation were also described by Strazzulla et al. [99]. Experienced users (the ones familiar with the experimental setup) needed less time to complete the tasks while naive subjects were much slower. The completion rates were similar for both groups of patients which made the researchers conclude that it was not a machine effect, but a human effect. Expert users were more adapted to the system so had a more efficient behaviour.

The data can be collected offline and used to train the model. Afterwards, the user gets online control creating a closed loop with a fixed algorithm. This online control has shown improvement with respect to its offline counterpart [85,97,110]. They found out that including the user into the learning loop and allowing him to interact in real-time helps to solve problems like arm position change or other non-stationary situations.The real-time feedback the user receives allows him to overcome the impact of interferences and to interact with the system instantaneously. Hahne et al. [91] added noise to the EMG signals to test how regression and classification methods behaved. Results exhibit that in online control experiments the user was more capable to compensate these external disturbances and that this ability was better for regression than for classification, due to the continuous feedback regression offers.

The data can also be collected, and the model trained, online. Therewith, the user becomes part of the closed loop while the algorithm is adapting, giving him the ability to interact with the machine learning process [111,112]. This leads to the concept of co-adaptation.

3.5. Co-Adaptation

Machine and human are capable of learning. But so far, each agent learns for its own interest. Making them co-operate and learn in the behalf of the whole system is what has been recently named as co-adaptation. The different agents have an online learning focusing on the same goal in order to maximize the model performance. In this case there will be two learners: human and machine. Co-adaptation also allows simpler algorithms to obtain high performance [14].

The key factor of co-adaptation is that both agents learn towards a common goal [113]. Knowing the output of the other and how it is working on that goal helps to achieve it. As previously introduced, a proper feedback has to be given to the user in order to generate optimal EMG patterns, which leads in combination with online learning to an optimal control model; in this situation, the inputs for the machine are the signals generated by the human. For the human, the input is the information that represents the machine estimation. Then, they will start a cooperative learning process. During training both are able to adapt continuously. This capability has been ignored for a long time.

Müller et al. [113] modeled the two-learners problem. Overall, there are two different channels of information. First, human to machine, through where the human sends the myoelectric signals, carrying the information of what the user intents. Second, machine to human, through where a programmed feedback system gives the human the information about what the machine is predicting. The human to machine channel has been developed for decades as described above. This channel is the one that has the more complex encoded information. The machine to human channel is used to show the user how his data is being understood and makes him able to correct it. Both sides of the problem have learning parameters that influence the co-adaptive learning process. This two adaptation processes will have an effect on the cost function. Adapting both channels at the same time has been a recent promising discovery for better prosthesis control. The user will minimize the error trying to adapt itself to the machine parameters. The user knows what the machine is understanding about what he is doing. With this information he will search in the signal spectrum the right signals to generate the desired output. At the same time the machine will adapt to the user signals with adaptive algorithms. The adaptation process will continue during the whole training. At the end, both will converge to a common solution. This collaboration allows to avoid convergence to local minima or other problems where the user can not be satisfied with the solution and forces to keep searching a better one. He will be able to try other inputs that will vary the machine’s parameters in order to achieve a better performance.

The need for a good interaction will shift the experimentation to an online form. The benefit is the immediate reaction of one system to the other actor’s adaptation. If some part adapts, the other receives the new information and will adapt itself too, to achieve the common goal. During an offline experiment the agent that is not activated is not able to respond to the adaptation of the other actor and interact with this new information immediately. At the same time the adapting part is not able to see the reaction of the system to its new data. This will make the process slower and maybe converge to a non-optimal solution. Real time reactions combined give more optimal solutions and avoid possible not desired outputs. Some experiments have been done with interleaved recording in short periods but the training is mainly online for co-adaptive systems.

All this is translated to a faster adaptation system that allows the use of simpler models with high accuracy. Igual et al. [96] applied this closed-loop real-time learning scheme. The new co-adaptation strategy presented instantaneous feedback to a real-time learning system reaching a peak performance for the linear regression algorithm. The linear regression gradually improved while the user learned how to control the system. This adaptation also helped to improve the usability for the individuals with congenital limb deficiency. Their results were comparable to that of able bodied subjects. The simplicity of the model and the interaction between the two learners proved to be really efficient and improved the state of the art regression control. The computational time was drastically reduced by switching to less complicated algorithms, and performance was improved in contrast to more complex methods in offline situations. The human agent could detect problems that the machine was not being able to solve, and found a solution for them. The patient pursued other ways to generate the desired pattern until the machine learned it properly.

Couraud et al. [114] generated data to experiment the effects of co-adaptation. A model of human adaptation was used to perform different levels of co-adaptation. The gain parameter determined the speed and the stability of adaptation being a value of 1 a fast adaptation. Low gains reduced the final error but did not performed a complete adaptation. While higher gains helped the adaptation correcting almost all the error in one trial, it increased the errors generated by the added noise. A trade-off between both proposals ended with a variable gain system that combined both options keeping the benefits of both.

4. Usability

While the research and prototype models are continuously upgraded, the transfer into commercial prostheses is still limited. The great majority of prostheses are still fit with simple two-electrode systems without machine learning and only two simple classification-based controllers that do not allow for simultaneous motions are on the market [115,116]. Newer and better models have been developed, but these control schemes do not have a consistent performance in non controlled environments yet. With the first tests of the algorithms in clinical situations a lot of problems that were not being taken into account appeared. So far the models relied on the EMG used to train, but with time, those EMG signals showed a non stationary behaviour. This has been related to electrode shifting [13], fatigue [98], donning/doffing [97], arm position [18,112], etc. Researchers had to overcome this situation to get a decent level of functionality for real life activities with more robust and stable systems, especially when user motivation and emotions must be taken into account.

One of the proposed solutions is the use of re-training or re-calibration [110]. The idea is to add a small amount of data that represents the untrained conditions to the training set. Chen et al. [51] also used this technique. Adding the data used for the test to the training set once the data was labeled correctly during the testing phase, improved the classification process. Increasing the amount of training data helps to deal with unknown inputs. Yeung et al. [117] proposed a new re-training system, in which depending on the new data added to the training set specific old data was erased. This directional forgetting deleted old data that was in the same direction that the new one added. So, this updating process of the direction induced less distortion to each region, while discarding training data that was obsolete.

Re-training is used in order to correct system’s performance degradation. However, with the development of newer and more robust models the re-training needs have been minimized [118]. Nevertheless, on the basis of this concept, the transfer learning protocol was developed [15,119]. Researchers saw that different tasks or movements had some relation between them [120]. After having learned a task or a movement, they realized that there are associated movements that can take advantage of the previous task model. Some tasks are similar to others, so the base of the information is related. This allows to, instead of re-training the entire system for a new move, learn new motion models using the previous data and adding a small amount of new data to overcome the differences. It can also be used when the data had suffered greater changes, so that the fundamental information is the same but the old model does not work. Paassen et al. [119] used the transfer learning concept instead of re-learning a new model, re-using an old version and adapting it to the new situation.

Long term use is also a problem for prosthesis control. Users have to wear the prosthesis during several hours during the day and the control has to be stable. Initially, this was associated with less efficient re-training protocols that were needed frequently [50]. Re-training, however, has been developed into more efficient ones [85,110]. Additionally, models have been developed taking this undesired needs in consideration, trying to avoid them. That is why some studies have focused on the long term use of their models [85,105] achieving models that perform properly during 8 hours. without re-training. These training protocols usually take several days, where in different days the model is trained to learn the non-stationarities that time and fatigue could generate in the EMGs. At the end, users want to re-train as fast (better re-training methods) and as less (long term stability) as possible.

Once the models improve their robustness against the possible non-stationarities of EMG data, it is interesting to see how they perform in real life tasks using prosthesis (Figure 3d). Daily life will challenge prosthesis control to perform tasks that have not been considered in a lab environment. Testing the control in tasks like the Clothespin Relocation Test [118] and holding and grasping objects [66,99]. These tasks represent a more realistic performance of what the end user will experience. Results showed that the academic algorithms that outperformed the commercial ones in lab environments are reducing the gap in usability terms with the industry. Some of them even started to outperform the commercial algorithms in daily life applications.

5. Discussion

We have presented the state of the art in prosthesis control and have given insights into the different elements in the control scheme. Furthermore, we reviewed how novel techniques have not accomplished a stable transition to the market. Therefore, we are going to outline some of the challenges this field will have to overtake in near future.

Hitherto, the ultimate goal to fully replace a human limb is far from being reached and more deep knowledge about neural and behavioral changes that result from amputation is mandatory [121]. However, the field of prosthesis control has been constantly growing for the last decades. Great advantages have been achieved recently and the rate of improvement seems to constantly grow. Despite the major goal of reaching high functionality, some of the advancements in academia have not reached the industry due to a lack of robustness and usability. Thus, some newer proposals, the prosthetic technology could benefit from, are not appropriate for a daily use. Therewith, giving the user a more natural control that comes closer to the way an intact hand is actuated and a better evaluation of device adaptation, utility, and motor learning is to remain an essential objective.

Prosthesis control consist of a bio-signal, a system to process the input, and the translation into actual control of the prosthesis. In the prosthesis control community, there is a huge gap between academia and industry. Commercial prostheses are far from the potential that literature has presented in the last decades.

Some recent classification systems are being used by some prosthetic devices, applying sequential control, which is far from being an accurate implementation of a biological limb behavior. Recently, regression schemes started to propose better solutions [15,47,96] compared to actually used classificators. Overall, the new developed learning models achieve great results in laboratory environments but they do not reach the commercial use. This is due a lack of robustness and overcoming non-stationarities that appear in a real-life use. Reducing this functionality problem has become the focus of the recent years with promising results. Users have to deal with systems that are very distant to offer a natural limb condition with antiquated one DoF control in a switching-between-functions system. New studies are starting to propose more natural controls [25,122,123]. Consequently, regressors need to be improved to develop a robust control in more than two to three DoFs, to be able to be implemented into commercial prostheses. Alternatively, instead of using classification or regression by their own, a merger of both systems could be favorable. With this system, the benefits of one method could be used to overcome the limitations of the other. This idea is the focus of some ongoing studies. Being there yet no perfect learning model, a mixed model of classification and regression seems to be a good proposal [122].

Feedback implementation is another of the main open and active fields. How to integrate the user into the system seems to be one of the keys to improve prosthesis control and give a more suitable handling to the user. Able-bodies with intact limbs are provided with several feedback information: touch, vision, pressure, etc. Therefore, as a next step, it is necessary that these information gathered by the sensors of the prosthesis become accessible for the users, to get closer to the functionality of a biological limb. Nowadays, actual prostheses are mostly limited to intrinsic visual and acoustic feedback, available by observation of the prosthesis and sounds of the motors, therefore, the implementation of all other kinds of feedback has to be further investigated, as it is the case for vibrotactors or pressure [124]. This will increase the acceptance of the prosthesis with the corresponding positive effects.

The computational power required by the new systems are higher than the ones actually used in commercial prostheses. In terms of hardware, the use of complex features, models, the control of multiple DoFs and the implementation of feedback requires high computational resources [125]. For some of the proposed features, it is still to be shown whether they can be computed on a minimalist low-power hardware within the short time available in order to meet real-time constraints. Some examples of complex systems working in real time using computer front-ends are [15,93,96]. Furthermore, some other of the novel models have been tested in embedded systems proving their capability to achieve the desired performance and overcoming the technological limitations [118,123], but they are still under development.

Currently, the market of upper limb prostheses is relatively small. Therefore, development becomes slow and expensive. However, the prosthetic industry can take advantage of additional new emerging technologies and benefit from the fast developments such as the smartphone sector, intelligent robotics and other consumer and industry sectors. Prostheses could take advantage of the fast growing 3D printing field lowering manufacturing cost [125,126]. The computational capabilities of low-power processors is constantly increasing, pushed in part by the large smart device market. This situation will help to generate new embedded systems, suitable for prosthesis control. Furthermore, Internet-of-Things [127,128,129,130] or 5G [131] could rise as a possible solution, using e.g., real-time cloud computing to withdraw the high computational requirements from the device. These new technologies allow the integration of new processing protocols such as data fusion. Cloud computing could be used to collect data from different users generating larger datasets. The use of this datasets to extract common information between patients could improve the model’s performance and adaptability. This can be understood as a transfer learning process between different users.

Prostheses have to be adapted to the significant variability in the population of users [132]. This variability demands to the prosthesis control to be usable in a wide range of physiological conditions. Moreover, prostheses have to be simple, in order to be configured by a non-signal processing expert such as orthopaedic technicians. The end user needs to have an easy usable prosthesis and he has to be able to understand how to use it without help. A simple user learning is necessary to reach a wider population. Complicated learning processes of the prosthesis control will not be suitable for beginners who could quit under the difficulty and consider it as not useful. The systems have to be accessible for everyone and not only for advanced users. So, working on the final training protocol and the setup of the prosthesis to make it as easy as possible, even using complex models, is a future task for the research community.

Lastly, one of the biggest problems is that most models have been tested extensively in controlled environments but the prove for robustness under the non-stationary conditions of daily life is often missing. This needs to be reflected in the evaluation procedures, that should include factors such as fatigue, arm positioning, sweat and long time use. The final proof of robustness has to be conducted in the daily life of the end users, revealing the need for large scale tests in clinical environments.