Enhancing Robustness of Surface Electromyography Pattern Recognition at Different Arm Positions for Transhumeral Amputees Using Deep Adversarial Inception Domain Adaptation

Abstract

Pattern recognition in myoelectric control that relies on the myoelectric activity associated with arm motions is an effective control method applied to myoelectric prostheses. Individuals with transhumeral amputation face significant challenges in effectively controlling their prosthetics, as muscle activation varies with changes in arm positions, leading to a notable decrease in the accuracy of motion pattern recognition and consequently resulting in a high rejection rate of prosthetic devices. Therefore, to achieve high accuracy and arm position stability in upper-arm motion recognition, we propose a Deep Adversarial Inception Domain Adaptation (DAIDA) based on the Inception feature module to enhance the generalization ability of the model. Surface electromyography (sEMG) signals were collected from 10 healthy subjects and two transhumeral amputees while performing hand, wrist, and elbow motions at three arm positions. The recognition performance of different feature modules was compared, and ultimately, accurate recognition of upper-arm motions was achieved using the Inception C module with a recognition accuracy of 90.70% ± 9.27%. Subsequently, validation was performed using data from different arm positions as source and target domains, and the results showed that compared to the direct use of a convolutional neural network (CNN), the recognition accuracy on untrained arm positions increased by 75.71% (p < 0.05), with a recognition accuracy of 91.25% ± 6.59%. Similarly, in testing scenarios involving multiple arm positions, there was a significant improvement in recognition accuracy, with recognition accuracy exceeding 90% for both healthy subjects and transhumeral amputees.

1. Introduction

Human arm amputation causes substantial functional and muscle loss for amputees. The loss of a hand or arm through amputation greatly decreases the quality of life of an individual [1]. Amputees sometimes vividly experience the presence of their missing limb and voluntarily execute “real” motions [2]. This form of motor execution, termed phantom limb movement (PML), exists with its own underlying neurophysiological mechanisms, and the associated muscle activity varies depending on the type of movement being executed [3]. Based on this theory, pattern-recognition-based myoelectric prostheses can recognize specific movement patterns by acquiring myoelectric signals from the residual limb through electrodes around the arm, visually linking the signals to the movements, and effectively enhancing the intuitiveness of prosthetic control [4,5,6,7,8,9,10,11].

However, transhumeral (TH—above the elbow) amputees suffer greater loss of muscles and functionality than transradial (TR—below the elbow) amputees. Operating a prosthesis requires excessive physical or mental effort on the part of the TH amputees because it necessitates the use of arm muscles that are not naturally involved in these motions, which results in inaccurate and unreliable recognition. It is difficult for the user to learn, making the prosthesis frustrating to use [11] and resulting in a higher rate of rejection compared to TR prostheses [12,13].

Enrichment of signal source techniques, including the use of high-density electrodes, filtering techniques and signal fusion, has become one of the main research directions to improve pattern recognition performance. High-density electrodes can cover a larger muscle area and provide rich feature information [14]. Filtering techniques improve the spatial resolution of EMG signals by reducing crosstalk between surface EMG signals [15]. Methods for monitoring brain activity, including electroencephalography (EEG) or functional near-infrared spectroscopy (fNIRS), which is non-invasive and has a relatively short time constant, have proven to be strong candidates for additional inputs for human–machine device control [16]. The fusion of brain activity information, complementing the limited EMG signals, can effectively classify the motor intentions of amputees. Although these methods have yielded satisfactory results in research, the complexity of devices associated with multiple types of information falls short of practical application scenarios.

In pattern recognition, traditional machine learning methods rely heavily on hand-crafted features and can only capture shallow features, resulting in limited accuracy. Previous research has shown that deep learning is a viable solution, deep learning can learn feature representations for classification tasks without domain-specific knowledge and outperform traditional machine learning in terms of recognition performance [17].

However, when subjects perform hand movements in different arm positions, the performance of the pattern recognition control scheme is inevitably affected. Several studies have shown that arm position increases classification errors when using training data from one arm position and test data from another [18,19,20,21,22]. Existing studies typically use data from multiple arm positions for training, using datasets or feature sets large enough to achieve the desired performance. However, this approach usually fails to achieve the desired performance and can place a significant burden on model updates.

To address the above issues, our research objective is to explore an algorithmic framework that can accurately recognize upper arm movements at different arm positions. In order to achieve this objective, a domain adversarial neural network named the Deep Adversarial Inception Domain Adaptation (DAIDA) is proposed. The main contributions of this paper are highlighted as follows:

• Leveraging the power of the Inception module, we incorporated parallel convolutional kernels of different sizes to enhance the feature-capturing capability of the model. This ensures the model’s ability to discern nuanced details crucial for upper arm motion recognition. Meanwhile, our algorithmic framework facilitates the seamless execution of two training phases without modifying any structures. These two training phases can eliminate heterogeneous features to ensure good motion recognition performance at the new arm position.
• To validate the effectiveness of our proposed model, we conducted experiments involving both able-bodied subjects and transhumeral amputees. This result showcases the model’s motion recognition accuracy and stability at different arm elevation positions, providing prospects for the advancement of myoelectric interfaces.

2. Methods

This section describes the proposed algorithmic framework to address the problem of accurate upper arm movement recognition and position change stability in transhumeral amputees. First, an experimental protocol was developed to acquire sEMG signals of upper arm movements in common arm positions, and the acquired signals were preprocessed as inputs to the model. Subsequently, the CNN model was determined by evaluating the performance of the parallel convolutional structure through several evaluation metrics. Finally, by combining domain adaptation and adversarial training, the DAIDA framework is proposed and evaluated under several training scenarios to solve the domain shift problem due to arm position changes.

2.1. Data Acquisition

A total of 12 subjects were recruited for the experiment, including 10 able-bodied subjects and 2 transhumeral amputees. Subjects gave consent to participate in the experiment, which was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The able-bodied subjects were adults with no musculoskeletal disorders or other serious health problems and who have good physical functioning. The eligibility criteria for the amputee subjects were preservation of the major muscles of the upper arm after the cutoff, good control in moving the phantom arm as well as repeatable phantom arm movements with a minimum amplitude of greater than 20 degrees, and absence of phantom arm pain. Based the joint project of the National Key Research and Development Program of China, the experiment has been approved by the Ethics Committee of Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences (YSB-2022-Y07026). Participants provided written informed consent to participate in this study. Information on the able-bodied and amputee subjects is shown in

Table 1. Information on the able-bodied subjects.

No.	Gender	Age	Dominant Hand	Upper Arm Length (cm)
S1	female	21	right	35
S2	male	21	right	33
S3	male	22	right	36
S4	female	22	right	30
S5	female	22	right	28
S6	male	25	right	34
S7	male	24	right	35
S8	female	23	right	31
S9	male	24	right	29
S10	male	22	right	32

and

Table 2. Information on the amputee subjects.

No.	Age	Dominant Hand	Amputation Side	Amputation Years	Amputation Reason	Amputation Side Length (cm)	Complete Upper Arm Length (cm)
TH1	41	right	left	9	industrial injury	23 cm	31 cm
TH2	39	right	left	5	industrial injury	25 cm	33 cm

, respectively.

Eight sEMG electrodes were arranged equidistant and parallel from each other around the upper arms of both types of subjects to ensure that the electrodes covered the major muscles of the subjects’ biceps, triceps, and brachialis muscles while capturing as many valid sEMG signals as possible [23,24]. The skin on the upper arm of the subjects was treated with hair clipping, removal of dead skin and alcohol cleansing to ensure the stable acquisition of the surface sEMG signals. The sEMG signals were acquired using a Noraxon sEMG signal acquisition device with a sampling frequency of 1500 Hz.

During signal acquisition, subjects were asked to sit in front of the table and place their upper arms at the most comfortable positions on the support platform by adjusting the seat height and the tilt angle of the support platform. To prevent signal recordings from being affected by tabletop compression, the sensors located in a slot in the center of the support plate do not touch the support plate and remain suspended. Changes in arm position can also lead to significant differences in sEMG signals, affecting recognition performance. Three arm elevation positions were selected to complete sEMG acquisition experiments, as shown in Figure 1. Within these three arm elevation positions, subjects are able to carry out most of the activities required for daily life, including the following:

• P1: upper arm parallel to the horizontal plane;
• P2: elevated upper arm to 45° from the horizontal plane;
• P3: elevated upper arm to 90° from the horizontal plane.

A video of the movements is displayed on the computer screen directly in front of the subject as a guide, and the subjects are asked to imitate the movement in the video synchronously with the guidance. The amputee subjects utilized their healthy arm as a mirror image to aid the stumped arm in replicating phantom limb movements, as shown in Figure 1. The phantom arm movements consisted of six movements of the hand, wrist and elbow, which were Hand Close, Hand Open, Wrist Pronation, Wrist Supination, Elbow Flexion and Elbow Extension, as shown in Figure 2.

The sEMG data were obtained by regular exercise–rest cycles. Each movement was acquired in ten repetitions. For each repetition, the subject held the muscle contraction for 5 s and then rested for 2 s. It was required that the amount of force with which the subject held the movement and the posture in which the movement was performed should be as consistent as possible. After the acquisition of ten repetitions of a gesture, a 1-minute rest was taken to avoid muscle fatigue.

sEMG signals were acquired from subjects for each set of movements at each arm position, with a 2 min rest after the end of the signal acquisition at each arm position. The guiding principle behind the adoption of the above procedure was the need to limit the recording time and fatigue, while avoiding possible stress effects of fully continuous movements or prolonged acquisitions. A total of 180 sets (3 arm positions × 6 motions × 10 repetitions) of experimental data were ultimately collected for each subject.

2.2. DAIDA Framework

2.2.1. Preprocessing

The sEMG signals are stochastic non-stationary signals and therefore need to be preprocessed before they can be used for subsequent classification. The preprocessing process of the data in this study consisted of filtering and noise reduction, normalization and active segment extraction.

According to the spectral energy distribution of the sEMG signal, the motion artifacts were removed using 4th-order 20–500 Hz Butterworth bandpass filtering, and the industrial frequency noise was removed using 50 Hz notch filtering. Because different subjects’ anatomical tissues, physiological states and other factors make the multichannel sEMG signals show large individual differences [25], in order to reduce the impact of individual differences on pattern recognition classification, it is necessary to standardize them by transforming sEMG signals from different subjects. The standardization method used in this article is Z-Score standardization. The transformation formula is (x − μ)/σ [26]. In this formula, μ is the mean and σ is the standard deviation. It converts data of different orders of magnitude into unitless values [27].

In order to obtain effective motion information, the determination of active segments is essential [28,29]. Therefore, in this study, an adaptive dual-thresholding method was used for the detection of active segments of sEMG signals [27]. According to studies, the optimal window length for sEMG-based pattern recognition (PR) falls within the range of 100 ms to 250 ms [30]. To ensure classification accuracy while taking into account Noraxon’s sampling rate, a window of 300 sample points in length was used, sliding over the data in incremental lengths of 150 sample points.

Eventually, the sEMG signals were converted into a suitable data format for input to the network, i.e., batch size × 1 × number of channels × window length.

2.2.2. The CNN Model

This section describes in detail the domain adaptive technique used to solve the problem of spatial inconsistency of features at different arm positions. Then, the architecture and training process of the proposed method are described.

In the practical application of myoelectric prosthetic hands, the effectiveness of pattern recognition may be greatly reduced due to changes in the distribution of signal features caused by changes in arm position. Different arm positions result in different muscle activations, leading to different data distributions [21]. This is one of the reasons why traditional models are unstable in action prediction under different arm positions. To address this problem, the DAIDA algorithmic framework proposed in this paper recognizes the sEMG signals of the upper arm under different arm positions. The idea of domain adaptation is utilized to make the feature learning distribution of the source domain close to the target domain.

The DAIDA structure was constructed from two networks with almost identical CNN structure. The CNN model used serves as the baseline model in the study. The CNN is composed of two modules, a feature extraction module and a classification module, as shown in Figure 3. The feature extraction module consists of two blocks, each of which encapsulates a convolutional layer, followed by a batch normalization layer (BN), a Parametric Rectified Linear Unit (PReLU) layer, a dropout layer (p = 0.5) and a max-pooling layer. Inspired by the success of excellent models like Xception, which perform well in classification tasks and demonstrate good generalization ability on new datasets through transfer learning with approaches such as feature extraction and fine-tuning, we adopt the inception structure to replace regular convolution in order to efficiently capture the features of various scales of the input data [27,29]. This structure employs multiple parallel branches with different convolution and pooling operations to extract multi-scale information from the input feature map. Among them, branch one uses 1 × 1 convolution for channel transformation of the input, branch two performs spatial dimensionality reduction through 3 × 3 max-pooling and 1 × 1 convolution and branches three and four use 5 × 1 convolution and 7 × 1 convolution with different dimensions, respectively, to extract features at different scales. The outputs of these branches are spliced in the channel dimension to generate the final feature map. The classification module also comprises two blocks, each consisting of a fully connected layer, a batch normalization layer, a PReLU layer and a dropout layer (p = 0.5). The number of neurons in the fully connected layer is 1000 and 500, respectively. Adaptive moment estimation (ADAM) is employed to optimize the CNN with an initial learning rate of 0.001.

In order to compare the effectiveness of feature extraction with score-parallel convolution, we compared various structures, as shown in Figure 3.

2.2.3. DAIDA Framework

The DAIDA framework we used, shown in Figure 4, has a similar structure to the baseline model, sharing the same convolutional module for feature extraction. The model then generates branches, one of which is a labeled classifier for the motion classification trained in the source and target domains. The second branch is a domain classifier using a Gradient Reversal Layer (GRL), which is set between the domain classifier and the feature extractor to determine whether the collected feature distributions are from the source or target domain. The GRL layer that maximizes the loss of the classifier forces the feature extraction block to learn features that are not related to the domain, i.e., domain-invariant features. Ultimately, the model learns features with better generalization performance. This operation takes place during the backpropagation phase and consists of multiplying the gradient by a negative value (−λ) while, during forward propagation, the gradient acts as an identity transform. The GRL can be easily implemented to any feed-forward network without any additional parameters.

Consider a classification problem with the input feature vector space X and Y = {1, 2, 3, 4, 5, 6} as the set of labels in the output space. Let {X_s, Y_s} and {X_t, Y_t} be unknown joint distributions defined over X × Y, referred to as the source and target distributions, respectively. The unsupervised domain adaptation algorithm requires input as the labeled source domain data, sampled from {X_s, Y_s}, and unlabeled target domain data, sampled from the marginal distribution {X_t}. The data from the source and target domains are fed into the feature extractor, and the features are classified by the domain classifier.

The architecture has two networks, and each network consists of a feature extractor and a classifier. The feature extractor $G_f$, shown in Figure 4, is a parallel convolution network and takes the preprocessed sEMG input vector $x_i$ and generates a feature vector $f_i$, i.e.,

$$f_i=G_f\left(x_i;{\theta }_f\right)$$

(1)

The feature vector $f_i$ extracted from $G_f$ is mapped to a class label by the label classifier $G_y$ and to a domain label by the domain classifier $G_d$, i.e.,

$$y_i=G_y\left(f_i;{\theta }_y\right)$$

(2)

$$d_i=G_d\left(f_i;{\theta }_d\right)$$

(3)

Both the label classifier and the domain classifier are multilayer feed-forward neural networks whose parameters are referred to as ${\theta }_y$ and ${\theta }_d$, respectively. During training, the goal of the feature extractor is to minimize the classification error while making the domain classifier unable to accurately distinguish between the feature representations of the source and target domains. In this way, the feature extractor is able to learn feature representations that have good generalization ability to both the source and target domains.

In the proposed DAIDA scheme, we utilize parallel convolution to extract effective motion features, and employ adversarial training to simultaneously minimize the classification loss and maximize the diversity of domain distribution. This enables us to eventually achieve a position-invariant model. The complete DAIDA algorithm is described in Algorithm 1.

Algorithm 1 Proposed DAIDA Algorithm

Input: (1) Training data: Labeled source $D_S$ and unlabeled target $D_T$. (2) learning rate η, $T_{max}$ and iteration $N_{max}$; (3) Initialized parameters: ${\theta }_f$ of feature extractors, ${\theta }_y$ of label classifier, ${\theta }_d$ of domain classifier, weighting hyper-parameter $\mathrm{\lambda }$. Output: Optimized parameters ${\theta }_f$, ${\theta }_y$ and ${\theta }_d$. for T = 1,2, …, $T_{max}$ do   for N = 1,2, …, $N_{max}$ do     Select mini batch: $d_S$, $d_T$ from $D_S$, $D_T$     Calculate weights: ${\theta }_f$, ${\theta }_y$ and ${\theta }_d$     Calculate the losses: $\begin{array}{c}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_y\left({\theta }_f,{\theta }_y\right)=\frac{1}{n}{\displaystyle \sum _{\left(x,y\right)\in d_s}}{y}^{T}log{G}_y(G_f\left(x_i;{\theta }_f\right);{\theta }_y)\end{array}$ $\begin{array}{c}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_d\left({\theta }_f,{\theta }_d\right)=\frac{1}{m}{\displaystyle \sum _{\left(x,y\right)\in d_T}}{y}^{T}log{G}_d(G_f\left(x_i;{\theta }_f\right);{\theta }_d)\end{array}$     Calculate the final loss: $\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}={\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_y-\mathrm{\lambda }{\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_d$     Update the parameters:       ${\theta }_f={\theta }_f-\mathrm{\eta }\nabla \mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}\left({\theta }_f\right)$      ${\theta }_y={\theta }_y-\mathrm{\eta }\nabla \mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}\left({\theta }_y\right)$      ${\theta }_d={\theta }_d-\mathrm{\eta }\nabla \mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}\left({\theta }_d\right)$   End for End for

2.2.4. Training Scheme

The section aims to validate DAIDA’s performance in adapting to arm positions from multiple perspectives. We collected data from different arm positions to use as the source domain, target domain and testing datasets to validate the generalization performance of DAIDA. For each subject, data from one single arm position containing 6 motions × 10 repetitions were collected as an acquisition. These experiments were designed to test the arm position stability of DAIDA, with a CNN as the baseline for without-domain adaption. When testing the CNN, an odd number of repetitions was used for training and an even number of repetitions was used for testing. And when testing DAIDA, the 1st and 2nd repetitions were used for retraining as the target domain, and the remaining eight repetitions were used for testing. The pre-training datasets of DAIDA under each strategy comprised the same data as the CNN training sets as the source domain, and the testing set was different.

The purpose of the baseline model is to verify the feasibility of the CNN model for upper arm motion recognition and the separability of arm positions. The data used as input came from the sEMG signals obtained from a single subject’s upper arm position when completing a movement at a single position as a set. There were 10 healthy subjects and 2 transhumeral amputees, and each subject completed six motions in 3 arm positions, i.e., 18 sets of data in total. The experimental data for arm position separability came from a single action for all subjects, with a total of six sets of motions i.e., six sets of data to validate the variability in the same action between different arm positions.

The datasets used for training and testing for both types of experiments are from the same set of experimental data, and both types of experiments use only the source network for recognition.

The DAIDA model is proposed to address the performance degradation of models at different arm elevation positions. In this study, this problem is regarded as a transfer learning problem. To achieve the desired transfer learning effect, the model needs to extract similar deep features from sEMG signals belonging to both the source domain and the target domain. The distributions of the source domain and the target domain can be used to assess the similarity of the extracted deep features.

To validate the recognition performance of the model at different arm elevation positions, the specific validation program is described as follows, and six model testing scenarios are designed, as shown in

Table 3. Arm position domain adaptation training scheme.

No.	Labeled Source Domain Data	Unlabeled Target Domain Data	Testing Data
1	x, y, z	\	x
2	x	y	x
3	x	y	y
4	x	y	z
5	x, y, z	y	y
6	x, y, z	y	NOT y

The characters x, y, z each represent one arm position, respectively (x, y, z ∈ (1,2,3) and x ≠ y ≠ z).

:

• Without domain adaptation, training is solely based on labeled source domain data; i.e., the training data are labeled motion data from all positions and the test data are labeled data from one single position (No. 1).
• Domain adaptation is performed on source domain data; i.e., the test data come from the source domain and do not overlap with the retraining data (No. 2).
• Domain adaptation is performed on data from the target domain; i.e., the test data are from the target domain and from the same domain as the retraining data (No. 3).
• The ability to generalize the model after domain adaptation is determined; i.e., test data are not derived from the source nor the target domain (No. 4).
• The performance of the domain adaptation effect on the total dataset is determined; i.e., the source domain data from the sum of all arm position data, the target domain from data of one single arm position and the test data from a single arm position that are identical to (No. 5) or distinct from (No. 6) the target data.

2.3. Statistical Analysis

The performance of the models was represented using the most common evaluation measures, such as accuracy, precision, recall, specificity and F1 score. These metrics were defined as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}\times 100\%$$

(4)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}\times 100\%$$

(5)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}\times 100\%$$

(6)

$$\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}}\times 100\%$$

(7)

$$\mathrm{F}1\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{\left(\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\right)}{\left(\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\right)}\times 100\%$$

(8)

where TP is the true positive rate, TN is the true negative rate, FN is false negatives and FP is false positives. Accuracy represents the ratio of correct predictions over total predictions. Recall measures the proportion of true positive predictions among all actual positive instances. Specificity calculates the proportion of true negative predictions among all actual negative instances. Precision quantifies the proportion of true positive predictions among all positive predictions. Finally, the F1 score combines precision and recall into a single metric. The error represents the standard deviation of the accuracy for different gesture classes. Both metrics and error information are expressed as percentages.

To quantify the effectiveness of the proposed approach, we conducted statistical analysis via the t-test and the Wilcoxon rank sum test. The significance level was set at p < 0.05. In Section 3.1, we statistically analyze the various metrics of different feature extraction modules. In Section 3.2, we compare the performance differences of different training schemes for statistical tests. In Section 3.3, we only compare the accuracy numerically due to the small number of amputees.

3. Results

3.1. CNN Model Recognition Performance

Since the proposed DAIDA was an improved version of the Inception network, we justified improving the model by using different Inception blocks, namely, Inception A, Inception B and Inception C. The block architecture is presented in Figure 3d. The average recognition accuracy, precision, recall, specificity and F1 score of the different Inception blocks used under a single arm position are shown in

Table 4. Recognition performance of different feature extraction modules.

Module	Accuracy (%)	Precision (%)	Recall (%)	Specificity (%)	F1 Score (%)
Stem	88.71 ± 9.43 *	91.53 ± 7.85 *	91.85 ± 7.15 *	97.82 ± 1.54 *	91.72 ± 7.45 *
Inception A	90.24 ± 3.55 *	93.58 ± 6.29 *	91.82 ± 6.14 *	97.82 ± 1.57 *	92.62 ± 6.98 *
Inception B	90.07 ± 5.71 *	93.68 ± 6.85 *	91.91 ± 6.97 *	96.87 ± 1.64 *	92.71 ± 5.14 *
Inception C	95.70 ± 1.27	94.34 ± 4.04	92.81 ± 6.35	97.94 ± 1.55	93.57 ± 6.19

* p < 0.05, bold indicates the best results.

. As illustrated in Table 4, the Inception C block outperforms the rest of the blocks in all parameter comparisons (p < 0.05). Meanwhile, the Inception structure outperforms conventional convolutional layers.

Figure 5 presents the performance of the CNN model at a single arm position, showing a confusion matrix with an average accuracy of 95.70 ± 1.27%.

The baseline model discriminates the separability of arm positions, and the recognition results are shown in Figure 6. The average recognition accuracy at position P1 is 94.83% ± 1.35%; at position P2, it is 93.59% ± 2.97%; and at position P3, it is 97.12% ± 3.59%. Recognition accuracy was highest at position P3 and lowest at position P1. The recognition accuracy at all positions was 92.00% ± 5.70%.

Similarly, the baseline model was used to determine the variability in the data for the six categories of motions with respect to arm position, with an average identification result of 96.32% ± 2.77%. The confusion matrices are shown in Figure 7.

3.2. Position Recalibration

We trained the source network using labeled data from three different arm positions as inputs and achieved an average accuracy of 95.70% ± 1.27%. Additionally, the average recognition accuracy at different arm elevation positions was 92.00% ± 5.70%, as shown in Figure 6.

As shown in Table 3, the sEMG signals at position x and position y are used as the pre-training inputs in the source domain and the retraining inputs in the target domain, respectively. The model test results unfold in three scenarios, and the test data are position x, position y and position z (x, y, z ∈ (1,2,3) and x ≠ y ≠ z), respectively.

Recognition accuracies of 15.01% ± 3.08%, 15.54% ± 3.29% and 15.91% ± 2.42% were tested using the CNN directly, i.e., using only the pretrained source network without retraining at the positions x, y, or z, respectively (No. 1).

However, significant improvements were observed when utilizing DAIDA. The accuracy testing at positions x, y and z yielded results of 44.03% ± 12.26% (No. 2), 92.01% ± 7.37% (No. 3) and 41.99% ± 10.98% (No. 4), respectively. Moreover, when employing all arm positions as the source domain data for pretraining, the accuracies achieved were 54.60% ± 12.86% (No. 6), 92.15% ± 8.47% (No. 5) and 49.60% ± 8.86% (No. 6) for positions x, y and z, respectively.

3.3. The Recognition Performance of Transhumeral Subjects

Table 5. Recognition results of different training programs for the transhumeral amputees.

No.	Labeled Source Domain Data	Unlabeled Target Domain Data	Testing Data	Accuracy (%)
1	x, y, z	\	x	42.08 ± 8.21
2	x	y	x	43.61 ± 8.77
3	x	y	y	94.93±3.21
4	x	y	z	40.60 ± 9.68
5	x, y, z	y	y	94.06±4.18
6	x, y, z	y	NOT y	58.43 ± 8.38

The characters x, y, z each represent one arm position, respectively (x, y, z ∈ (1,2,3) and x ≠ y ≠ z). Bold indicates the best results.

shows the recognition performance of DAIDA for transhumeral amputees under different training schemes. The recognition accuracy tested directly after pretraining the model using only sEMG signals under all arm positions is 42.08% ± 8.21%. The recognition accuracies are 43.61% ± 8.77%, 94.93% ± 3.21% and 40.60% ± 9.68% when using data from a single arm position to pretrain and test the performance of the domain adaptation in the source and target domain data as well as the generalization performance of the models corresponding to scenarios 2 to 4, respectively. When pretraining using data from all arm positions to detect the performance of domain adaptation effects on the total dataset, the recognition accuracies were 94.06% ± 4.18% and 58.43% ± 8.38%, corresponding to schemes 5 and 6, respectively.

4. Discussion

In order to achieve accurate recognition of upper arm motions and to mitigate the impact of arm position effects on the performance of myoelectric interfaces during real-world use, we propose a network framework with an Inception feature extraction structure that combines the techniques of adversarial training in the domain adaptation domain. In this paper, we design the DAIDA to effectively extract and classify domain-invariant features across domains. Specifically, by optimizing the three components of the adversarial network, namely the feature extractor, label predictor and domain discriminator, the network can learn arm-position-specific features by aligning the feature distributions of different arm positions to reduce the impact of domain migration due to changes in arm position. The feature extractor learns discriminative features of sEMG signals through the deep network. The label predictor is used to distinguish between different tasks, while the domain discriminator is used to reduce the inter-domain variation in features. Thus, this framework effectively achieves better accuracy and adaptability in the recognition of upper arm motions.

This study first demonstrates that the classification performance of the parallel structured network outperforms a network using ordinary convolutional layers in upper arm motion recognition (Table 4). The network structure proposed in this paper combines the feature maps output from the parallel convolutional kernels with different kernel sizes to construct feature maps with more comprehensive semantic information in order to improve the model’s ability to recognize upper arm movements. More complex parallel structures look to exhibit better model performance while ensuring that the network size is appropriate. Therefore, in this study, the Inception C structure was chosen as the feature extraction module of the baseline model.

The model performs well in recognizing upper arm motions under the three commonly used arm positions, as can be seen in Figure 6, with recognition accuracies greater than 88% for all of them. The average recognition accuracy was also greater than 90% when using all arm positions as inputs. There is a distinction between the recognition results at the three arm positions, which may be due to the different characteristics of the corresponding EMG data.

In comparison to previous studies, Gaudet et al. found that it was possible to classify eight upper-limb phantom limb movements and one no-movement category in transhumeral amputees with an average accuracy of 81.1% [31]. The inclusion of kinematic features increased the accuracy by an average of 4.8%. Jarrassé et al. successfully classified up to 14 phantom arm movements, achieving an average recognition accuracy of more than 80% when only six hand, wrist and elbow movements were considered [3]. In contrast to previous work, Huang et al. successfully improved the identification accuracy of 15 motions of humeral amputation patients to more than 85% by using a higher-order spatial filter to obtain more informative high-density sEMG signals [32]. Li et al. found that the classification performance of fused sEMG and EEG signals was significantly better than that of single sEMG or electroencephalogram (EEG) signal sources [33]. The accuracy of this fusion method was as high as 87%. Nsugbe et al. found that eight discrete gestures of the upper arm could be recognized with a best classification accuracy of 81% [34]. While the number of movements recognized in previous studies is different from ours, it is worth noting that these works required the use of either high-density electrodes or information fusion as inputs to the classifier, which means that more complexity is introduced in practical use. Our study achieves effective recognition of commonly used motions while using routine EMG electrodes as input.

Previous studies show that the arm position has a significant impact on gesture recognition [35]. As shown in Figure 7, the average recognition accuracy for classifying arm positions using the baseline model is over 95%. This indicates that there is great variability in the motion signals for different arm positions, which is consistent with existing research [21,36,37,38]. It is also illustrated that different arm elevation position recognition results are not obtained due to the dissimilarities in the EMG signals under each arm position. Therefore, we adopt DAIDA to alleviate this impact; i.e., after pre-training with data from a single arm position, migration can be completed following a simple calibration to make the EMG interface applicable to another arm position.

From Figure 8, it can be observed that when a recognition model pretrained on the source domain arm position is retrained with target domain data (different from the source domain arm position) and tested on the source domain arm position (the same arm position as the source domain), the variation caused by different arm positions results in a maximum recognition accuracy of 44.03% ± 12.26%. As can be observed from the first column of Figure 8, the recognition accuracy is improved by 29.02% ± 9.18% compared to the direct use of CNN when tested on the source domain data (p < 0.05).

When the recognition model pretrained on the source domain position is tested on the target domain arm position, the recognition accuracy can reach up to 91.25% ± 6.59% at the test arm position. As can be observed from the second column of Figure 8, this is an improvement of 75.76% ± 2.30% over the result obtained from testing using the CNN directly (p < 0.05). Using the whole dataset (three arm positions) as the source domain, the best recognition results were achieved when testing on the target domain data, with a recognition accuracy of 93.33% ± 6.86%. When not tested on the target domain data, the recognition results declined significantly, with the highest recognition accuracy being 54.60% ± 12.86%, which was still significantly higher than the direct use of CNNs. This meets the expected results while using only a small amount of data for retraining to achieve a performance that approximates the source domain model, meets the expected results and achieves the goal of recognizing different arm elevation positions [21].

When the recognition model trained on the source domain position is tested on data that are different from the source and target domains, the highest arm position recognition accuracy can be 41.99% ± 10.98% in the test. As can be observed from the third column of Figure 8, this is an improvement of 29.08% ± 8.56% over the result obtained from testing with the CNN directly (p < 0.05). This demonstrates the weak generalization ability of the provided domain adaptation model.

A similar distribution of results was demonstrated in the experimental results for the transhumeral amputees. Regardless of whether the source domain data comprised a single arm position or all arm position data, better recognition results with a recognition accuracy greater than 90% were only shown when tested on the target domain. When not tested on the target domain, the recognition accuracy was less than 60%. This fits well with the use scenario of pattern recognition based on sEMG signals, i.e., motion recognition of transhumeral amputees controlling a myoelectric prosthesis in different arm positions. The modeling framework proposed in this study meets the needs of myoelectric prosthesis motion recognition for transhumeral amputees and solves the problem of performance degradation caused by arm position changes.

In previous studies addressing position effects, Fougner et al. used sEMG and accelerometer sensors attached to 10 subjects with normal arms to reduce the average classification error from 18% to 5% [21]. Khushaba et al. presented their proposed spectral moments to achieve significant reductions in the classification error rates of up to 10% on average across subjects and five different limb positions [22]. Geng et al. proposed a cascade classification scheme using a position classifier and multiple motion classifiers responsible for different arm positions using multichannel EMG signals and tri-axial accelerometer mechanomyography signals to reduce the interpositional classification error of amputees by up to 10.8% [19]. Mukhopadhyay et al. used time domain power spectral descriptors as the feature set to train a Deep Neural Network classifier, obtaining 98.88% recognition accuracy on five limb positions [36]. The aim of our study was to address TH amputees’ problem of decreased recognition performance due to arm position changes, and few studies have focused on this group. However, TH amputees similarly and inevitably need to make arm position changes while using their prostheses to complete daily activities [39]. Our proposed algorithm, which does not require the introduction of other inputs or computation of features, simply uses two action repetitions for retraining to achieve stable recognition at different arm elevation positions, which is in line with the actual usage scenario of myoelectric prostheses.

Our proposed DAIDA is able to achieve accurate upper arm movement recognition while realizing adaptation at different arm elevation positions, which is useful in practical applications, by improving the model performance from the source domain to the target domain through knowledge migration. Second, the deep domain adaptation network is able to capture the shared features between the source and target domains, thus achieving better model generalization capabilities. Therefore, DAIDA is uniquely suited to address motion recognition at different arm elevation positions.

Although the DAIDA presented in this paper enables the stable detection of multiple limb positions in transhumeral amputees, we have only covered the most common limb positions in daily life and not all degrees of freedom of the upper arm. At the same time, we were able to achieve a limited number of movement patterns. Practical prosthesis use may also involve factors such as electrode displacement and muscle fatigue, which will be a direction for future refinement.

5. Conclusions

The DAIDA algorithmic framework proposed in this paper first applies a parallel feature module, which significantly improves action recognition for transhumeral amputees. At the same time, this study and existing studies have shown significant variability in the sEMG patterns of the upper arm due to changes in arm position, which adversely affect the performance of models trained in a domain-independent environment. The DAIDA proposed in this paper utilizes a domain adversarial neural network to learn unsupervised domain-invariant features directly from the source arm position signal. By using a small amount of unlabeled arm position sEMG data for adversarial training based on domain adaptation, the domain shift problem present in changing arm position scenarios shows promising performance. Due to its satisfactory classification performance and reduced training burden, the proposed DAIDA framework is a promising practical myoelectric interface technique. Therefore, further work will be carried out to include more transhumeral amputees and to access the model in complex real-life scenarios to realize real-time usability.