Concurrent Adaptive Control for a Robotic Leg Prosthesis via a Neuromuscular-Force-Based Impedance Method and Human-in-the-Loop Optimization

Abstract

This paper proposes an adaptive human–robot concurrent control scheme that achieves the appropriate gait trajectory for a robotic leg prosthesis to improve the wearer’s comfort in various tasks. To accommodate different wearers, a neuromuscular-force-based impedance method was developed using muscle activation to reshape gait trajectory. To eliminate the use of sensors for torque measurement, a disturbance observer was established to estimate the interaction force between the human residual limb and the prosthetic receptacle. The cost function was combined with the interaction force and tracking errors of the joints. We aim to reduce the cost function by minimally changing the control weight of the gait trajectory generated by the Central Pattern Generator (CPG). The control scheme was primarily based on human-in-the-loop optimization to search for a suitable control weight to regenerate the appropriate gait trajectory. To handle the uncertainties and unknown coupling of the motors, an adaptive law was designed to estimate the unknown parameters of the system. Through a stability analysis, the control framework was verified by semi-globally uniformly ultimately bounded stability. Experimental results are discussed, and the effectiveness of the adaptive control framework is demonstrated. In Case 1, the mean error (MEAN) of the tracking performance was $3.6°$ and $3.3°$, respectively. And the minimized mean square errors (MSEs) of the tracking performance were $2.3°$ and $2.8°$, respectively. In Case 2, the mean error (MEAN) of the tracking performance is $2.7°$ and $3.1°$, respectively. And the minimized mean square errors (MSEs) of the tracking performance are $1.8°$ and $2.4°$, respectively. In Case 3, the mean errors (MEANs) of the tracking performance for subject1 and 2 are $2.4°$, $2.9°$, $3.4°$, and $2.2°$, $2.8°$, $3.1°$, respectively. The minimized mean square errors (MSEs) of the tracking performance for subject1 and 2 were $1.6°$, $2.3°$, $2.6°$, and $1.3°$, $1.7°$, $2.2°$, respectively.

1. Introduction

The main objective of leg prosthesis is to improve the mobility and quality of life of an individual who has been amputated for a seriously affected leg [1]. Various prototypes and commercialized products have been designed to address different requirements [2]. Owing to the unaccustomed weight-bearing of the soft tissue and skin under the stump of the thigh, excessive sustained contact force may lead to the risk of degenerative tissue ulcers at the end of the stump [3]. Hence, the contact force between the residual limb and the prosthetic receiver plays an important role in improving the satisfaction and comfort of amputees. Moreover, to accommodate different wearers, gait trajectory should be reshaped to increase the mobility of lower-limb amputees [4]. The requirement for a comfortable feeling for a wearer with a leg prosthesis is largely related to the physical interaction between the prosthesis and stump, as well as trajectory tracking [5].

While the leg prosthesis in practical applications is well defined and largely invariable, the demands are more diverse when assisting the subject, encompassing various tasks and different wearers. Therefore, the ability of online learning to perform various walking tasks is important for leg prostheses to provide effective assistance [6]. Hence, an adaptive framework based on human-in-the-loop (HIL) optimization was introduced to improve the online learning ability of leg prostheses [7]. It is important for the leg prosthesis to adaptively regulate the control parameters. In [8], the control method was modeled based on unified data-driven impedance control. The control parameters were obtained from a previously collected kinematic data set of eight normal humans. The impedance parameters can be regulated under different walking conditions. In [9], a control method based on a finite-state machine (FSM) was proposed and compared to the performance of direct myoelectric control. The control parameters can be varied for different walking tasks. In [10], the mechanical structure of a leg prosthesis was redesigned and developed using a high-torque actuator and low-reduction transmission. The motor control parameters are produced from able-bodied data and can be regulated. In [11], an adaptive stair ascent controller for an above-knee leg prosthesis was discussed. With the adaptation to the changed heights of different stairs, the amputee can climb stairs with preferred cadence and gait patterns. The control parameters can be adjusted without the help of amputees’ experience. To adaptively control the motions of the leg prosthesis, a cost function is usually designed to tune the control parameters. In [12], researchers designed a cost function based on the metabolism of the subject to tune the control parameters of the exoskeleton joints. In [13], a data-driven based on the continuous-phase variable was used during the stance phase, and kinematic control was used during the swing phase. A cost function was built to optimize the impedance parameters under different walking conditions. In [14], different impedance controllers were used and compared to reduce the metabolic cost of the subject. The cost function was designed and optimized to tune the control parameters and balance the system weight and assistance efficiency. However, the search process for optimizing the cost function usually requires hours. In [15], experiments showed that the cost function based on the subject’s metabolism is not suitable for real-time adaptation. In [16], the combined cost function based on both the tracking errors and the subject’s metabolism was used to assess the subject’s discomfort during walking. Hence, to achieve the real-time adaptation of the combined cost function, an effective tuning method is needed. In [17], the robot’s motion pattern was regulated iteratively and automatically to minimize the physiological cost function. The real-time adaptive motion pattern generated various assisting forces when the subject walked on the ground with a leg prosthesis. The motion pattern parameters were searched using human-in-the-loop optimization. The physiological objective of human-in-the-loop optimization is to minimize the cost function to obtain a suitable assistance control method. Although the optimization algorithms were different in these studies, the adaptive control schemes were based on the metabolic cost function. The objective process of optimization is to minimize the metabolic state, as the amount of oxygen consumption can depict whether the leg prosthesis can assist the subject with a positive effect during walking [18].

However, it usually requires several hours to evaluate oxygen consumption. The cost function based on metabolic processes also contains massive amounts of noise [19]. Hence, other physiological responses should be chosen as objective costs. In [20], the muscle activity signal was selected as the optimization objective. Surface electromyography (sEMG) was used to measure energy consumption during muscle activity. The muscle activity signal was easily obtained and calculated in less time than the collection of metabolic costs. The signal of muscle activity has been adopted as the objective cost to regulate the control parameters of robotic leg prostheses [21]. In [22], the movement intention of the amputee was identified from the change in surface electromyography (sEMG). The performance of the classifier based on sEMG is not sufficiently accurate. Hence, multi-modal switching was adopted to combine sEMG with the signals from the inertial measurement unit (IMU) to distinguish between the stance and swing phases [23]. In [24], researchers presented a normal model for obtaining sEMG signals within a socket with soft and low-profile electrodes. The subject achieved a comfortable experience with the soft electrodes on level ground. The quantitative questionnaire characterized the comfort of the subject for soft electrodes. Qualitative analyses also suggest the feasibility of using soft electrodes for real-time sEMG data collection. In [25], a parallel classifier approach was proposed to recognize the motion intent of a subject. By analyzing the muscle activity data, a movement classifier for the subject was constructed to guide the control of the leg prosthesis. Compared to the traditional sequential approach, the parallel approach has a faster computing speed and higher computing accuracy. To achieve a suitable assistance pattern, the cost function of the robot was constructed based on muscle activity. In [26], an optimization process was proposed to regulate the control parameters to minimize the cost function. Various assistance patterns are designed for each gait condition. By recording the change in leg muscle activities, the cost function of the robot for different assistance patterns was obtained in a timely manner.

However, complex coupling uncertainties between the human body and prosthetic systems can affect the tracking performance and stability of the control system. Adaptive control strategies have been used to address the complex coupling uncertainties. The adaptive controllers estimate the uncertainties of the system and regulate the control parameters to achieve the desired tracking performance. The persistence of excitation (PE) condition should be satisfied to guarantee the convergence of the estimation process [27]. Moreover, a finite excitation condition was extended to replace the PE condition in the concurrent learning control. The concurrent adaptive control method displays the exponential convergence of the adaptive estimation process [28]. The concurrent adaptive control method stores historical input and output data. The sequence was populated to full rank until sufficient data were collected [29]. This finite excitation condition ensured a more practical method for the convergence of the estimation process without considering strict parameter conditions. Concurrent adaptive control has been proposed for different human prosthesis systems to achieve comfortable contact between the human body and the robot [30]. However, it is a great challenge to promote concurrent adaptive control of robotic leg prostheses. Owing to the existence of nonlinear muscle models and uncertain coupled motor dynamics, it is highly difficult to achieve the desired control performance using parameter updates [31].

The aim of this study was to regulate the parameters of the CPG model to find a suitable trajectory for a leg prosthesis using human-in-the-loop optimization. To realize the natural regulation of the leg joint trajectories on the lower-limb prosthesis, the neuromuscular-force-based impedance method was used for different walking conditions. To incorporate the tracking performance of the joints and the user’s comfort caused by the contact force between the stump and leg prosthesis, a combined cost function was designed. Human-in-the-loop optimization was proposed to adaptively regulate the parameters to minimize the cost function.

The main novel contributions of this study are to optimize the control parameters to minimize the overall system’s cost function under different walking terrains. The cost function included combined joint tracking errors and changes in contact force. sEMG was used to assist the subject to reshape the track trajectory under various walking conditions. To incorporate both the tracking performance of the joints and user comfort resulting from the contact force, a combined cost function was designed. Human-in-the-loop optimization was proposed to adaptively regulate the control parameters to minimize the cost function. The main innovation of this study lies in combining the human-in-the-loop optimization with adaptive control. Experimental results demonstrate that the suitable control parameters for the leg prosthesis were found using the human-in-the-loop optimization method, achieving a balance between tracking performance and changes in contact force for different walking tasks.

The major contributions of this study are outlined as follows: (1) An individualized approach for the trajectory of a leg prosthesis was developed. A Central Pattern Generator (CPG) was used to parameterize the leg joint trajectory. (2) To accommodate wearer’s comfort, a neuromuscular-force-based impedance method was developed using muscle activation to reshape gait trajectory. (3) The cost function was combined with the interaction force and tracking errors of the joints. The control scheme was mainly based on human-in-the-loop optimization to search for a suitable control weight for the CPG model to regenerate the appropriate gait trajectory.

2. Materials and Methods

2.1. Dynamic Model of the Lower Limb Prosthesis

The 2-DOF powered robotic leg prosthesis has an active ankle joint and an active knee joint. As shown in Figure 1, the dynamics of the leg prosthesis in the joint space can be written as

$$\begin{array}{c}\hfill M\ddot{q}+C\dot{q}+G=\tau +{\tau }_e\end{array}$$

(1)

where $q={[q_1,q_2]}^T\in {\mathbb{R}}^2$ denotes the real angle of joints, $q_1$ denotes the angle of the ankle joint, and $q_2$ denotes the angle of the knee joint. $\tau \in {\mathbb{R}}^2$ denotes the control torque to drive the motor. ${\tau }_e\in {\mathbb{R}}^2$ denotes the interaction torque derived from the contact force between the stump and the leg prosthesis.

$M\in {\mathbb{R}}^{2\times 2}$ denotes the inertia matrix, $M=\left[\begin{array}{cc}{M}_{11}& M_{12}\\ M_{21}& M_{22}\end{array}\right]$, where $M_{11}=m_1{d}_1^2+I_1+I_2+m_2(l_1^2+d_2^2+2l_1{l}_{2}cos{q}_2)$, $M_{12}=m_2{d}_2^2+I_2+m_2{l}_2{d}_{2}cos{q}_2$, $M_{21}=m_2{d}_2^2+I_2+m_2{l}_2{d}_{2}cos{q}_2$, $M_{22}=m_2{d}_2^2+I_2$.

$C\in {\mathbb{R}}^{2\times 2}$ is the centripetal–Coriolis matrix, $C=\left[\begin{array}{cc}{C}_{11}& C_{12}\\ C_{21}& C_{22}\end{array}\right]$, where $C_{11}=-m_2{l}_1{d}_{2}sin{q}_2{\dot{q}}_2$, $C_{12}=-m_2{l}_1{d}_{2}sin{q}_2({\dot{q}}_1+{\dot{q}}_2)$, $C_{21}=m_2{l}_1{d}_{2}sin{q}_2{\dot{q}}_1$, $C_{22}=0$.

$G\in {\mathbb{R}}^2$ is the gravitational torque, $G=\left[\begin{array}{c}{G}_1\\ G_2\end{array}\right]$. where $G_1=m_1{d}_{1}gcos{q}_1+m_{2}g(d_{2}cos(q_1+q_2)+l_{1}cos{q}_1)$, $G_2=m_2{d}_{2}gcos(q_1+q_2)$.

In these equations, $l_1$ denotes the distance from the ankle joint to the knee joint, and $l_2$ denotes the distance from the knee joint to the mass center of human body. $m_i$ denotes the mass of i-th limb, $d_i$ represents the distance from the joint to the center of the i-th limb, $I_i$ represents the moment of inertia of i-th limb, g is the gravity, and i = 1, 2. The mathematical notations and definitions are shown in

Table 1. Mathematical notations and definitions.

Notations	Definitions	Notations	Definitions	Notations	Definitions
M	inertia matrix	$q_1$	angle for ankle joint	$R_j$	amplitude
C	centripetal–Coriolis matrix	$q_2$	angle for knee joint	$g_0$	constant
G	gravitational torque	g	the gravity	$R_0$	total offset
q	real angle of joints	$q_d$	desired trajectory	$q_{i0}$	raw trajectory
$\tau$	control torque	$q_{di}$	desired trajectory	T	gait cycle
			of i-th joint
${\tau }_e$	interaction torque	$r_0$	total offset	J	cost function
$m_1$	mass of 1-th limb	$r_j$	amplitude	$\lambda$	weight
$m_2$	mass of 2-th limb	${\omega }_j$	frequency	$\tilde{Y}$	estimation error
$d_1$	distance from joint	t	time	$\Gamma$	gradient descent
	to center of 1-th limb
$d_2$	distance from joint	$s_j$	phase	$k_{cl}$	gain
	to center of 2-th limb
$l_1$	distance from ankle	$g_j$	constant	$y_p$	auxiliary output
	joint to knee joint
$l_2$	distance from knee	${\Omega }_j$	frequency	$\Phi$	auxiliary regressor
	joint to mass center of
	human body
$I_1$	moment of inertia	$S_j$	phase	$\Psi$	auxiliary regressor
	of 1-th limb
$I_2$	moment of inertia	${\psi }_{ms,0}$	initial value	L	observer gain
	of 2-th limb
$W_{past}$	previous parameter	$C_d$	damping matric	$P_{\tau e}$	observer gain
$J_{past}$	previous cost function	$K_d$	stiffness matric	A	constant matrix
$W^{*}$	suitable parameter	$q_r$	reference trajectory	${\sigma }_{\tau e,1}$	upper bound
$v_{na}$	neural activation	e	tracking error	${\sigma }_{\tau e,2}$	upper bound
$s_{emg}$	amplitude of sEMG	r	assistant variable	${\beta }_{\tau e}$	convergence rate
${\alpha }_{na}$	constant	$\overline{v}$	upper bound	${\tilde{\tau }}_e$	estimation error
${\beta }_{na,1}$	constant	$\overline{a}$	upper bound	$V_1$	Lyapunov function
${\beta }_{na,2}$	constant	$\alpha$	constant	V	Lyapunov function
$C_{na,1}$	constant	Y	regressor matrix	$∆V$	Lyapunov function error
$C_{na,2}$	constant	$k_r$	constant gain matrix	${\psi }_{ms}$	pennation angle
$F_{ms}$	neuromuscular force	$\widehat{Y}$	estimation	$l_{ms,0}$	initial value
$l_{ms}$	length	${\widehat{\tau }}_e$	estimation	$v_{ms}$	extension speed

.

2.2. Human-in-the-Loop Optimization

The aim of the proposed method is to adaptively tune the trajectory $q_d$ of leg joints to minimize the cost function J to achieve a balance between the tracking errors and the change in ${\tau }_e$. The principle of this control method is illustrated in Figure 2.

The interface was designed on a computer with Visual Studio 2010 to monitor the change of signals of sensors mounted on the leg prosthesis, sampled at 1 kH. Using this interface, information such as the position, velocity, and torque can be stored and analyzed.

First, the CPG model parameterizes the trajectory of the joints for a fixed walking gait. The trajectory of the joints was reshaped by switching between the different parameter sets. With the continuous variation in the parameters, minor adjustments of the trajectory can be achieved with a single parameter change.

The formulation of the CPG model can be described by (2) and (3).

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{q}_{di}=r_0+{\Sigma }_{j=1}^N{r}_{j}sin({\omega }_{j}t+s_j)\hfill \\ {\dot{\omega }}_j=g_j[{\displaystyle \frac{g_j}{4}}({\Omega }_j-{\omega }_j)-{\omega }_j]\hfill \\ {\dot{s}}_j=g_j[{\displaystyle \frac{g_j}{4}}(S_j-s_j)-s_j]\hfill \\ {\dot{r}}_j=g_j[{\displaystyle \frac{g_j}{4}}(R_j-r_j)-r_j]\hfill \\ {\dot{r}}_0=g_0[{\displaystyle \frac{g_0}{4}}(R_0-r_0)-r_0]\hfill \end{array}\right.\end{array}$$

(2)

where $q_{di}$ denotes the desired trajectory of i-th joint of the leg prosthesis and can be generated from the superposition of the total oscillators. $r_0$ denotes the total offset of the oscillator. N denotes the number of total oscillators. j denotes the number of the definite one. The variables $r_j$, ${\omega }_j$, and $s_j$ are the amplitude, frequency, and phase of the j-th oscillator. $g_j$ and $g_0$ denote the constants and determine the speed of convergence. ${\Omega }_j$, $R_j$, and $S_j$ are the frequency, amplitude, and phase of the j-th sine series function. $R_0$ denotes the total offset of the sine series function. According to the Fourier series expansion, we have

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\Omega }_j=j{\omega }_1\hfill \\ S_j=arctan{\displaystyle \frac{a_j}{b_j}}\hfill \\ R_j=\sqrt{a_j^2+b_j^2}\hfill \\ R_0=a_0\hfill \\ a_j={\displaystyle \frac{2}{T}}{\int }_0^T{q}_{i0}cosj{\omega }_{1}tdt\hfill \\ b_j={\displaystyle \frac{2}{T}}{\int }_0^T{q}_{i0}cosj{\omega }_{1}tdt\hfill \\ a_0={\displaystyle \frac{2}{T}}{\int }_0^T{q}_{i0}dt\hfill \end{array}\right.\end{array}$$

(3)

where ${\omega }_1={\displaystyle \frac{2\pi }{T}}$, T denotes the gait cycle of the leg prosthesis. $q_{i0}$ denotes the raw trajectory of i-th joint. Then, ${\Omega }_j$, $R_j$, and $S_j$ can be calculated. According to the singular perturbation theory, the limit cycle can be converged from the dynamics of the CPG model. ${\omega }_j$, $r_j$ and $s_j$ will converge to ${\Omega }_j$, $R_j$, and $S_j,$ respectively.

Next, the combined cost function was designed as follows:

$$\begin{array}{c}\hfill J={\displaystyle \frac{\lambda }{T}}{\Sigma }_i^n{(q_{di}-q_i)}^2+{\displaystyle \frac{1-\lambda }{T}}\left(\alpha {\Sigma }_i^n{\tau }_{e,i}^2\right)\end{array}$$

(4)

where n denotes the number of total leg joints and i denotes the number of definite joint. $\alpha$ denotes the constant scale gain for ${\tau }_e$. The first term ensures a better tracking performance, and the second term assists the leg in achieving a lower interaction torque during various walking cases. $\lambda \in (0, 1)$ denotes the weights of the two terms. The cost function focuses on the tracking performance with $\lambda =1$ and the strength of ${\tau }_e$ with $\lambda =0$.

Finally, to adaptively minimize the combined cost function J, suitable parameters of the CPG model were searched during the optimization process. $W_{past}$ and $J_{past}$ denote the previous parameter set of the CPG model and associated cost function value. Minimizing (4) can yield a variable and individualized trajectory, assisting the joint in finding a compromise trajectory to balance the effectiveness of the assistance and the wearer’s comfort. This process enables iterative updates of $W_{past}$ and $J_{past}$. A suitable parameter set $W^{*}$ was then searched to accelerate convergence during the optimization process. The proposed optimization framework works effectively with various wearers and tasks.

2.3. Control Development

2.3.1. Neuromuscular Force Based Impedance Method

The neuromuscular model can be formulated into two parts: muscle activation dynamics, and the relationship between neuromuscular force and muscle activation. Muscle activation dynamics are usually built as a second-order damping system.

$$\begin{array}{c}\hfill v_{na}\left(t\right)={\alpha }_{na}{s}_{emg}-{\beta }_{na,1}{v}_{na}(t-1)-{\beta }_{na,2}{v}_{na}(t-2)\end{array}$$

(5)

where $v_{na}\left(t\right)$ denotes the value of neural activation, $s_{emg}$ represents the amplitude of the sEMG channel, and ${\alpha }_{na}$, ${\beta }_{na,1}$, and ${\beta }_{na,2}$ are constant gains. Then we obtain the following:

$$\begin{array}{c}\hfill {\beta }_{na,1}=C_{na,1}+C_{na,2}\\ \hfill {\beta }_{na,2}=C_{na,1}·C_{na,2}\end{array}$$

(6)

where the gains have been limited as follows:

$$\begin{array}{c}\hfill |C_{na,1}|<1,|C_{na,2}|<1\end{array}$$

(7)

$$\begin{array}{c}\hfill {\alpha }_{na}-{\beta }_{na,1}-{\beta }_{na,2}=1\end{array}$$

(8)

The muscle activation was calculated as follows:

$$\begin{array}{c}\hfill a_{na}\left(t\right)={\displaystyle \frac{s_{emg}^{A_{na}{v}_{na}\left(t\right)}-1}{s_{emg}^{A_{na}}-1}}\end{array}$$

(9)

where $A_{na}\in [-3,0]$ determines the degree of nonlinearity. A linear relationship is obtained when $A_{na}=0$. In contrast, an exponential one was obtained when $A_{na}=-3$.

Then, the neuromuscular force $F_{ms}$ can be described as follows:

$$\begin{array}{c}\hfill F_{ms}=F_{ms,max}[f\left(l_{ms}\right)f\left(v_{ms}\right)a_{na}\left(t\right)+f_p\left(l_{ms}\right))]cos\left({\psi }_{ms}\right)\end{array}$$

(10)

where $F_{ms}$ denotes neuromuscular force. $f\left(l_{ms}\right)$, $f\left(v_{ms}\right)$, and $f_p\left(l_{ms}\right)$ are the assistant variables. $F_{ms,max}$ denotes the maximum of $F_{ms}$. $l_{ms}$ and $v_{ms}$ denote the length and extension speed of the muscle fiber, respectively. The pennation angle ${\psi }_{ms}$ can be calculated as

$$\begin{array}{c}\hfill {\psi }_{ms}={sin}^{-1}\left({\displaystyle \frac{l_{ms,0}sin\left({\psi }_{ms,0}\right)}{l_{ms}}}\right)\end{array}$$

(11)

where ${\psi }_{ms,0}$ and $l_{ms,0}$ denote the initial values of ${\psi }_{ms}$ and $l_{ms}$, respectively. To simplify the calculation, we obtain the following:

$$\begin{array}{c}\hfill f\left(l_{ms}\right)=\left\{\begin{array}{cc}{w}_0+w_1·l_{ms}+w_2·l_{ms}^2& 0.5\le l_{ms}\le 1.5\\ 0& otherwise\end{array}\right.\end{array}$$

(12)

where $w_0=-2.06$, $w_1=6.16$, and $w_2=-3.13$ are set to be constants, and let $f_p\left(l_{ms}\right)=e^{10·l_{ms}-15}$, $f\left(v_{ms}\right)=1$.

Then, the sEMG based impedance model can be formulated as follows:

$$\begin{array}{c}\hfill C_d(\dot{q}-{\dot{q}}_d)+K_d(q-q_d)=F_{ms}\end{array}$$

(13)

where $C_d$ and $K_d$ are the damping and stiffness matrices, respectively. (13) allows for the leg joint to reshape the reference trajectory $q_d$. When the change in sEMG increased, $F_{ms}$ was reduced, leading to a large deviation from $q_d$ with lower impedance parameters. In contrast, the impedance parameters returned to their initial values. Then, the reference vector is introduced as follows:

$$\begin{array}{c}\hfill q_r=q_d-K_d^{-1}{C}_d(\dot{q}-{\dot{q}}_d)+K_d^{-1}{F}_{ms}\end{array}$$

(14)

2.3.2. Controller Design

The tracking performance of the leg joints can be measured using the tracking error e. Assistant variable r is designed as follows:

$$\begin{array}{c}\hfill e=q_r-q\end{array}$$

(15)

$$\begin{array}{c}\hfill r=\dot{e}+\alpha e\end{array}$$

(16)

where $\alpha$ is a constant gain. $q_d={[q_{1d},q_{2d}]}^T\in {\mathbb{R}}^2$ is the desired trajectory, and $\parallel {\dot{q}}_d\parallel \le \overline{v}$, $\parallel {\ddot{q}}_d\parallel \le \overline{a}$. $\overline{v}$ and $\overline{a}$ denote the positive bounds. According to (16), we obtain $\dot{q}={\dot{q}}_d+\alpha e-r$ and $\ddot{q}={\ddot{q}}_d+\alpha \dot{e}-\dot{r}$. By substituting the dynamics in (1), we obtain the following:

$$\begin{array}{c}\hfill M\dot{r}=Y-\tau -{\tau }_e-Cr\end{array}$$

(17)

where Y is the regressor matrix of the uncertainty of system.

$$\begin{array}{c}\hfill Y=M({\ddot{q}}_d+\alpha \dot{e})+C({\dot{q}}_d+\alpha e)+G\end{array}$$

(18)

According to (17), the control torque is formulated as follows:

$$\begin{array}{c}\hfill \tau =k_{r}r+e+Y-{\tau }_e\end{array}$$

(19)

where $k_r$ denotes the constant gain matrix. We assume that $\widehat{Y}$ denotes the estimation of Y, and ${\widehat{\tau }}_e$ denotes the estimation of ${\tau }_e$. Hence, the control torque is redesigned as follows:

$$\begin{array}{c}\hfill \tau =k_{r}r+e+\widehat{Y}-{\widehat{\tau }}_e\end{array}$$

(20)

The control torque consists of two terms: the tracking term $k_{r}r+e$ and the estimation term $\widehat{Y}$. $\tilde{Y}$ denotes the estimation error between $\widehat{Y}$ and Y.

$$\begin{array}{c}\hfill \tilde{Y}=Y-\widehat{Y}\end{array}$$

(21)

To obtain an accurate estimation $\widehat{Y}$, the adaptive law is designed as

$$\begin{array}{c}\hfill \dot{\widehat{Y}}=\Gamma r+k_{cl}\Gamma \left(y_p^T{y}_p\right)\tilde{Y}\end{array}$$

(22)

where $\Gamma$ denotes the adaptive gradient descent along the direction of the error reduction. $k_{cl}\Gamma \left(y_p^T{y}_p\right)\tilde{Y}$ denotes the concurrent learning term that drives the adaptive law toward a smaller estimation error. $k_{cl}\Gamma \left(y_p^T{y}_p\right)\tilde{Y}$ guarantees the convergence of the estimation process. $\Gamma$ denotes the positive constant matrix, and $k_{cl}$ is the selectable gain. The auxiliary output $y_p$ function is defined as follows:

$$\begin{array}{c}\hfill y_p=\Phi +{\int }_0^t\Psi d{y}_p\end{array}$$

(23)

The auxiliary regressors $\Phi$ and $\Psi$ are formulated as follows:

$$\begin{array}{c}\hfill \Phi =M\dot{q}-M{\dot{q}}_{pre}\end{array}$$

(24)

$$\begin{array}{c}\hfill \Psi =-\dot{M}\dot{q}+C\dot{q}+G\end{array}$$

(25)

By substituting (20) into (17), the closed-loop error system can be obtained as follows:

$$\begin{array}{c}\hfill M\dot{r}=-k_{r}r+\tilde{Y}-e-Cr\end{array}$$

(26)

Combining (16) and (26), the closed-loop dynamics can be transformed from (21) and (22), as follows:

$$\left[\begin{array}{c}\dot{e}\\ M\dot{r}\\ \dot{\tilde{Y}}\end{array}\right]=\left[\begin{array}{c}r-\alpha e\\ -k_{r}r+\tilde{Y}-e-Cr\\ -\Gamma r-k_{cl}\Gamma \left(y^{T}y\right)\tilde{Y}\end{array}\right]$$

(27)

To eliminate the use of sensors for torque measurement, ${\widehat{\tau }}_e$ was adopted to estimate the interaction force ${\tau }_e$ as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\dot{y}}_{\tau e}=-L{y}_{\tau e}-L[-C\dot{q}-G+P_{\tau e}]\hfill \\ {\widehat{\tau }}_e=y_{\tau e}+P_{\tau e}\hfill \end{array}\right.\end{array}$$

(28)

where ${\widehat{\tau }}_e$ denotes the estimated interaction force. The auxiliary vector $y_{\tau e}$ is designed to make the observer independent of the acceleration information. The observer gains L and $P_{\tau e}$ are formulated as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}L=A^{-1}{M}^{-1}\hfill \\ P_{\tau e}=A^{-1}\dot{q}\hfill \end{array}\right.\end{array}$$

(29)

where A is a constant matrix, and $A^{-1}={\displaystyle \frac{1}{2}}({\sigma }_{\tau e,1}+2{\beta }_{\tau e}{\sigma }_{\tau e,2})I$, $\parallel \dot{M}\parallel \le {\sigma }_{\tau e,1}$, $\parallel M\parallel \le {\sigma }_{\tau e,2}$. ${\beta }_{\tau e}$ determines the convergence rate. By the observation error as ${\tilde{\tau }}_e={\widehat{\tau }}_e-{\tau }_e$, the closed-loop equation of the DOB can be expressed as follows:

$$\begin{array}{c}\hfill {\dot{\tilde{\tau }}}_e=-L{\tilde{\tau }}_e-{\dot{\tau }}_e\end{array}$$

(30)

2.3.3. Stability Analysis

To analyze the stability of the proposed control method, the Lyapunov function was selected as follows:

$$\begin{array}{c}\hfill V_1={\displaystyle \frac{1}{2}}{e}^{T}e+{\displaystyle \frac{1}{2}}{r}^{T}Mr+{\displaystyle \frac{1}{2}}{\tilde{Y}}^T{\Gamma }^{-1}\tilde{Y}\end{array}$$

(31)

where ${\beta }_1{\parallel z\parallel }^2\le V_1\le {\beta }_2{\parallel z\parallel }^2$, $z={[e^T,r^T,\tilde{Y}]}^T$ and the constants ${\beta }_1$ and ${\beta }_2$ are defined as ${\beta }_1=min((1/2),(\underline{m}/2),(1/2){\lambda }_{min}{\Gamma }^{-1})$, ${\beta }_2=max((1/2),(\overline{m}/2),(1/2){\lambda }_{max}{\Gamma }^{-1})$, where $\underline{m}$ and $\overline{m}$ are bounds of M, ${\lambda }_{min}(·)$, and ${\lambda }_{max}(·)$ denotes the minimum and maximum eigenvalues of $(·)$, respectively. The notation $(·)$ denotes the value of the variable. According to (27), the time derivative of (31) can be formulated as follows:

$$\begin{array}{cc}\hfill {\dot{V}}_1=& e^T(r-\alpha e)+r^T(-k_{r}r+\tilde{Y}-e)\hfill \\ \hfill & +r^T({\displaystyle \frac{1}{2}}\dot{M}-C)r-{\tilde{Y}}^T{\Gamma }^{-1}\dot{\widehat{Y}}\hfill \end{array}$$

(32)

By substituting (22), (32) can be rewritten as follows:

$$\begin{array}{c}\hfill {\dot{V}}_1=-\alpha e^{T}e-r^T{k}_{r}r-k_{cl}{\tilde{Y}}^T\left(y^{T}y\right)\tilde{Y}\end{array}$$

(33)

Then, (33) is bounded as follows:

$$\begin{array}{c}\hfill {\dot{V}}_1\le -\alpha {\parallel e\parallel }^2-\parallel k_r\parallel {\parallel r\parallel }^2-k_{cl}\underline{\lambda }{\parallel Y\parallel }^2\end{array}$$

(34)

Assuming $\Lambda =min(\alpha ,\parallel k_r\parallel ,k_{cl}\underline{\lambda })$, (34) can be deduced as follows:

$$\begin{array}{c}\hfill {\dot{V}}_1\le -\Lambda {\parallel z\parallel }^2\end{array}$$

(35)

According to (31), we can $-{\displaystyle \frac{\Lambda }{{\beta }_2}}{V}_1\ge -\Lambda {\parallel z\parallel }^2$. Then, ${\dot{V}}_1\le -{\displaystyle \frac{\Lambda }{{\beta }_2}}{V}_1$. The control system achieves exponential convergence as follows:

$$\begin{array}{c}\hfill V_1(t+∆t)\le V_1\left(t\right)exp(-{\displaystyle \frac{\Lambda }{{\beta }_2}}∆t)\end{array}$$

(36)

where $∆t$ denotes arbitrary time.

The candidate Lyapunov function is extended as follows:

$$\begin{array}{c}\hfill V=V_1+{\tilde{\tau }}_e^T{A}^{T}MA{\tilde{\tau }}_e\end{array}$$

(37)

Substituting (30) into (37), we obtain the time derivative of V as follows:

$$\begin{array}{cc}\hfill \dot{V}=& {\dot{V}}_1-{\tilde{\tau }}_e^T(A+A^T-A^T\dot{M}A){\tilde{\tau }}_e\hfill \\ \hfill & +{\dot{\tau }}_e^T{A}^{T}MA{\tilde{\tau }}_e+{\tilde{\tau }}_e^T{A}^{T}MA{\dot{\tau }}_e\hfill \end{array}$$

(38)

According to the Schwartz inequality and bound of $\parallel {\dot{\tau }}_e\parallel$, (38) can be deduced as follows:

$$\begin{array}{cc}\hfill \dot{V}& \le {\dot{V}}_1-{\lambda }_{min}(\Gamma )\parallel {\tilde{\tau }}_e{\parallel }^2+2\zeta {\lambda }_{max}{\left(M\right)\parallel A\parallel }^2\parallel {\tilde{\tau }}_e\parallel \hfill \\ \hfill & ={\dot{V}}_1-(1-\rho ){\lambda }_{min}(\Gamma )\parallel {\tilde{\tau }}_e{\parallel }^2-\rho {\lambda }_{min}(\Gamma ){\parallel {\tilde{\tau }}_e\parallel }^2\hfill \\ \hfill & +2\zeta {\lambda }_{max}{\left(M\right)\parallel A\parallel }^2\parallel {\tilde{\tau }}_e\parallel \hfill \end{array}$$

(39)

where $\rho \in (0,1)$. Because $V-V_1={\tilde{\tau }}_e^T{A}^{T}MA{\tilde{\tau }}_e\le {\lambda }_{max}{\left(M\right)\parallel A\parallel }^2{\parallel {\tilde{\tau }}_e\parallel }^2$, when $\parallel {\tilde{\tau }}_e\parallel \ge {\displaystyle \frac{2\zeta {\lambda }_{max}\left(M\right){\parallel A\parallel }^2}{\rho {\lambda }_{min}(\Gamma )}}$, (39) can be deduced as follows:

$$\begin{array}{cc}\hfill \dot{V}& \le {\dot{V}}_1-(1-\rho ){\lambda }_{min}(\Gamma ){\parallel {\tilde{\tau }}_e\parallel }^2\hfill \\ \hfill & \le {\dot{V}}_1-{\displaystyle \frac{(1-\rho ){\lambda }_{min}(\Gamma )}{{\lambda }_{max}\left(M\right){\parallel A\parallel }^2}}(V-V_1)\hfill \end{array}$$

(40)

Then, we obtain the following:

$$\begin{array}{c}\hfill \dot{V}-{\dot{V}}_1\le -{\displaystyle \frac{(1-\rho ){\lambda }_{min}(\Gamma )}{{\lambda }_{max}\left(M\right){\parallel A\parallel }^2}}(V-V_1)\end{array}$$

(41)

And, the exponential convergence was formulated as follows:

$$\begin{array}{cc}\hfill V(t+∆t)& -V_1(t+∆t)\hfill \\ \hfill & \le (V\left(t\right)-V_1\left(t\right))exp(-{\displaystyle \frac{(1-\rho ){\lambda }_{min}(\Gamma )}{{\lambda }_{max}\left(M\right){\parallel A\parallel }^2}}∆t)\hfill \end{array}$$

(42)

where $∆V=V-V_1$ achieved exponential convergence. According to (36), $V_1$ also achieves exponential convergence. Hence, $V=V_1+∆V$ also achieved exponential convergence. The stability of the closed-loop system is guaranteed, and exponential convergence is ensured.

3. Experiments and Results

3.1. Experiment Setup

To evaluate the proposed control method for various walking conditions, we designed three walking cases: level-ground walk, uphill walk, and loaded walk. Each experiment was repeated for 10 min. To avoid the impact of human fatigue on the case, there was a 30 min interval between each case. For uphill walking, the tilt angle of the treadmill was changed to imitate different slopes. In the case of the loaded walk, the subject wore different load vests. The load weight was evenly distributed throughout the subjects. A wireless EMG system was used to measure the sEMG signals on the able-bodied legs. During the experiment, the subject first experienced walkingwithout the proposed control method for 10 min and was then allowed to use the proposed method. Each test lasted for 2 min to ensure a steady optimization process, and the data were recorded for 60 s in 2 min.

3.1.1. Study Volunteers

Two participants were recruited for this study. Both the subjects were tested by a psychologist beforehand and agreed to participate in the experiments. They understood all the experimental procedures and signed informed consent forms. Subject 1 was 33-years-old and had a right leg stump for 4 years. Subject 2 was 34-years-old and had a right leg stump for 5 years.

All experimental procedures were approved by the Ethics Committee of the Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine. The experiments were registered at the China Clinical Trial Registration Center, numbered as ChiCTR2000031162. The protocol was authorized, and is numbered as 2019-014 (the date of approval is 17 April 2019).

3.1.2. Robotic Leg Prosthesis

The robotic leg prosthesis consists of a mechanical structure, a control system, and a sensor system. The mechanical structure was constructed using an aluminum alloy and nylon fibers. As shown in Figure 1, the total mass of the leg prosthesis is 4.8 kg. The range of motion of the active knee joint was approximately $0°$ to $120°$. The active ankle joint was about $-45°$–$45$. The motion of the joint was driven using a Maxon DC motor (Maxon Group, Sachseln, Switzerland). The encoder of the DC motor collects the rotation angle data. The length of the leg prosthesis can be changed by regulating the position between the ball nut of the coupler and pyramid connector. The parameters of leg prosthesis were designed as $m_1=1.5$ kg, $m_2=3$ kg, $d_1=0.15$ m, $d_2=0.15$ m, $I_1=1.3$$\mathrm{kg}·{\mathrm{m}}^2$, $I_2=1.9$$\mathrm{kg}·{\mathrm{m}}^2$, $l_1=0.222$ m, and $l_2=0.173$ m.

Data recordings acquired from the sEMG and motor encoder are used in the analysis during experiments. The Trigno wireless wearable sensors of Delsys Co., Ltd., Natick, MA, USA are mounted on the user’s contralateral leg in the experiments to measure the sEMG signal. The joints of the prosthesis are driven by a Maxon dc flat brushless motor EC45. The servo driver Elmo connects with the computer via a CAN bus to control the Maxon EC 45 Power Max brushless motors. The interface was designed on the computer with Visual Studio 2010 to monitor the change in the signals of the sensors mounted on the leg prosthesis, sampled at 1 kH. Using this interface, information such as the position, velocity, and torque can be stored and analyzed. The mechanical limit for the joints of the prosthesis can avoid excessive movement in the experiments.

3.2. Case 1: Various Walking Speed Experiment

3.2.1. Experimental Protocol

In this experiment, the subjects were required to walk from low speed 0.5 ± 0.1 m/s to high speed 1.0 ± 0.1 m/s. In the first 20 s, the walking speed was maintained at 0.5 ± 0.1 m/s. Then, the subject walk from low speed 0.5 ± 0.1 m/s to middle speed 0.7 ± 0.1 m/s for 18 s. Finally, the walking speed was maintained at 1.0 ± 0.1 for 20 s.

3.2.2. Results

The control performance of the proposed method is displayed. Figure 3a,b shows the tracking trajectories of the leg joints. In the first 20 s, the walking gait cycle was approximately 2 s and the walking speed was approximately 0.5 m/s. The gait cycle was changed as 1.5 s for 18 s. Finally, the gait cycle was maintained for 1 s. Figure 3c shows the tracking errors of the leg joints. Despite the change in the walking speed, the tracking errors of the joints do not changed significantly. Figure 3d shows the change in the control torque of the leg joints. Figure 3e shows the change in $\widehat{Y}$. Figure 3f shows the change in ${\widehat{\tau }}_e$.

In

Table 2. Trajectory tracking performance in Case 1.

Subject	Walking Speed (m/s)	MEAN (Degree)	MSE (Degree)
	0.5	3.6	2.3
1	0.7	2.9	2.3
	1.0	3.4	2.6
	0.5	3.3	2.8
2	0.7	2.8	1.7
	1.0	3.1	2.2

, the tracking errors are statistically calculated for the two subjects. The mean errors (MEANs) of the tracking performance were $3.6°$ and $3.3°$, respectively. And the minimized mean square errors (MSEs) of the tracking performance were $2.3°$ and $2.8°$, respectively.

3.3. Case 2: Various Walking Stride Experiment

3.3.1. Experimental Protocol

In this experiment, the subjects were required to walk with various strides. In the first 20 s, the subject walked on level ground with a small stride. Then, the walking stride gradually changed from small to normal and then to large, each lasting for 20 s. The gait cycle was maintained for 2 s.

3.3.2. Results

The control performance of the proposed method is displayed. Figure 4a,b shows the tracking trajectories of leg joints. In the first 20 s, the subject walked on level ground with a small walking stride, and the walking speed was approximately 0.5 m/s. Then, the walking stride gradually changed from small to normal and then to large, each lasting for 20 s. Figure 4c shows the tracking errors of the leg joints. Despite the change in walking stride, the tracking errors of the joints did not seem to change significantly. Figure 4d shows the change in the control torque of the leg joints. Figure 4e shows the change in $\widehat{Y}$. Figure 4f shows the change in ${\widehat{\tau }}_e$.

In

Table 3. Trajectory tracking performance in Case 2.

Subject	Walking Stride	MEAN (Degree)	MSE (Degree)
	small	2.7	1.8
1	normal	2.9	2.3
	large	3.4	2.6
	small	3.1	2.4
2	normal	2.8	1.7
	large	3.1	2.2

, the tracking errors are statistically calculated for the two subjects. The mean errors (MEANs) of the tracking performance are $2.7°$ and $3.1°$, respectively. And the minimized mean square errors (MSEs) of the tracking performance are $1.8°$ and $2.4°$, respectively.

3.4. Case 3: Various Walking Uphill Experiment

3.4.1. Experimental Protocol

In this experiment, the subjects were required to walk from a level ground to different slopes. The slope angle was varied from $3°$ to $5°$ and $7°$, respectively. In the first 20 s, the subject walked at the level ground when the slope angle was $3°$. Then, the slope angle gradually changed from $3°$ to $5°$ and then to $7°$, each lasting for 20 s. The gait cycle was maintained for 2 s.

3.4.2. Results

The control performance of the proposed method is displayed. Figure 5a,b shows the tracking trajectories of the leg joints. In the first 20 s, the slope angle was approximately $3°$ and the gait cycle was approximately 2 s. The slope angle was then increased to $5°$ for 20 s. Finally, the slope angle was maintained at $7°$. Figure 5c shows the tracking errors of the leg joints. With an increase in the slope angle, the load for the joints increased and the tracking errors of the joints changed simultaneously. Figure 5d shows the change in the control torque of the leg joints. Figure 5e shows the change in $\widehat{Y}$. Figure 5f shows the change in ${\widehat{\tau }}_e$.

In

Table 4. Trajectory tracking performance in Case 3.

Subject	Up Slop Angle (Degree)	MEAN (Degree)	MSE (Degree)
	3.0	2.4	1.6
1	5.0	2.9	2.3
	7.0	3.4	2.6
	3.0	2.2	1.3
2	5.0	2.8	1.7
	7.0	3.1	2.2

, the tracking errors are statistically calculated for the two subjects. The mean errors (MEANs) of the tracking performance for Subject 1 and 2 are $2.4°$, $2.9°$, $3.4°$, and $2.2°$, $2.8°$, $3.1°$, respectively. The minimized mean square errors (MSsE) of the tracking performance for Subject 1 and 2 were $1.6°$, $2.3°$, $2.6°$, and $1.3°$, $1.7°$, $2.2°$, respectively.

4. Discussion

This paper proposed an adaptive human–robot concurrent control scheme that achieves an appropriate gait trajectory for a robotic leg prosthesis to improve the wearer’s comfort in various tasks. To accommodate wearer’s comfort, a neuromuscular-force-based impedance method was developed using muscle activation to reshape gait trajectory. To eliminate the use of sensors for torque measurement, a disturbance observer was established to estimate the interaction force between the human residual limb and the prosthetic receptacle. To incorporate the tracking performance of the joints and the user’s comfort caused by the contact force between the stump and leg prosthesis, a combined cost function was designed. The control scheme was primarily based on human-in-the-loop optimization to search for a suitable control weight to regenerate the appropriate gait trajectory. The major contributions of this study are outlined as follows: (1) An individualized approach for the trajectory of a leg prosthesis was developed. A Central Pattern Generator (CPG) was used to parameterize the leg joint trajectory. (2) To accommodate different wearers, a neuromuscular-force-based impedance method was developed using muscle activation to reshape gait trajectory. (3) The cost function was combined with the interaction force and tracking errors of the joints. The control scheme was mainly based on human-in-the-loop optimization to search for a suitable control weight for the CPG model to regenerate the appropriate gait trajectory.

5. Conclusions

The aim of this study was to regulate the parameters of the CPG model to find a suitable trajectory for the leg prosthesis using human-in-the-loop optimization. To realize the natural regulation of the leg joint trajectories on the lower-limb prosthesis, the neuromuscular-force-based impedance method was used for different walking conditions. The cost function was combined with the interaction force and tracking errors of the joints. We aimed to reduce the cost function by minimally changing the control weight of the gait trajectory generated by the Central Pattern Generator (CPG). Human-in-the-loop optimization was proposed to adaptively regulate the parameters to minimize the cost function. To handle the uncertainties and unknown coupling of the motors, an adaptive law was designed to estimate the unknown parameters of the system. Using a stability analysis, the control framework was verified by semi-globally uniformly ultimately bounded stability. The experimental results were discussed, and the effectiveness of the adaptive control framework was demonstrated. Future research will focus on bipedal robotic leg prosthesis adaptive control and human-in-the-loop optimization.