1. Introduction
Hybrid exoskeletons are rehabilitation tools for disorder movements and they are comprised of two main parts: (1) exoskeleton and (2) functional electrical stimulation (FES). The exoskeleton moves the joints mechanically using its actuators, which are motors. FES stimulates muscles with an electric current [
1]—it sends current pulses to muscles and moves the joints by the stimulated muscles [
2]. Although muscle electrical stimulation improves rehabilitation by stimulating the nervous system, it causes muscle fatigue and shortness of rehabilitation [
3]. Hybrid exoskeletons can also be categorized in a general form, consisting of (1) multi-joint and single-joint lower limb hybrid exoskeletons and (2) multi-joint and single-joint upper limb hybrid exoskeletons [
4]. One single-joint lower limb hybrid exoskeleton is the knee hybrid exoskeleton, which rehabilitates the knee joint [
5]. The knee hybrid exoskeleton moves the knee joint towards the desired path or regulates it at the desired angle. In this paper, the knee hybrid exoskeleton is studied.
The automatic operation of hybrid exoskeletons is one of their essential features. Closed-loop controllers for electrical motors and FES play a central role in automating these devices [
6]. The control objective of the hybrid exoskeletons is to solve the tracking or regulation problem of joints. The most critical control challenge for these systems is the loss of system performance due to decreased actuators’ efficiency, i.e., stimulated muscles, due to fatigue. The control of the knee hybrid exoskeletons is a research topic of interest; various studies have been conducted to control this device [
7].
Switching control is one of the techniques employed to prevent the increase of fatigue in hybrid exoskeletons [
8,
9,
10,
11,
12,
13,
14], wherein a class of switched systems is presented to describe switched hybrid exoskeletons [
12]. In the design of the switching law for the lower limb hybrid exoskeletons, consideration is given to the threshold fatigue value, system stability, and attraction region of the controllers [
9,
11,
12]. In [
8,
10], the fatigue region method is used for switching in the knee hybrid exoskeleton. In this method, two minimum and maximum threshold values for fatigue are defined. If the fatigue exceeds the maximum threshold, the switch will be made to the motor controller, and if the fatigue is below the minimum threshold, the switch will be made to the FES controller. The practical implementation constraints of the switching method are the possibility of chattering, delayed actuators, and physical limitations.
Hierarchical control is another method used to overcome the fatigue challenge [
15,
16,
17]. In this method, a closed-loop control signal is calculated. Then, this signal is distributed between the motor and FES controllers by solving the optimization problem, one of the constraints of which is fatigue reduction. The computational complexity and the time-consuming nature of solving the optimization problem make implementing this method challenging. Determining the value of the local extremum instead of the global extremum can prevent muscle fatigue reduction by improper control distribution.
Model predictive controllers (MPCs) are used to regulate knee hybrid exoskeletons. These controllers use optimization methods to calculate the motor and FES control signals in a time frame [
18,
19,
20,
21]. One of the goals in the control signal distribution between the FES and the motor is to reduce muscle fatigue. Different MPCs regulate the knee hybrid exoskeleton, such as tube-based, penalty barrier function, and nonlinear MPC. In [
19], a heuristic method is presented in the motor and FES control distribution to reduce muscle fatigue; however, in the results, a significant muscle fatigue increase is observed. In studies based on MPC methods, the muscle fatigue increase is significant [
18,
20,
21].
Learning-based methods are used to estimate parameters or identify the model of hybrid exoskeletons. A learning-based controller is designed to overcome muscle fatigue [
22]. A recurrent neural network, with supervised learning, identifies the system model, and a feedforward neural network, with a learning rule based on reinforcement learning, designs the controller. Neural network training requires a significant amount of data, which are not easily accessible. The training phase in learning-based methods may cause risks for the patient, and maintaining safety creates limitations in the learning process.
In some studies, the knee joint is controlled along with the hip joint or the ankle joint in cases where multi-joint exoskeletons of the lower limb are used for sitting−standing or walking. The sliding mode control has successfully controlled the joints, but the effect of the proposed method on muscle fatigue still needs to be investigated [
23]. In [
24], muscle synergies are used to control the hybrid exoskeleton. The complexity of muscle synergies calculation in practical implementation and the high dependence of this method on the system model are its disadvantages. A cooperative controller with a finite state machine is used to deal with fatigue [
25,
26,
27]. The use of a finite state machine does not match the automation of the system. This method needs to be stronger for facing disturbance and environmental changes.
The studies used before to deal with muscle fatigue have some drawbacks: (1) In MPC-based methods, to face the muscle fatigue challenge, the control distribution between the motor and FES is achieved by solving the optimization problem, but the muscle fatigue increase in the obtained results is significant [
18,
19,
20,
21]. (2) In hierarchical control and MPC-based methods [
15,
16,
17,
18,
19,
20,
21], which use optimization to prevent muscle fatigue increase, sub-optimal control values may be calculated at the boundary points instead of optimal values. In this way, reducing muscle fatigue is not optimal, and preventing the increase of muscle fatigue is associated with errors. (3) In the reported results of hierarchical control methods, for people with spinal cord injury, the muscle does not respond to FES, the muscle fatigue is not calculated, and it is impossible to determine the FES control signal weight to decrease fatigue [
15,
16,
17]. (4) In the methods based on switching, to compensate for muscle fatigue, FES is deactivated in specific time intervals; as a result, it becomes impossible to use the therapeutic benefits of electrical stimulation in those intervals [
8,
9,
10,
11,
12,
13,
14]. (5) In addition, in switching-based methods, when FES is active, it is impossible to regulate muscle fatigue, and the amount of fatigue increases. (6) In learning-based methods, a large amount of data are needed to train the controllers. The controller is trained for simple movements, and it is not easy to access the training data set with various movement patterns [
22]. In addition, the training stage is time-consuming and tiring for the patient. The disadvantages of the hybrid exoskeletons control methods, when dealing with muscle fatigue, include the optimization and learning challenges and the stimulation deactivation in switching. It is important to use a control method that prevents muscle fatigue increase by regulating it in a desired value, thus allowing the therapeutic advantages of FES usage without deactivating the electrical stimulation.
FES is important in hybrid exoskeletons. Electrical stimulation by stimulating the nervous system plays an influential role in the patient’s sensory and motor rehabilitation. Electrical pulses stimulate sensory fibers, especially la fibers, that synapse with
—motoneurons in the spinal cord. In addition, motor neurons carry the electrical signal in the muscle and cause a muscle reflex. The release of calcium ions,
, in muscle cells causes muscle recovery, and high-frequency electrical stimulation disrupts the calcium production in muscles and causes muscle fatigue [
28]. In the proposed method of our study, it is possible to use sensory and motor stimulation of FES without increasing fatigue. Muscle fatigue regulation can prevent the need to stop the stimulation in rehabilitation.
Adaptive control is one of the effective methods for controlling hybrid exoskeletons [
29]; due to the non-linear and time-varying nature of muscles, adaptive control is a suitable method for controlling FES [
30]. In this article, the adaptive controller is used to control muscle activation. The muscle fatigue variable cannot be regulated directly with the control signal. Muscle activation is controllable through a control signal, and muscle fatigue is affected by the muscle activation variable; so, in this study, muscle activation control was used to regulate muscle fatigue. This study presents an innovative cooperative structure to control muscle activation and the knee joint’s angle. In this structure, a proportional-integral (PI) controller controls the joint, whose gains are determined by the optimal control method. The proof of the uniformity ultimately boundedness (UUB) of the muscle activation error is done by the Lyapunov analysis method, which determines the bound of this error. The simulation of the proposed method is conducted on a model of a knee hybrid exoskeleton. The knee joint angle and the muscle activation are regulated to the desired references. Muscle fatigue is also regulated, and increased fatigue is prevented. The proposed method is compared with the model prediction control and switching control methods, which provides the best results in joint angle error and muscle fatigue value.
In
Section 2, the description of the problem is presented. In this section, the dynamic models of the joint, muscle, and knee hybrid exoskeleton state space model are discussed. In
Section 3, the presented method is elucidated. In this section, the design of joint and muscle activation controllers is presented, the mathematical correlation between the muscle activation and muscle fatigue is investigated, and the system stability is demonstrated through the utilization of a muscle activation controller. In
Section 4, the simulation of the proposed method is conducted on a knee hybrid exoskeleton model, the results are analyzed, and the efficiency of proposed method is evaluated. In
Section 5, the discussion is presented. In this section, the proposed method’s performance, its advantages, and its clinical application privileges are discussed, and the limitations of the proposed method are pointed out. In
Section 6, conclusions are made based on the previous sections, and future works are introduced.
2. Methods and Materials
The two primary actuators of the knee hybrid exoskeleton are the motor of the exoskeleton and the muscles stimulated by the electrodes connected to the FES;
Figure 1 shows a picture of the knee hybrid exoskeleton and its actuators. To describe the system, equations that describe the dynamics of muscles and joints based on system inputs are needed [
31]. The joint movement is caused by torque, the torque produced by the exoskeleton motor, and the torque produced by the electrical stimulation of the muscles.
Figure 2 shows a diagram of the knee hybrid exoskeleton, control signals, torques, and knee joint angles. FES sends an electrical current,
, through electrodes to the quadriceps muscles. As a result of stimulation, the muscles generate torque,
, and move the knee. The electrical motor produces mechanical torque,
, in the knee and moves it. The joint moves with
angle, and the sensor transfers the
value to the controller, which is a computer or microcontroller. Based on the joint angle error, the controller calculates control signals for the motor,
, and FES,
. The controller sends these signals to the motor and FES to regulate the joint angle and muscle activation.
is the knee equilibrium angle relative to the vertical axis [
32]. Parameter,
, is the knee anatomical angle and is calculated by Equation (1).
In
Figure 2, the center of mass of the shank, to which the
force is applied, is specified;
, are the mass of the shank and the acceleration gravity constant.
, is the distance from the center of mass to the knee [
32].
2.1. Joint Dynamic Equation
The dynamic equation of the knee joint, according to Equation (2), is described by the Euler−Lagrange equation [
33]. The equation’s inputs are the torques of the system,
and
. The equation outputs are joint angle,
; angular velocity,
and angular acceleration,
. The constant parameter,
, is the shank’s moment of inertia.
The torque created by applying gravity to the center of mass of the shank at the knee joint is the gravitational torque,
. Equation (3) shows the expression of gravitational torque.
The passive dynamics of muscles, ligaments, and tendons create torque in the knee, which is called passive torque and is modeled by
. Expression (4) shows passive torque;
, is a function of the knee anatomical angle. The parameters
and
are patient-specific and model the stiffness and damping of the knee joint [
32].
The torque caused by the stimulated muscles,
, is expressed in Equation (5); this torque is created in the knee extension during electrical stimulation [
31]. The term
is the torque length, and the term
is the torque−velocity relationship. Parameters
are the patient-specific constants. The term
is the muscle activation, and
is the muscle fatigue obtained from the muscle dynamics equations.
2.2. Muscle Dynamic Equations
The dynamics of the stimulated muscle are described by two parameters of muscle activation and muscle fatigue [
34,
35]. The reason for the muscle stimulation is the electric current applied to it; Equation (6) expresses the electric current expression,
.
is the minimum current amplitudes that cause movement in the knee joint and is the minimum current amplitude that causes maximum muscle contraction.
Dynamic of muscle activation,
, is expressed in Equation (7). The electrical stimulation control signal,
, influences the dynamic of muscle activation. The parameter,
, is the muscle activation time constant and is proportional to the duration of muscle stimulation.
The muscle fatigue,
, dynamic is described by Equation (8). The expression,
, is the recovery term and the term,
is the fatigue term. The parameter,
, is the recovery time constant, and parameter,
, is the fatigue time constant.
The parameter , determines the lower bound of muscle fatigue. , is equivalent to the non-fatigue state and , is equivalent to maximum fatigue of the muscle.
2.3. State Space Equations
Joint and muscle dynamics describe the state space of the knee hybrid exoskeleton [
31]. State space variables are,
. The input signals are
, which is
. To determine
, the expression (9) is written using Equation (2).
By selecting
, the expression
is obtained. The state space equations of the knee hybrid exoskeleton are written according to (10).
5. Discussion
5.1. Proposed Method Assessment
In this study, the regulation of muscle activation, muscle fatigue, and joint angle in the knee hybrid exoskeleton is provided by designing two controllers for muscle and joints in an innovative cooperative structure. Based on the simulation results, the proposed method can be clinically implemented to rehabilitate knee extension movement by stimulating the quadricep muscles through the hybrid knee exoskeleton. The innovations and contributions of the proposed method can be expressed as follows.
Innovations: (1) The control structure is designed for the knee hybrid exoskeleton in which the joint and muscle are controlled separately. (2) The estimation rule is designed to determine the time-varying parameter in the dynamic model of muscle activation. (3) The innovative adaptive controller is designed to regulate muscle activation. The control law consists of the state feedback and output error terms. The adaptation law is created by estimating the muscle activation time parameter. (4) The optimal PI controller is designed to control the knee joint; separate control of muscle activation causes disturbance to the joint, the PI controller, making the joint robust against that disturbance. (5) The relationship between muscle fatigue and muscle activation is computed, then muscle fatigue is regulated by muscle activation control.
Contributions: (1) This study presents the control method for the knee hybrid exoskeleton that automatically regulates knee joint angle and muscle activation. (2) In this study, the regulation of muscle fatigue is presented as the solution to increasing muscle fatigue due to electrical stimulation. Muscle fatigue regulation avoids stopping FES in knee rehabilitation due to increased muscle fatigue. (3) The muscle activation Hill-type-based model is improved by replacing the time-varying parameter instead of the muscle activation time constant and this replacement eliminates the pretest stage to estimate the parameter of the muscle activation model [
35]. (4) The muscle control method’s stability is proven by Lyapunov analysis, and the error bound of muscle activation is determined.
5.2. Proposed Method Advantages in Comparison with Other Methods
In iterative learning controller (ILC) methods, the joint tracking error in initial iterations is significant [
11,
13,
17]. The time required to learn the controller is time consuming, which can hurt the patient psychologically. Part of the treatment session time is spent on controller learning, and treatment effectiveness is reduced [
44]. A large volume of data are not readily available for patients who have undergone rehabilitation using a hybrid knee exoskeleton. The lack of device testing results and the legal restrictions on access to personal health data make it challenging to collect this data. Therefore, training neural networks for model identification or controller design and validating the neural network methods in hybrid exoskeletons is challenging. The proposed method can control the knee joint hybrid exoskeleton without requiring a large amount of data and without iteration errors, which is the advantage of the proposed method over learning-based methods.
Methods based on model predictive and hierarchical control must solve complex optimization problems. Utilizing a hybrid exoskeleton faces a severe challenge in the laboratory implementation of these calculations. The delay in the experimental applications challenges the efficiency of lengthy and time-consuming calculations. The need for high-level hardware to quickly perform complex calculations is expensive and increases the cost of hybrid exoskeleton rehabilitation. The proposed method can be implemented on standard microcontrollers and average computers without complex and time-consuming calculations, which is the advantage of the proposed method over model predictive and hierarchical controllers.
The numerical evaluation of the proposed method, with other hybrid exoskeleton control methods, is completed by comparing the results of this study with the results of nonlinear model predictive control (NMPC) [
20], linearized model predictive control (Linearized MPC) [
20], shared iterative learning control (Shared ILC) [
17], and Adaptive Low dimension control (ALDC) [
45], based on
Table 10. The evaluation is performed on the two main hybrid exoskeleton performance indexes: RMS error of angle and muscle fatigue. The improvement percentage for RMS error of angle and the muscle fatigue converged value by the proposed method of this study is determined compared with other methods. Compared with the other methods presented in
Table 10, the main advantage of the proposed method is that muscle fatigue can be regulated at any desired value; in
Table 10, the desired fatigue is valued as
. In all cases in
Table 10, the extension movement of the knee joint is considered.
Compared with the NMPC, Linearized MPC, Shared ILC, and ALDC methods, the proposed method resulted in 63%, 91.47%, 90.05%, and 93.26% improvement in angle RMS error, respectively. In addition, the proposed method led to 18.75%, 35.71%, and 26.66% improvements in muscle fatigue compared with NMPC, Linearized MPC, and ALDC, respectively. It is imperative to clarify that despite the requirement for a clinical examination for the validation of the proposed approach, wherein factors such as the nature and extent of the patient’s injury have a significant impact on the outcomes, this comparison served to validate the efficacy of the proposed control approach from a theoretical standpoint and to establish the significance of its clinical evaluation.
Regulating muscle fatigue at a desired reference is difficult in switching-based methods. In switching control methods, muscle fatigue is limited in a region, but fatigue also increases in a region. In addition, switching faces limitations such as chattering and actuator depreciation due to many switches, but the proposed method prevents chattering and ensures the safety of actuators. The simulation results of the proposed method and switching-based methods are discussed to further investigate the comparison of these methods.
The proposed method has a 99% improvement in the knee joint angle steady state error compared with the hybrid PID switching method. In the simulation results in
Figure 15, intermittent fluctuations in the joint angle are observed in a steady state when the FES is active for the hybrid PID switching method. These fluctuations increase the steady state error of the hybrid PID switching method. In the method proposed in this study, the oscillations are prevented by separating the muscle controller from the joint controller. Compared with the hybrid PID method, smooth knee joint regulation is an important advantage of the proposed method for clinical applications. The proposed method results in an 84% improvement in RMS error compared with the VGSTSMC switching method. The VGSTSMC switching method to reach the sliding surface has a significant error in the transient state. In the transient state, the proposed method smoothly passes the transient phase and performs better than the VGSTSMC switching method. The vital advantage of the proposed method in knee joint regulation is its smooth movement. The presence of intermittent oscillations, such as those observed in the hybrid PID switching method, is annoying for the patient in practical applications. In addition, overshooting to reach the sliding surface, as seen in the VGSTSMC switching method, is challenging for a patient with mobility disorders.
The most important feature of the proposed method is the regulation of muscle fatigue at a desired value. Switching methods are also used to keep muscle fatigue in a specific region and to prevent the decrease of from a threshold value of . The value is considered the threshold value in the simulations. The hybrid PID switching method has not succeeded in preventing the decrease of from the threshold value, and by decreasing the value of , the switch from FES to the motor occurs at . Although the difference in the value that the FES switches to the motor does not have a significant difference with the threshold value, in cases where the muscle fatigue threshold is defined at the border of complete fatigue, it can be challenging to perform the switch at a value lower than .
In this study, the average and variance of
were used for the switching analysis, based on
Table 9, without counting the switch numbers. A low variance for a mean value close to the fatigue threshold indicates more switches. The VGSTSMC switching method maintained the value of
at the fatigue threshold, but the number of switches was enormous. In practice, implementing these switch numbers faces challenges due to limitations such as electromechanical delay and depreciation of actuators. The proposed method maintained the
value higher than the fatigue threshold throughout the simulation. In the VGSTSMC switching method, muscle fatigue increased very quickly, but in the proposed method, the fatigue increased gently and did not decrease from the threshold value. In the hybrid PID switching and VGSTSMC switching methods, when
was less than the threshold value, FES was disabled, and
, according to
Figure 17. Muscle activation due to electrical stimulation is one of the main goals of rehabilitation. In
Figure 17, it can be seen that by using the proposed method, the muscle activation was regulated at a desired value. In addition, by regulating muscle fatigue, there was no need to deactivate the FES, and the primary goal of rehabilitation was achieved.
5.3. Proposed Method Advantages in Clinical Implementation
The proposed method has a significant advantage over other hybrid exoskeleton control methods. In other methods, to estimate the time constant of muscle activation, it is necessary to perform a pre-experiment stage [
19,
21]. This stage prolongs the rehabilitation time and can cause physical and mental fatigue in the patient [
37]. The proposed method eliminates the pre-experiment stage for estimating the muscle activation time constant. Therefore, the practical use of the hybrid knee exoskeleton becomes more convenient. Adaptive controller design utilizing the model parameter time-varying estimation as an adaptation law increases the possibility of a patient’s personalized rehabilitation.
Although the pre-experiment stage is still needed to determine the recovery time and fatigue time parameters, this stage can be eliminated by time-varying estimating these parameters. Due to muscle activation’s controllability and the importance of muscle activation in determining muscle fatigue, this article focuses on muscle activation control.
Muscle fatigue desired value regulation and muscle activation error bound prediction can help therapists design a more accurate plan for the patient’s rehabilitation. The therapist can plan the number of sessions, the duration of each session, and the intensity of the patient’s exercises by simulating the proposed method before rehabilitation. In addition, the therapist can mix the knee hybrid exoskeleton, which the proposed method controls, along with other rehabilitation methods in therapy processes [
46].
5.4. Limitations
As a result of the presence of delay in natural systems, the effect of delay on the proposed method is a limitation in its implementation [
12]. This study did not consider the disturbance’s effects on the system’s performance [
47]. The existence of disturbances, especially in biological systems, is another challenge to the proposed method’s implementation. The expert supervision necessity for practical testing is another limitation in the proposed method implementation. Various factors, such as mental factors, affect muscle fatigue; these factors differ from one individual to another depending on the rehabilitation conditions [
28]. The unmodeled muscle fatigue factors are another limitation of the proposed method’s implementation [
48,
49]. Based on the simulation results, the proposed method can be implemented, but for more accurate evaluation, it is necessary to test it in a laboratory and on sufficient subjects.