2.2. Multi-Source Signals Acquisition Hardware
Input data in this work include lower limb joint angle signals, plantar pressure signals and sEMG signals. The hardware characteristics required for signal acquisition will be introduced next.
To collect lower limb joint angle signals, the JY61 six-axis angle sensor (Witmotion Shenzhen Co. Ltd., Shenzhen, China) with Kalman filtering algorithm is used in this paper, as shown in
Figure 2. There are two communication modes that can be selected: serial port communication and I2C communication. In order to cooperate with the microcontroller, serial port communication is selected for this topic. The TX, RX, VCC, and GND pins corresponding to the serial communication are used to connect to the microcontroller.
Three angle signals have to be collected at the same time, including hip joint, knee joint and ankle joint. The Arduino MEGA 2560 (Hesai Shenzhen Co. Ltd., Shenzhen, China) has four serial ports to meet the experiment requirements. Therefore, Arduino MEGA 2560 is used for angle signals collection. The programming interface is shown in
Figure 3. In order to facilitate the later analysis of the data, PLX data acquisition (PLX-DAQ) is taken to communicate with the microprogrammed control unit (MCU) through the virtual serial port, then the angle signals are recorded into the custom excel file in real time. The interface of PLX-DAQ is shown in
Figure 4.
In this paper, the selected pressure sensor and the insole are combined to make the pressure insole, so that it can measure the pressure signals from the bottom of the feet. The pressure sensor model is IMS-C20B (Aidong Wuxi Co. Ltd., Wuxi, China), which has the advantages of low price, small volume and high sensitivity.
Table 1 shows the specific parameters.
The selected pressure sensor must be connected in series with the resistance for signal acquisition. To get the appropriate resistance, debugging of the hardware circuit had to be finished on the breadboard, as shown in
Figure 5.
Through the analysis of numerous debugging results, the resistance of 10 kΩ was the best, under lower error and higher sensitivity conditions. Finally, the pressure sensors were connected in series with a 10 kΩ resistance and placed in the insole to make a pressure insole. In this work, the insoles were put into the sneakers that people usually wear to perform the experiment, as shown in
Figure 6, so that the results are more effective.
For sEMG signals acquisition, the MyoWare EMG sensors (Advancer Technologies Littleton Company, Littleton Colorado, CO, USA) were used to measure EMG signals of thigh muscles. The structure of MyoWare EMG sensor is shown in
Figure 7. Then the universal serial bus data acquisition (USB DAQ) card (HKTECH Zhengzhou Co. Ltd., Zhengzhou, China) was taken to collect the sEMG signals through analog ports. In terms of power supply, the DAQ card was powered by the personal computer (PC) through USB cable, and the EMG sensors were powered by the DAQ card.
Figure 8 shows the structure of the DAQ card.
For data storage, LABVIEW was used to write a signal acquisition program and then data was collected in a custom file for subsequent data processing. The front panel and block diagram of LABVIEW are shown in
Figure 9. The effective frequency of the EMG signal is mainly distributed in the range of 10–500 Hz; according to Nyquist’s sampling theorem, the sampling frequency must be set to at least 1000 Hz to ensure the validity of the collected signal. In this work, the sampling frequency of each channel is set to 2000 Hz.
2.3. Data Acquisition
Human movement of walking on level ground generally occurs in the sagittal plane, so the pitching angle during the lower limb swing process is collected as the angle signal. According to systematic anatomy, the muscles related to hip joint swing are rectus femoris (RF), biceps femoris (BF) and semitendinosus (ST), etc. Therefore, in the case of the absence of lower leg and foot, the sEMG signal of these three major muscles is collected. The input and output functions of the prediction model can be expressed as follows:
where
is hip joint angle, and
is other motion information (such as sEMG signal). The current and previous angle signal and sEMG signal are taken as inputs to estimate the joint angle at the next time.
Throughout the study, it is found that if the hip angle is only used to predict the lower limb joint angle at the next moment, the performance is poor, so the sEMG signals are taken into the input data to improve the output. However, there are still errors between the prediction results and the actual data in the transition position between the support and swing phase. It is worth considering that during the walking process, the plantar pressure also contains information about the movement of the lower limbs, which is closely related to the gait phase. Thus, the plantar pressure is added into input data, including three locations where the pressure is concentrated: The pressure at big toe, forefoot and heel. In conclusion, the input signals of the neural network include hip joint angle, sEMG signal of three major muscles, and three plantar pressure signals.
2.4. Data Pre-Processing
The angle signals cannot be measured simultaneously with the EMG signals. Therefore, the angle signals and pressure signals were synchronously collected, and the EMG signals and the pressure signals were synchronously collected. Based on the number of cycles of plantar pressure signals, the angle signal and the EMG signals could be time synchronized together.
The hardware used to collect the angle signal is Arduino Mega. However, when the microcontroller communicates with the upper computer, because of the rate limit of serial port communication, the real sampling frequency is affected and the frequency is also different from that of the sEMG signals. So it is necessary to preprocess the angle signal. The processing methods are as follows: fitting the angle signal every 20 sampling points into a time curve, and then resampling the fitting results with the sampling frequency of the pre-processed sEMG signal.
The plantar pressure signal is also subjected to window processing every 20 sampling points. The average value in each window is taken as the feature value of the signal.
Because of the interference of external environmental factors such as temperature and so on, the original sEMG signal contains a lot of noise, causing a problem where the signal cannot be directly used for data prediction. Therefore, it needs be rectified and filtered to finish feature extraction. The useful EMG signal power spectrum is distributed between 30–300 Hz. The energy is mainly concentrated between 20–500 Hz. So, in this work, a Butterworth band-pass filter is designed to filter sEMG signals.
Table 2 shows the major parameters of band-pass filter. There are many sEMG feature values in the time and frequency domains, such as integrated EMG (iEMG), root mean square (RMS), mean power frequency (MPF), median frequency (MF), and so on [
20,
21]. The RMS that can reflect the energy of the signal is chosen in this paper, which is often used to evaluate muscle activity in real time without injury [
22]. The formula is as follows:
where
is the
ith sample point of sEMG filtered by band-pass filter, and
N is the number of the sample.
Finally, in order to improve the training and convergence speed of the neural network and reduce the impact of amplitude differences on the results, all signal data are normalized and the equation is:
where
is the signal amplitude,
is the normalized amplitude of signal, and
and
are the maximum and minimum of the signal, respectively. The sEMG signal and joint angle signals are normalized into [0, 1].
2.5. GRNN Network Training and Prediction Results
GRNN is a kind of radial basis neural network, which has pretty good properties such as nonlinear mapping, approximation functions, quick convergence speed, and high prediction accuracy. With less sample data, the performance is also quite good. Besides, the network can also process unstable data. The Gaussian kernel function is taken as the transfer function of GRNN, and the network consists of four layers. As shown in
Figure 2, there is an input layer, pattern layer, summation layer, and output layer. The input is
X = [
x1,
x2,
x3, …,
xn]
T, and the corresponding output is
Y = [
y1,
y2, …,
yk]
T.
Suppose that the joint probability density of random variable
x and
y is
f(
x,
y), and given that the observed value of
x is
X, so the regression of
y with respect to
X, that is, the prediction output of the neural network is:
where
is the prediction result of
Y under the condition that the input is
X;
Xi and
Yi are the
ith observation value of
x and
y respectively;
n is the number of samples;
σ is smooth factor (
σ > 0); and
is the square of the Euclidean distance between
X and
Xi.
The transfer function of the pattern layer is:
One unit in the summation layer is the denominator of Equation (1), and its transfer function is:
The other unit in the summation layer is the numerator of Equation (1), and its transfer function is:
Thus, the result of the output layer is:
The only factor that influences the performance of the network is the smoothing factor
σ, which needs to be optimized by one-dimensional optimization method. Establishing the mean square error as the objective function between prediction results of joint angles
Yi and actual data
Y to get the optimal smoothing factor:
where
N is the number of samples. To quickly get the optimal smooth factor, the GS algorithm is used for optimization. The algorithm, with a wide range of applications, belongs to the interval contraction method, as shown in
Figure 10, where
is objective function,
is termination, and the range of
σ is (
a,
b).
A GRNN model is established, whose input layer contains seven neurons, including hip angle, three sEMG signals, and three plantar pressure signals; and the output layer has three neurons, including angle signal of hip, knee, and ankle joint angles, respectively.
Figure 11 shows the structure of the neural network. The first two-thirds of data are used as the training set and the last third is taken as the testing set.
It was found that the prediction results of GRNN mutate within a short time, which results in a large deviation. That’s because the data becomes unstable under the influence of high-frequency signals. For the purpose of accuracy improvement, the wavelet de-noising method is adopted for signal processing by MATLAB. The basis function is coif5, and the decomposition layer number is 6. The signals are decomposed to obtain low-frequency and high-frequency signals of each layer, and then the signal reconstruction is performed by using the low-frequency wavelet coefficients obtained by decomposition so the processed signal can be obtained after removing the high-frequency noise. Finally, the signal should be de-normalized to get the predicted angle of the lower limb joint.