Inclusive Human Intention Prediction with Wearable Sensors: Machine Learning Techniques for the Reaching Task Use Case ^†

Abstract

Human intentions prediction is gaining importance with the increase of human–robot interaction challenges in several contexts, such as industrial and clinical. This paper compares Linear Discriminant Analysis (LDA) and Random Forest (RF) performance in predicting the intention of moving towards a target during reaching movements on ten subjects wearing four electromagnetic sensors. LDA and RF prediction accuracy is compared to observation-sample dimension and noise presence, training and prediction time. Both algorithms achieved good accuracy, which improves as the sample dimension increases, although LDA presents better results for the current dataset.

1. Introduction

Predicting human intentions by collecting and analyzing body signals is one of the main goals in human-robot interaction [1]. Accurate and real-time recognition of human motion intention could help in achieving suitable human-machine coordination [2] for both interactive robotic interfaces, like collaborative robots, and diagnostic systems, such as rehabilitation devices [3].

Several kinds of sensors are currently used to detect body signals, like surface electromyography [2,4], electroencephalography [3], and accelerometers. In recent years, research in human movement pattern recognition with the support of wearable sensors was widely conducted [2,3,5], also considering the effect of the positioning of the sensors in the obtained data [6,7]. Actually, wearable sensors allow noninvasive motion detection, full integration with commercially available devices [1], the possibility to acquire acceleration and velocity to reconstruct the detected movement [8], and the adaptation to inter- and intra-individual variability [9]. Since body signals are strongly affected by repeatability lack [2] and motion is subject-dependent, the challenge in predicting human intention even increases in specific scenarios, like in a clinical environment, where the pathological subject can present peculiar motion patterns. In particular, laboratory-based optical motion analysis systems are widely adopted for periodical stroke condition assessment during rehabilitation [10] to obtain multiple bio-signals, useful in recognizing pathological symptoms and improving the healing rate of rehabilitation [11]. Therefore, the knowledge of expected natural behavior and movement patterns for the healthy subject becomes crucial to perform a correct evaluation.

Among all the possible movements, the reaching task is fundamental for the activity of daily living [12] because of the relevance of its functional aim.

Since many different strategies can be used to perform the same task [5,13], predictive models and machine learning algorithms are particularly suitable for analyzing the signals and predicting movement intention [2]. Developing relevant working methodologies is necessary, and machine learning techniques can face the limit of small data amounts. The literature provides various examples of machine learning techniques applied to human motion analysis. For instance, in [14] Linear Discriminant Analysis (LDA), Support Vector Machine and k Nearest Neighbor algorithms were applied for the identification of natural hand gesture, whereas Li et al. [15] exploited the Random Forest algorithm to discriminate eight different motions of the upper limb.

This study aims to compare LDA and RF machine learning techniques’ performance in predicting the subject’s intention of moving towards a specific direction or target in the illustrative scenario of a reaching movement, using data gathered from wearable electromagnetic sensors.

2. Materials and Methods

2.1. Participants

A convenience sample of ten healthy subjects (nine right-handed) were recruited from January to October 2009. Inclusion criteria were: (i) age over 18 years old, (ii) no current or previous neurological or orthopedic pathology of the upper arm. The study was approved by CPP Ile de France 8 ethical committee; recruited subjects gave their written informed consent to study participation, and procedures were conducted according to the Declaration of Helsinki.

2.2. Protocol

Testing sessions were performed during the morning, in the same environmental conditions. In each session, after a first preliminary trial for procedure familiarization, the operator asked the subject to perform six repetitions of unilateral sitting reaching movement, three with the right arm and three with the left arm. As depicted in [16], each subject was asked to perform the movement three times for each combination of directions (internal, middle, external), quotes (high, low), and distances (close, far).

The order of the target submitted to the subjects was standardized: close-middle (CM), far- internal (FI), high-external (HE), far-middle (FM), close-external (CE), high-internal (HI), close- internal (CI), far-external (FE), high-middle (HM). The subjects were required to touch each target with the provided pointer and coming back to the initial condition, moving at a comfortable speed.

2.3. Experimental Setup

Subjects were seated on a chair, adjusted so that the table was at the navel level. They wore a wrist splint to which a pointer was rigidly attached to simulate an extended index finger. The subjects’ trunk was fixed to the chairback using a wide strap. For each subject, four electromagnetic sensors were placed by a trained operator on i) acromion, ii) upper third of humerus, iii) wrist dorsum and iv) manubrium, respectively. During the acquisitions, the Polhemus Fastrak electromagnetic tracking system was used, which provides the position and orientation of each sensor as timestamped vector triplets (X, Y, Z) and (α, β, γ), at an output frequency of 30 Hz [17]. The system presented a Root Mean Square (RMS) static accuracy of 0.8 mm for X, Y, and Z receiver position, and ° for receiver orientation, whereas the resolution was 0.0005 cm/cm of range and 0.025°, respectively.

Nine targets were positioned along three directions: (i) middle, on a parasagittal line emanating from the subject’s shoulder, (ii) internal, and (iii) external, with an incline of ±45° with respect to the parasagittal line. Targets were composed of a red tape of 10 mm with a vertical stick of 15 mm of diameter. Figure 1 depicts the experimental setup. Distances between targets and subject were parametrized with respect to the anatomical upper limb length, i.e., the distance between the acromion and the end of the pointer. Two distances were considered: (a) far, corresponding to 90% of the total upper limb length, and (b) close, equal to 65% of the upper limb length. Six targets were placed at 70 mm of height from the table level, and three were placed above the distal sensor, at the same quote of the acromion from the table surface.

2.4. Data Treatment

Data were elaborated in MATLAB environment using only the information coming from a first sample of the acquired data. To create a dataset of comparable data among trials and among subjects, the acquired signals were trimmed to the actual movement portion. To identify motion starting and ending points, the absolute value of the velocity of the hand was considered. Therefore, the absolute value of the hand position was calculated for each instant t, and the velocity was then computed according to a two-point derivative approximation. According to the literature [18,19], the velocity was filtered with a fourth-order, zero-phase, low-pass Butterworth filter to remove noise, and a cut- off frequency of 3 Hz was selected for the filter [20]. The subject resting condition was identified as the mean value of the first and last ten acquired data samples, corresponding to an interval of 0.33 [s]. The starting and ending points of the movement were automatically selected by a custom-made code, as the first and last time instant in which the absolute value of the position first derivative is higher than a selected threshold. This threshold was iteratively identified by comparing the variance in the observation sample with the variance that the subject presents at the resting condition: if the variance is higher than 5 × 10⁻³ [mm], the threshold values are reduced by 1 × 10⁻³ [mm]. Acquired data were then normalized in amplitude with respect to subject anthropometric quantities, computed for each patient from hand, arm and shoulder positions.

The relative sensors’ shoulder-to-trunk, arm-to-shoulder and hand-to-arm distances along the subject resting phase were computed for each trial. For each subject, the average values of these nine quantities were calculated and used as reference values for the data normalization. To simulate data coming from accelerometers placed on subjects, the second derivatives of sensors’ position were computed, applying two-point derivative twice and filtering the result.

Linear and angular position, velocity and acceleration signals were analyzed to identify a set of features for the implementation of the machine learning algorithms, and a portion of the overall motion was considered as the Observation Window (OW). Since the motion duration is unknown in advance, two approaches, subject- and trial-dependent, were used to evaluate the OW size: (i) custom window, computing an observation time for each trial using the information on the motion length, and (ii) average window, exploiting all the available data of all the subjects to compute a fixed OW. The evaluated features are the minimum, maximum and root-mean-square of: (i) sensor position (SP) components, (ii) sensor velocity modulus, or first derivative of SP, (iii) sensor acceleration modulus, or the second derivative of SP and (iv) Euler angles, respectively. For each trial and subject, the computed features were rescaled to [−0.80, +0.80].

LDA and RF algorithms were implemented and trained using, respectively, 85% and 90% of the data; in both cases, the data chosen for the training phase were randomly selected. In the testing phase, the remaining 15% and 10% of the dataset were used for LDA and RF, respectively. Twenty different combinations of these parameters were tested;

Table 1. The selected features in the conducted tests. In the table, tests with an OW width of 1/7 of the Table 1. are in regular type.

Tests	Sensors Position (SP)	Sensors Velocity	SensorsAcceleration	SensorsEuler Angles
1, 6, 11, 16	x	x
2, 7, 12, 17	x	x		x
3, 8, 13, 18	x
4, 5, 9, 10, 14, 15, 19, 20			x

depicts the selected features in the conducted tests. Two different conditions were evaluated for the OW size: 1/10 and 1/7 of the total motion time length. The first ten tests use the custom average approach, whereas the remaining tests use the custom window one. For each test, both the machine learning algorithms were tested. To analyze the algorithms’ robustness, the same tests were repeated adding Gaussian Noise (noise sample power 0.04 dBW, load impedance 0.4 ohm) to the recorded data before computing the features.

LDA and RF algorithms’ prediction accuracy was computed and compared with respect to data sample dimension, number of considered features, and OW type. Accuracy was computed according to [21,22], and, in the case of RF accuracy, an out-of-bag (OOB) approach was also used.

3. Results

Table 2. Obtained results in RF and LDA algorithms: ori_sgn is the original signal, sgn_N the original signal plus the Gaussian Noise contribute. Results are averaged over 200 consecutive tests.

Test n.	RF					LDA
	OOB Accuracy	SD	OOB Accuracy sgn_N	SD	ori_sgn–sgn_N Residual	Accuracy	SD	Accuracy sgn_N	SD	ori_sgn–sgn_N Residual
1	72.85%	0.032	49.40%	0.017	23.45%	81.00%	0.042	57.04%	0.056	23.96%
2	73.73%	0.015	57.48%	0.015	16.25%	86.13%	0.036	68.73%	0.044	17.40%
3	61.78%	0.016	47.91%	0.017	13.87%	73.11%	0.045	53.12%	0.053	19.99%
4	59.71%	0.015	58.71%	0.014	1.00%	58.27%	0.054	57.40%	0.049	0.87%
5	71.97%	0.013	70.69%	0.013	1.28%	73.56%	0.047	72.81%	0.047	0.75%
6	82.92%	0.011	70.10%	0.014	12.82%	88.97%	0.032	81.07%	0.044	7.90%
7	84.60%	0.01	76.38%	0.014	8.22%	92.80%	0.027	86.60%	0.038	6.20%
8	76.58%	0.013	69.69%	0.014	6.89%	85.45%	0.041	76.36%	0.044	9.09%
9	64.48%	0.016	64.34%	0.016	0.14%	63.84%	0.054	63.26%	0.052	0.58%
10	75.70%	0.012	75.68%	0.011	0.02%	78.82%	0.045	78.76%	0.044	0.06%
11	68.04%	0.015	49.55%	0.017	18.49%	78.80%	0.041	57.47%	0.049	21.33%
12	71.40%	0.015	58.21%	0.016	13.19%	83.26%	0.042	71.17%	0.05	12.09%
13	60.05%	0.016	50.04%	0.017	10.01%	69.56%	0.049	54.78%	0.045	14.78%
14	60.92%	0.015	60.69%	0.016	0.23%	59.57%	0.054	58.65%	0.05	0.92%
15	72.57%	0.013	72.47%	0.013	0.10%	74.09%	0.048	74.31%	0.043	-0.22%
16	80.27%	0.012	65.22%	0.015	15.05%	86.80%	0.04	74.97%	0.045	11.83%
17	82.88%	0.013	74.31%	0.013	8.57%	90.61%	0.034	84.06%	0.041	6.55%
18	73.65%	0.012	66.25%	0.014	7.40%	81.15%	0.044	73.81%	0.047	7.34%
19	66.26%	0.014	65.80%	0.015	0.46%	70.04%	0.048	69.62%	0.045	0.42%
20	78.17%	0.011	77.99%	0.011	0.18%	83.59%	0.04	82.06%	0.044	1.53%

depicts all the obtained results, averaged over 200 consecutive tests. Considering an OW equal to 1/10 of the total movement (average time length of 0.27 [s]), LDA presents, in the best case, an accuracy in the intention prediction of 86.13%, with a Standard Deviation (SD) of 0.036, and an RF with an accuracy of 73.73%, with an SD of 0.015. Increasing the sample at 1/7 of the motion (average time length of 0.37 s), the intention prediction accuracy rises at 92.80% with an SD of 0.027 for LDA, and 84.60% with an SD of 0.010 for RF. Comparing the results obtained from the analysis of data with and without noise, LDA presents a maximum difference equal to −1.53% and an average difference of −0.61%; these values decrease to −1.28% and −0.43% respectively, for RF. Finally, the RF algorithm demands an average training time of 1.14 s (range: 0.87–1.88), which decreases to an average value of 0.078 [s] (range: 0.035–0.28) for LDA. The prediction time has been computed only for the tests with higher accuracy, i.e., tests 1, 2, 6 and 7. The average prediction time is 31 × 10⁻⁴ [s] (range 30 × 10⁻⁴–33·10⁻⁴) for RF and 11 × 10⁻⁵ [s] (range 10 × 10⁻⁵–12 × 10⁻⁵) for LDA.

4. Discussion

Comparing the intention prediction performance of the algorithms with respect to the OW size, better results were obtained when larger windows were considered. Nevertheless, a reasonable limit should be imposed to the window size to avoid the intention being predicted when the movement is close to its end. For features regarding Euler angles, sensor position and speed, as the window width increases from 1/10 to 1/7, accuracy improves by more than 10% for both algorithms in all the tests. When acceleration features are considered, the improvement achieved by a wider OW is about five percentage points, with an SD close to 1%. This behavior can likely be explained considering that the features calculated on the acceleration are subjected to the noise generated by the double derivation. Nevertheless, it represents a qualitative estimation of the results that the algorithms could provide, processing acceleration data from an accelerometer or inertial measurement units (IMU). This interpretation is supported by the results of the noise-added data: in those tests where the acceleration features are considered, noise does not significantly affect the accuracy. This could be justified by the fact that the computed acceleration signal is already noisy. Focusing on the time dimension, LDA reveals considerably shorter training times than RF. To decrease RF training time, the number of trees in the forest can be reduced: a preliminary analysis revealed that after about 40 trees, the accuracy of the algorithm tends to a horizontal asymptote. In the same way, LDA presents significantly shorter prediction times than RF in all the tests.

5. Conclusions

This paper investigates the human reaching movement comparing the performance of LDA and RF to predict subjects’ intention of moving towards a specific direction or target, when analyzing data gathered from wearable electromagnetic sensors. A campaign on ten healthy subjects was performed, and features on measured and computed signals were evaluated. The analyses revealed that the OW size is a crucial quantity: the wider the window, the better the prediction performance. The introduction of noise does not significantly affect the prediction performance of both the algorithms when acceleration features are also considered. For both machine learning techniques, a good accuracy is demonstrated, although LDA presents more promising results in terms of accuracy, training time and prediction time with the current dataset.

Further experimental campaigns, including different kinds of sensors or their positioning strategies, are currently under evaluation. In fact, actual acceleration data gathered from accelerometers and/or IMU inertial sensors would allow an experimental validation of the hypotheses about the acceleration features. Besides, the employment of different wearable sensors could make the acquisition system less invasive for the subject and more flexible, promoting, for instance, the use of widely spread and cheaper electronic devices, like smartphones or smartwatches.