Validation of a Wearable IMU System for Gait Analysis: Protocol and Application to a New System

Abstract

Miniaturized wearable Inertial Measurement Units (IMU) offer new opportunities for the functional assessment of motor functions for medicine, sport, and ergonomics. Sparse reliability validation studies have been conducted without a common specific approach and protocol. A set of guidelines to design validation protocol for these systems is proposed hereafter. They are based on the comparison between video analysis and the gold standard optoelectronic motion capture system for Gait Analysis (GA). A setup of the protocol has been applied to a wearable device implementing an inertial measurement unit and a dedicated harmonic oscillator kinematic model of the center of mass. In total, 10 healthy volunteers took part in the study, and four trials of walking at a self-selected speed and step length have been simultaneously recorded by the two systems, analyzed, and compared blindly (40 datasets). The model detects the steps and the foot which supports body weight. The stride time and the cadence have a mean absolute percentage error of 5.7% and 4.9%, respectively. The mean absolute percentage error in the measurement of step’s length and step’s speed is 5.6% and 13.5%, respectively. Results confirm that the proposed methodology is complete and effective. It is demonstrated that the developed wearable system allows for a reliable assessment of human gait spatio-temporal parameters. Therefore, the goal of this paper is threefold. The first goal is to present and define structured Protocol Design Guidelines, where the related setup is implemented for the validation of wearable IMU systems particularly dedicated to GA and gait monitoring. The second goal is to apply these Protocol Design Guidelines to a case study in order to verify their feasibility, user-friendliness, and efficacy. The third goal is the validation of our biomechanical kinematic model with the gold standard reference.

1. Introduction

Gait Analysis (GA) is one functional test used to determine the motor capability of subjects for clinical evaluation [1]. It is currently carried out with advanced technologies including a multifactorial setup: an optoelectronic motion capture system for body kinematics collection, force platforms for ground reaction force measurement, and wireless electromyography (EMG) systems for recording the muscles activations. This GA setup is the gold standard reference technology following its proven and consolidated high accuracy. However, GA is complex and expensive in time and equipment, only few steps in a path can be analyzed, and it requires a skilled operator to perform it. It is true that to increase the number of steps to be analyzed within a small area, instrumented treadmills with an optoelectronics system could be used, taking advantage of the stationary subject’s position and static monitoring equipment’s position. However, it should be emphasized that on a treadmill, the subject’s walking cycle differs from the natural one, resulting in decreased stride length (4%). In addition, it increases the step cadence (6%) and this results in a shorter walking cycle and consequently some variations for the decreased stance (7%) and the increased swing phases (5%). It also increases the step width related to balance (22%) [2,3,4]. Because of these differences, the subjects should be adequately trained for about 1 h to be confident to walk over the treadmill [5] prior to testing.

Thanks to the availability of miniaturized wearable Inertial Measurement Units (IMUs), embedding 3D accelerometers, gyroscopes, and magnetometers, a new generation of systems may offer new opportunities for the functional assessment of motor functions for medicine, sport, and ergonomics. Some of them are based on multisensory platforms (e.g., Xsense, Yost, etc.) [6,7], while others prefer a single IMU approach to collect kinematic data [8,9,10]. These systems are lightweight, very small, and inexpensive. New textile technologies allow researchers to integrate connection with mesh wire, producing sensorized clothes [11,12,13]. The integration between clothes and sensors is also helped by the development of flexible devices. It is possible to use the IMUs both as a single device and as a multisensor cooperative system. The Wearable Health Systems (WHS) are able to measure, monitor and evaluate all parameters of biomedical interest from the human body [14]. This solution allows researchers to better identify activity or events. Relative to human walking, using these systems, it is possible to monitor events as the fall of a subject, or also to track in real time the trajectory of anatomical segments in total body configuration and to estimate the kinematic parameters of the gait cycle [8,9,10,11,12,15,16,17,18,19,20,21,22]. The system could be very useful for patients, caregivers, and even for clinicians to support their clinical decision, as long as it is: (a) a user-friendly system; (b) suitable to be used outdoors, in real life, and not only in the laboratory; (c) capable of monitoring the subject in natural conditions for a long period of time, without altering the natural and normal execution of movements and activities; and (d) without a very complicated calibration process. In fact, a system with this kind of flexibility and adaptability could objectively, quantitatively, and qualitatively measure the gait and mobility of patients with orthopedic or neurologic impairments that affect motor control and support clinicians with new data crucial for decision-making. A solution could be implemented by IMUs.

In our previous research [23,24], over the classical pendulum model, we proposed a new and original kinematic model using a harmonic oscillator to study spatiotemporal parameters of the walking cycle by only one accelerometer without the drift correction and integration of acceleration data. This model is used for data analysis using raw data acquired by a single IMU next to the center of mass. The model, the analysis software, the IMU, and the relative setups and protocols of our system are designed for gait analysis. The preliminary validation [23,24] was carried out on a population of healthy subjects with different genders, Body Mass Index (BMI) values, and anthropometric values; the subject walks over a linear path with a fixed step length. Thee constraints were introduced to have a controlled experimental protocol, leaving the subject with the freedom to adopt a self-selected speed and walking time. The use of the system is not limited to these constraints. More protocol setups and experiments have been carried out to test it when functional tests are submitted to subjects that are healthy and pathological. The proposed system can be used stand-alone or as a modular component of a sensorized WHS t-shirt. Every new system for measurements in motion analysis parameters needs a comparison with a reference gold standard.

In this study, we enforce the validation approach of our system in stand-alone modality, using a comparison with optoelectronic GA which represents the gold standard system reference. This requirement is a milestone. No specific standard protocols were found in the literature, specifically dedicated to single trunk IMU systems for GA gold standard reference validation. Several studies have been carried out to assess the gait analysis reliability with wearable sensors [8,9,10,11,12,15,16,17,18,19,20,21,22], but the evaluation protocols have not been homogeneous; they are mostly based on recording gait trials in a controlled setup to produce some reference datasets. These datasets have then been used to validate the methods. One study has presented the validation of temporal gait metrics from three IMU locations to the GA using Bland-Altman diagrams [25,26]. We have chosen to use a structured methodology that adopts a robust approach to complete the relative task for the validation protocol, as required by the general methodology defined in [24], to validate the use of the new system. To obtain the validation of the system with respect to the true values of the GA gold standard reference, we first had to define the protocol guidelines.

Therefore, the goal of this paper is threefold. The first goal is to present and define structured Protocol Design Guidelines, where the related setup is implemented for the validation of wearable IMU systems particularly dedicated to GA and gait monitoring. The second goal is to apply these Protocol Design Guidelines to a case study in order to verify their feasibility, user-friendliness, and efficacy. The third goal is the validation of our biomechanical kinematic model with the gold standard reference. We do not consider this validation usable only in the laboratory with the protocol setup setting, but it means that the biomechanical kinematic model is robust and new validation protocols can be designed and carried out for accuracy measurements to prove that our system works when it is applied to the walking functional tests and balance tests.

2. Materials and Methods

2.1. Validation Protocol Design Guidelines

To define how a wearable IMU system for gait analysis can be evaluated with respect to GA systems, we define some guidelines to design a robust validation protocol. We think that this approach can help to define a validation protocol that can be followed in future studies. These aspects have to be considered: (a) coherent measurement; (b) statistical significance; (c) robust dataset consistency; (d) assurance of objectivity; and (e) statistical methods.

Coherent measurement means using the gold standard technique as a reference for the measurements conducted with the system under validation, and the two protocols of measurements have to be structured so that the same variables are the outputs to allow for a real data comparison, point-to-point (for continuous matching) or for single values (for overall matching).

Statistical significance means that the dataset allows for a reliable statistical analysis. It requires one specific setup to be defined or to cover all the different predictable experimental conditions for the measurements: the first option is preferable for a complex situation where several variables or conditions could be defined in order to have a segmented approach and a step-by-step systematic validation.

Robust dataset consistency means having a proper number of subjects recruited in clusters of homogenous populations, as required by the selection protocol rules.

Assurance of objectivity means ensuring that the protocol of data analysis is not dependent on how the operator interprets this procedure. Using a blind protocol data analysis, it must have access only to the necessary data and guarantee the non-accessibility to other information that may condition or direct the processing of data by the operator.

Statistical methods have to be properly identified in accordance to protocol and data typology (normal distribution, accuracy analysis, etc.).

From all the above the guidelines, Figure 1 has been defined for designing an experimental protocol dedicated to the validation of a wearable system for gait analysis.

2.2. Experimental Setup for Validation

Subject selection. In accordance with the criteria for sample sizing and subject inclusion (reliable statistical computation and homogeneous/comparable data), the research protocol defines the proper number and typology of subjects to be studied. If required, the experimental protocol must be approved by the Institutional Review Board and informed consent must be obtained from each subject. A collaboration with skilled clinicians permits us to define specific inclusion criteria for healthy and pathological subjects following the focus of the research. With respect to the normal gait, a population of pathological subjects has walking differences induced by their proper specific pathology. This selection has to be specific because the motor impairment could result in different patient outcomes. Single patients or a small population of impaired patients can then be considered as a case study or for functional assessment. It is true that having large numbers on which to do statistics makes the results even more robust, but we must also consider that, especially when studying pathological subjects, it is not always so easy to recruit homogenous clusters of pathological subjects. If this is the case, to have a comparison with normal or reference data, double the number of subjects needs to be examined. In this study, the voluntary subjects were selected and recruited by the clinical staff of Villa Beretta Rehabilitation Center Hospital, Costa Masnaga, Lecco, Italy. Informed consent was obtained from each subject. The selection for this study includes healthy subjects but with different genders (50%) and anthropometric features so to increase the reliability of the experiment with respect to human variability. In total, 10 subjects were examined: five males and five females, aged between 19 to 49 years with (mean value 32.1 ± 9.5 years). All of them were in good health. The weight of the subjects ranged from 46 to 152 kg (mean value 73.1 ± 30.3 kg). Their height ranged from 154 to 187 cm (mean value 170.6 ± 10.7 cm). Their BMI ranged from 19.4 to 43.5 (mean value 24.5 ± 7.0). The subjects have been selected to include a wide range of variation in relation to anthropometric parameters. S_anth anthropometric step length was calculated [23,24,27,28] and ranged from 47.8 to 64.0 (mean value 59.1 cm) (

Table 1. Mean and standard deviation for personal data and parameters of the 10 subjects. S_anth = mean anthropometric step length.

Subject	Weight (kg)	Height (cm)	Age (years)	BMI	Gender	Santh (cm)
1	58	168	31	20.5	F	59.5
2	50	160	49	19.5	F	56.6
3	62	164	19	23.1	F	50.9
4	152	187	30	43.5	M	64
5	70	169	32	24.5	M	57.8
6	87	183	31	26.0	M	62.6
7	46	154	29	19.4	F	54.5
8	68	179	23	21.2	M	61.3
9	78	178	29	24.6	M	60.9
10	60	164	48	22.3	F	58
average	73.1	170.6	32.1	24.5	50%	59.1
standard deviation	30.3	10.7	9.5	7.0		3.9

).

Laboratory setup. All testing was conducted in the same location: the Villa Beretta Rehabilitation Center Hospital, Costa Masnaga, Lecco, Italy. We considered the minimum number of steps for the trial equal to 8. At least these steps should be required from the subjects so to exclude the first two steps (starting gait) and the last two steps (ending and braking steps to stop walking). In this way, it is possible to consider the four (up to five to six) central steps to be analyzed as a straight gait. It means that we have two strides into the calibrated volume of GA for data analysis. This implies a proper linear length of the laboratory pathway of 8–10 m and a correct setup definition of the start and finish line of the walk (Figure 2). The measurement with the IMU includes all the movement of the subject along the path and not only the steps during the straight gait. Along the distance, no lines to identify the start and stop were used; distance is not fixed to avoid constraints and to generate natural walking.

The measurements of reference GA kinematic are carried out by the Elite (BTS Milano, Italy) motion capture system in an eight-cameras setup at a 100 Hz sampling rate. Two dynamometric force platforms (Kistler, Winterthur, Switzerland) measured the ground reaction forces. The GA volume of interest is calibrated.

Subject setup. Some anthropometric segments of the subject are measured: lower limb (ground-greater trochanter), ground-malleolus, lateral condyle-greater trochanter, malleolus-lateral condyle, and fifth metatarsal-malleolus. Width of the foot, length of the foot, and outer distance between the feet are acquired to the ground plane when the subject is resting in natural balance. Age, weight, height, and gender are collected for each subject too. Anthropometric measurement includes the shoes. These values are given as the input to the wearable kinematic model. Other anthropometric measurements are registered following the usual procedure of the clinical GA acquisition.

According to the requirements that correspond to our biomechanical model for data processing and parameters computation, the wearable IMU should be placed close to the Center of Mass (COM). Usually, it is placed onto the pelvis, in a posterior central position, in correspondence with the sacrum bone. It is preferable to anterior positioning at the mid-point between the Superior Anterior Iliac Spines because artefacts due to abdominal breathing could be present.

Passive retroreflective markers are placed on the body over the anatomical landmark of the subject. In our tests, this is carried out following the Davis protocol setup [29].

Each subject is asked to walk over a linear path crossing the working volume of the optoelectronic system at a self-selected speed, as naturally as possible, and to introduce, before and after the walking phase, 5-s of standing to have the reference acceleration baseline (g = 1 along the vertical direction). Before starting the acquisitions, as training, the subject conducts some practice walking along the testing path to become familiar with it. When the subject feels himself/herself practiced with the environment and the setup, the test can start.

Measurement. It is necessary to define the number of gait repetitions at a self-selected speed for each test session, because data are to be collected to verify inter-trial variability or repeatability validation. We define that each subject should execute at least three gait repetitions. When the subject is pathological, three gait repetitions is a burdensome task and the fatigue may result in a modification of the test performance. It means that the values of the spatiotemporal parameters result in increased variability so that mean values and standard deviations will be abnormal. If this is taken into account, the choice to increase the number of trials to have a larger statistical sample conflicts with the repeatability and the precision of data. We have chosen to define four trials in the protocol because it was compatible with healthy volunteer time management and the time available for the GA evaluation without creating problems for patient examination planning. We have analyzed 40 datasets (10 subjects × four trials = 40 datasets).

The recording of data must be simultaneous and synchronized with the device and the motion capture system. The synchronizing trigger could be hardware, software, or functional (for example, making the subject execute a standard start/end movement as a jump).

The sampling rate for data collection from the two systems has to be coherent to allow data comparison.

Contextual video recording is adopted for a further reference on qualitative and quantitative information (e.g., side of the step, number of steps, step duration).

The operator verifies if the dynamometric force platforms are activated correctly. The subject executes each test session four times, walking with his/her shoes. There is a rest period of a few minutes between one session and the next. The step length S is self-selected and the walking time is not a constant depending on the subject’s speed. For each task, an expert medical operator registers the test execution time (using a stopwatch), the number of steps, and the first foot of support (left or right) on a paper report. These values, together with those generated by the GA report, constitute the reference data set.

Data analysis. Only coherent measurements have to be analyzed and compared. The comparison between the wearable system and the gold standard GA is done according to the analysis of the following parameters: (a) number of steps and related limb (left/right); (b) basic temporal parameters: walking time, step times (stance/swing), stride time, cadence; (c) basic spatial parameters: step length (single, average, right/left limb), walking distance (if possible in the laboratory setup). When the optoelectronic GA analysis standard report is generated, it has to be remembered that GA provides data of a single step or stride (usually the central one in the recorded trial selected by the operator or automatically by the system from the Ground Reaction Force data).

When comparing data, a blind data analysis approach is preferable to avoid operator’s bias in data processing and interpretation. This ensures better impartiality in verifying the matching of the results. Access to the data output of the GA test is allowed to the operators in charge of the processing of data generated by the wearable device only when data elaboration has been completed.

We used statistics for data analysis based on absolute and absolute percentage error ε (computed as the difference between corresponding parameters assuming the optoelectronic GA as the true reference). A classification criterion of acceptability is used to identify the outcome of the measurement [24]. The adopted criterion is based on four categories defined by standard statistical thresholds for significance analysis [30], i.e., the commonly used values of 5%, 10%, and 20%:

• Excellence: for ε_% < 5%
• Good: if 5% ≤ ε_% < 10%
• Sufficient: if 10% ≤ ε_% < 20%
• Not acceptable: in case of ε_% ≥ 20%

The reference threshold of the mean absolute percentage error in accuracy (ε_%) of step length and distance is usually fixed at 5% [19]. Starting from this value and considering the setup, this study uses the reliability assessment grid shown in

Table 2. Reliability assessment grid for the analysis of wearable IMUs for GA. ε_% is the mean absolute percentage error in accuracy.

Parameter	Reliability Criteria
	Excellent	Good	Sufficient	Not Acceptable
Step length	ε% < 5% (<3 cm)	5% ≤ ε% < 10% (3 ≤ ε% < 6 cm)	10% ≤ε% < 20% (6 ≤ ε% < 12 cm)	ε% ≥ 20% (≥12 cm)
Distance	ε% < 5% (<0.35 m)	5% ≤ ε% < 10% (0.35 ≤ ε% < 0.70 m)	10% ≤ ε% < 20% (0.70 ≤ ε% < 1.40 m)	ε% ≥ 20% (≥1.40 m)
Counting steps	ε% < 2% (<1 steps)	2% ≤ ε% < 4% (1 ≤ ε% < 2 steps)	4% ≤ ε% < 6% (2 ≤ ε% < 3 steps)	ε% ≥ 6% (≥3 steps)
Time	ε% < 4% (<1.2 s)	4% ≤ ε% < 6% (1.2 ≤ ε% < 1.8 s)	6% ≤ ε% < 8% (1.8 ≤ ε% < 2.4 s )	ε% ≥ 8% (≥2.4 s)

.

The proposed criterion has only been applied to the absolute percentage error. This implies a more strict evaluation of method reliability because the relative accuracy percentage error could introduce some compensations (having both positive and negative values).

2.3. Specifications of the New Wearable System and Its Data to Be Validated

The experiment used a commercial wearable 3D accelerometer (Protheo of SXTsrl, Cinisello Balsamo, Milan, Italy), with the following technical specifications: 85 (l) × 53 (w) × 16 (h) mm of size, 70 g of weight, four digits LCD, on board ARM7 microprocessor, and raw acceleration sampling frequency of 128 Hz. This device is able to log tri-axial accelerations and ECG signals into the internal memory (up to three days of continuous monitoring), which can be downloaded at the end of the acquisition by Bluetooth^® data transmission. Therefore, the data storage allows the recording of different tests in sequence. Integrity and quality of data have been examined only at the end of the test after downloading raw data and post processing. IMU calibration is performed off line. During this experiment, the system is worn at a sacrum position on the back of the subject, by means of an elastic belt with a pocket to fix the device firmly to the body and avoid artifacts in raw data originating from movements of the device with respect to the COM position. In the pocket of the belt, in addition to the device, during the subject setup, a user-customized paper or plastic element has been placed in order to align the device with the vertical axis along which the force of gravity acts. IMU alignment with the gravity direction is verified with a small bubble. A control and correction procedure with respect to the gravity base line is provided in the data analysis software. The subject removes the wearable device after all four trials of the validation test are completed. Data are downloaded via Bluetooth^® from the wearable device to a PC for offline processing.

Concerning the biomechanical approach, the wearable IMU-based system for kinematic gait analysis implements the pendulum and harmonic oscillator models [9,10,18,29] with an original approach [23,24]. When the subject walks with the rigid legs of length L and makes a step of a length S, the COM moves forward along a sinusoidal curve both in the vertical and in the medio-lateral planes [31,32]. In this study, only the sagittal plane is considered (Figure 3). When the subject is stepping in double stance phase, the hip angle is θ_max, and COM descends from its highest point to the lowest one. During the step cycle, the vertical distance between the maximum and the minimum height of the COM position defines a distance: the maximum amplitude of oscillation hCOM. Each step (either left or right) is carried out following this pattern. The cycle is repeated at every step with its characteristics, (i.e., the pattern is very similar but not exactly the same). The use of a harmonic oscillator model allows us to define a correlation between the cycle of COM positions and the step cycle during walking. One harmonic oscillator is associated with each step cycle; thus, we look at the walking as a set of oscillators describing human kinematics. The oscillation period defines gait frequency and cadence.

According to the pendulum and kinematic model [9,10,18,23,24], the length of the single step (S) is given by:

$$\mathrm{S}=2\times \sqrt{2\times \mathrm{L}\times \mathrm{hCOM}-{\mathrm{hCOM}}^2}$$

(1)

The length L of the lower limb is known for each subject through direct anthropometric measurement; hCOM is evaluated from the acceleration data measured with the device according to the harmonic oscillator model [23,24], step by step, according to the following equation:

$$\mathrm{hCOM}=2\times {\mathrm{a}}_{\mathrm{V}}\times {\left(\frac{\mathrm{T}}{2\mathsf{\pi }}\right)}^2$$

(2)

where ${\mathrm{a}}_{\mathrm{V}}$ is the measured maximum value of the vertical acceleration in the considered step and T is the measured period (=time between two consecutive acceleration peaks) of the considered step. For every walking cycle, by assessing hCOM amplitude using Formula (2), the step length S is computed using Formula (1). The incremental traveled distance is the incremental sum of these steps. The discussion about this biomechanical model and the method used for data analysis is explained in [23,24] and it is briefly recalled here.

The analysis processing raw accelerations is implemented in the Matlab© software suite. The detection of the peaks in raw acceleration signals corresponds to the steps. Then, different filters are applied according to the walking speed (high i.e., >1.4 m/s, normal i.e., 1.4 m/s < speed < 0.9 m/s, or low i.e., <0.9 m/s). At normal and high speed, the raw signal is processed with a fifth order low-pass Butterworth filter. Antero-Posterior, Vertical, and Medio-Lateral accelerations have the following cut-off frequencies, respectively: 1.8 Hz, 1.8 Hz, and 0.9 Hz. At low speed, the raw signal is processed with a fourth order low-pass Butterworth filter, and Antero-Posterior, Vertical, and Medio-Lateral accelerations have the following cut-off frequencies, respectively: 35 Hz, 5 Hz, and 3 Hz.

When the peaks are detected, a further 19th order Butterworth filtering is applied to the original raw signals (Antero-Posterior, Vertical, and Medio-Lateral accelerations are filtered respectively with the following cut-off frequencies: 6 Hz, 35 Hz, and 8 Hz). A fixed cut-off amplitude threshold over hCOM is arbitrarily set to 0.06 m. The hCOM threshold only works on the $a_V$ amplitude; the signal periodicity is preserved. hCOM is a distance and thus a relative measure unit cuts off its amplitude variation over time. We compute hCOM using Formula (2) and S using Formula (1) according to our harmonic oscillator model approach. No integration or drift correction is used.

The side of the step is determined by identifying the peaks on the medio-lateral acceleration track: their positive/negative value corresponds to the right/left footstep.

3. Results

The total number of walk tests performed was 40.

Table 3. Novel system validation: comparison between gold standard GA results and wearable system data (* GA data corrupted/not coherent. ε error.).

Subj. No.	Trial	Side of the First Supporting Foot True/False	Step No. ε	Walking Time ε%	Stride Time ε%	Cadence ε%	Step Length ε%	Right Foot Step Length ε%	Left Foot Step Length ε%	Step Speed ε%	Right Foot Step Speed ε%	Left Foot Step Speed ε%
1	1	True	0	−8.3	*	*	*	*	*	*	*	*
	2	True	0	−16.3	−4.8	3.7	−8.4	−2.7	−13.2	−13.0	−13.2	−12.2
	3	True	0	−5.6	7.7	−7.6	−6.3	−4.1	−8.1	−20.7	−27.8	−17.4
	4	True	0	−0.6	0.0	0.4	−10.1	−12.0	−8.2	−17.7	−14.6	−20.9
2	1	True	0	−5.5	4.2	−4.1	−1.3	7.3	−8.7	−15.3	−16.0	−15.0
	2	True	0	−9.0	−1.0	0.3	−1.4	3.7	−5.8	−11.0	−7.7	−14.9
	3	True	0	−13.3	4.2	−3.9	−5.6	−6.9	−4.4	−18.4	−15.9	−21.0
	4	True	0	−2.2	5.7	−5.8	−1.2	−2.4	0.3	−14.4	−10.3	−21.3
3	1	True	0	5.6	2.5	−2.6	6.4	7.8	5.6	−5.7	−4.1	−8.4
	2	True	0	−3.6	1.2	−2.0	3.0	7.8	−1.1	−8.4	−5.5	−12.6
	3	True	0	0.4	2.3	−2.2	4.2	2.5	6.3	−5.5	−7.0	−5.4
	4	True	0	−14.1	−2.7	2.2	1.4	−4.2	7.8	−6.2	−6.3	−6.8
4	1	True	0	4.6	−2.4	−7.6	−1.9	8.6	−10.1	−18.0	−10.5	−27.7
	2	True	0	−23.5	−6.3	6.7	−7.9	1.8	−17.4	−13.4	−8.0	−17.4
	3	True	0	10.4	11.2	−10.9	1.7	5.1	−1.4	−17.7	−21.8	−20.1
	4	True	0	−20.6	−2.8	3.7	−10.1	1.8	−21.1	−17.8	−16.8	−18.2
5	1	True	0	−9.9	−6.8	7.8	−5.0	6.0	−15.9	−10.1	−7.0	−8.3
	2	True	0	−9.1	−0.5	0.6	−4.0	7.7	−15.5	−11.8	−13.5	−10.7
	3	True	0	−2.4	3.0	−2.6	1.3	6.0	−3.4	−11.4	−10.2	−13.2
	4	True	0	−4.3	−8.6	9.4	*	*	*	−13.4	−6.9	−18.4
6	1	True	0	−29.3	9.8	−8.1	−2.3	5.1	−7.5	−19.4	−28.7	−14.6
	2	True	0	−31.8	−2.0	0.2	0.6	16.4	−10.8	−7.9	−0.3	−14.0
	3	True	0	−28.8	−3.2	2.2	−3.1	14.3	−15.5	−8.8	−4.3	−12.3
	4	True	0	−24.2	−7.1	7.7	−23.5	−29.7	−17.5	−24.6	−23.6	−26.8
7	1	True	−1	−29.9	*	*	*	*	*	*	*	*
	2	True	0	−21.6	7.3	−6.6	9.5	15.6	3.5	−8.6	−4.5	−14.6
	3	True	0	−21.9	−19.7	6.7	0.2	−9.2	9.9	−5.1	−6.0	−5.1
	4	True	0	−22.6	−4.8	5.0	−3.1	−5.5	−0.7	−8.0	−7.7	−12.4
8	1	True	0	−30.4	−7.9	9.1	−7.2	2.8	−17.1	−10.5	−10.2	−8.2
	2	True	0	−20.8	−5.8	5.9	−6.9	8.1	−21.4	−13.4	−8.9	−14.8
	3	True	0	−19.3	−17.6	5.2	−4.8	7.1	−16.5	−11.2	−4.5	−15.3
	4	True	0	−21.3	−11.0	12.7	−9.4	−20.5	1.2	−9.2	−11.5	−7.2
9	1	True	0	−7.7	−2.6	0.9	−1.4	7.2	−7.9	−9.7	−10.9	−7.4
	2	True	0	−7.2	−4.8	2.0	−0.5	10.8	−9.2	−8.7	−8.9	−9.2
	3	True	0	−18.5	−4.4	3.6	−1.2	8.9	−9.2	−9.1	−9.0	−9.3
	4	True	0	−12.8	11.6	−10.1	−3.4	−3.1	−3.7	−21.7	−13.8	−30.0
10	1	True	0	−17.3	−3.8	4.0	−10.9	−8.2	−13.7	−21.5	−20.4	−17.4
	2	True	0	−28.3	−4.7	2.9	−18.5	−23.4	−12.1	−22.5	−22.8	−23.0
	3	True	0	−16.4	7.5	−5.3	−10.9	−6.9	−14.9	−22.9	−21.0	−28.9
	4	True	0	−17.3	4.5	−4.3	−9.4	−14.8	−4.1	−19.5	−27.2	−17.9
min			−1.0	−31.8	−19.7	−10.9	−23.5	−29.7	−21.4	−24.6	−28.7	−30.0
max			0.0	10.4	11.6	12.7	9.5	16.4	9.9	−5.1	−0.3	−5.1

and

Table 4. Novel system validation: comparison between gold standard GA results and wearable system absolute error data.

Parameter	Side of the First Supporting Foot	Step No.	Walking Time	Stride Time	Cadence	Step Length	Right Foot Step Length	Left Foot Step Length	Step Speed	Right Foot Step Speed	Left Foot Step Speed
Value	True/False	Absolute Error	abs % Error	abs % Error	abs % Error	abs % Error	abs % Error	abs % Error	abs % Error	abs % Error	abs % Error
mean	0.0	0.0	14.9	5.7	4.9	5.6	8.5	9.5	13.5	12.3	15.2
st.dev	0.0	0.2	9.4	4.3	3.2	5.1	6.2	5.9	5.6	7.3	6.6
min	0.0	0.0	0.4	0.0	0.2	0.2	1.8	0.3	5.1	0.3	5.1
max	0.0	1.0	31.8	19.7	12.7	23.5	29.7	21.4	24.6	28.7	30.0
reliability assessment	excellent	excellent	sufficient	excellent	excellent	good	good	good	sufficient	sufficient	sufficient

present the comparison between the proposed kinematic model and GA output for each subject. Table 3 presents the spatial temporal parameters of the gait of the four trials carried out by all subjects. The value of the side of the first foot which supports body weight is set to true if there is correspondence between the real situation determined by observing the video recording of the trial and the output of the algorithm. The errors in step’s number and in walking time are computed as the difference between the observed value, determined by a skilled clinician, and the output of the algorithm. For the other parameters, we have computed the absolute percentage error with respect to the true values measured by the optoelectronic system (stride time, cadence, step length, right foot step length, left foot step length, step speed, right foot step speed, left foot step speed).

Two optoelectronic GA datasets (subject 1—trial no. 1 and subject 9—trial no. 1) are corrupted and not possible to process: in this case only the manual measurements taken by the skilled clinician have been analyzed. The outcome of the processing of optoelectronic GA data of trial no. 4 of subject 5 has presented incoherent values (anomalies found in GA tracking reconstruction), so they have been excluded from the analysis.

In three cases (subject 1 trial—no. 4, subject 7—trial no. 1, and subject 9—trial no. 1), a discrepancy (−1 step in all these cases) between the number of steps manually counted by the skilled clinician and the output from the IMU raw data processing has been elicited. A check using video recording of the trials has allowed us to verify that the true value was the one computed by the algorithm, so that the table was amended. Unfortunately, the video recording of optoelectronic GA trial for subject 7 trial no. 1 is not available and the table reported the difference. The experimental data have demonstrated that the model counted correctly without error for the number of steps for all the healthy subjects according to the operator’s manual count, except for one case (subject 7, trial no. 1 without video recording for the check), where one step less was counted.

The proposed model correctly detected the side of the first foot which supports body weight (left or right). This result is similar to the one reported in previous different tests [23,24].

Examining the percentage error of the gait cycle time (stride time), the error is lower than that of the total walking time. The latter is defined as the time between the moment in which the subject receives the command to begin walking and the one in which he receives the command to stop. As expected, the error of the cadence is comparable to that of the stride time.

4. Discussion and Conclusions

The validation of wearable motion capture systems is becoming a large and important issue due to the increasing availability of these technologies and solutions. In this study, a set of guidelines to design specific and dedicated validation protocols for wearable motion capture systems in comparison with GA gold standard optoelectronic systems is presented. It helps us to provide an approach of a framework for validation studies.

To verify the applicability of the proposed guidelines, we have designed and carried out a case study with a single IMU-based system. The processing of raw data to extract gait parameters is based on a harmonic oscillator biomechanical model that we want to validate. Kinematic joints were not calculated, but only some spatiotemporal parameters. This study investigates the wearable IMU system’s accuracy with respect to optoelectronic GA as the gold standard reference. The GA and the wearable measurements have been carried out simultaneously for a population of 10 subjects with four trials (40 datasets). Operators not involved in the experimental activity performed blind data analysis to assure the best impartiality. Better ergonomic fit can be achieved by using specially designed t-shirts and using a smaller IMU.

The protocol we have designed according to the proposed guidelines has been demonstrated to be operatively simple and effective; the operators (one technician, two biomedical engineers, one physicist, and two clinicians) and the subjects involved in the experiments expressed good compliance and appreciation for its high usability. Data from the selected healthy cluster was collected without the interruption of programmed activities to evaluate clinical examinations over the pathological subjects present as usual in the GA laboratory. The time available allowed us to complete all the scheduled tests. Data were properly collected, except for two trials, whose reference GA data have been not successfully recorded by the optoelectronic system; these data were corrupted and have not supported the parameters computation except from the manual measurements taken by the skilled clinician. This could be described as a failure of the standard optoelectronic system (two cases out of 40 trials equal to a rate of 5%) with respect to the 100% success of the wearable system. The complexity of the optoelectronic systems has this intrinsic possibility.

A more detailed discussion has to be done about the reference walking time value. The subject begins and ends walking outside the GA work volume. This means that a set of photocells could be used to measure the temporal duration of the task; alternatively, a skilled clinician with a stopwatch has to take these measurements. We do not have photocells and therefore in our case, this is a manual measurement taken by the skilled clinician. Thus, the percentage error of the total walking time is variable and operator-dependent. Its large values should be thought of in relation to the fact that the operator has an intrinsic reaction time in taking the measurement through a stopwatch. A typical reaction time of 0.5 s in the laboratory setup we have used (i.e., with 4–5 s duration of the trials) produces, respectively, for a total time of 4 s, the percentage accuracy error equal to 100 × 0.5/4 = 12.5% and for a total time of 5 s, the percentage accuracy error equal to 100 × 0.5/5 = 10%. It could be observed that the operator has to use the stopwatch twice (start and stop), so the estimated error could be even greater. In fact, according to the set up adopted and to the previous considerations, the value of the computed average % error is 13.9, in line with the expected outcomes.

A point to be considered in such validation is that the motion capture system only uses very few steps and only one (the central stride) is considered as the regular representative cycle step. Therefore, when we use the wearable IMU system, we have to compare an average step resulting from a set of steps with a single stride. Intrinsic differences or errors are obviously present. According to the natural variability of the movement, the reliability assessment grid allows us to classify these comparisons.

The default value used to set the threshold amplitude should be flexible to better match the output of the motion capture device. For this validation, because the data processing is blinded, for a healthy population, we have considered it acceptable to adopt a fixed value of 0.06 m as the threshold for hCOM amplitude and the small error confirms its reliability in healthy subject analysis. If information on total walk distance and step’s number is held, then we can compute an expected hCOM threshold (modality 1 of reference [23,24]). More work is necessary to better define the thresholds of hCOM amplitude in different clusters of subjects, e.g., in specific populations for pathology, anthropometry, etc.

Another important methodological aspect is the analysis of the possible sources of errors. We have identified two basic issues. Wearable systems have to be fixed properly and firmly to the human body so as to avoid vibration artefacts on the IMU measurements. This seems banal, but the artefacts are in the frequency band of the signal, so not removable by filtering. A second important aspect is that a bad alignment of the measuring device could also introduce drift in the measurement, so this setup parameter has to be controlled and compensated.

Concerning the third outcome of this study, the data have confirmed the validation of the proposed wearable system. It completed the relative task n.5 of the segmented validation approach to test the wearable sensor in walking presented in [24]. This good advice confirms the old preliminary results about the reliability of this wearable device for carrying out ambulatory, long-term, and ecologic kinematic GA. The expectation is to develop a dedicated tool for supporting diagnosis and rehabilitation in the hospital and/or at home. In fact, our instrument has many advantages with respect to the gold standard considered (optoelectronic GA system) in terms of flexibility, portability, and cost. A portable GA can provide the clinician with much data collected in an everyday, ecological setting, data not available with other instruments, not even with optoelectronic GA, but nonetheless crucial for decision-making.

Future reporting activity will be dedicated to understanding the limits and the potential of this wearable kinematic system and its underlying harmonic oscillator model following the tasks presented in [24]. We are going to explore its applicability in balance and in some classical functional tests routinely used in clinical rehabilitation and to evaluate a subject’s performance (the 6MWT—Six Minutes Walking Test, the TUG—Timed Up and Go, the 50-m walking test with normal or maximum self-selected speed, and the 10-m walking test with normal or maximum self-selected speed).