1. Introduction
Running is among the most popular physical activities, and it is the basis of many sports or leisure activities. The practice of running, outside or on a treadmill, has greatly increased in recent years [
1] with high health benefits [
2,
3]. To evaluate its effects and improve performance [
4,
5], analyzing running patterns is increasingly performed. Traditionally, rating scales and subjective observations are used for this analysis, but these approaches are less sensitive to performance changes in training or injury [
6,
7].
To provide objective, reliable, and reproducible outcomes, running gait analysis can be performed using 3D motion capture [
8]. This non-invasive motion system is considered to be a gold standard for walking and running gait analysis [
9,
10,
11] and helps to measure locomotion in depth through spatiotemporal, kinematic, and kinetic parameters. However, the principal disadvantages of this system are its long operation time, dedicated space requirements, technical expertise requirements, and high costs, preventing an assessment of running in an ecological context [
9]. Significant progress has been made with the development of markerless systems and smartphone apps for performing gait analysis in clinical and sports contexts [
12,
13,
14]. They have the advantage of reducing data collection and processing time [
13]. However, these systems have limitations, including limited capture volume (based on the number of cameras) or partial accuracy (e.g., joint angle estimation) [
12,
13].
To overcome these 3D gait recording limitations, wearable devices are increasingly accepted and used by runners and clinicians [
15,
16,
17]. Wearable systems (Inertial Measurement Unit—IMU) are based on a combination of accelerometers, gyroscopes, and magnetometers, allowing the description of normal or pathological human locomotion in a wide variety of environments [
18]. In running gait analysis, IMUs help to measure running gait parameters, notably spatiotemporal parameters, and quantify running performance [
19,
20]. Despite their good feasibility and performance, there are major remaining technical disadvantages, such as sensor attachment errors, external signal noise, signal filtering errors, and integration drift [
18].
To record foot movements and orientations, IMUs can be fixed on or under the foot. In particular, DSPro
® insoles, developed by DigitSole SAS, have the advantage of being able to collect data during different tasks. For instance, this device has been validated for walking gait analysis compared to 3D motion analysis [
21]. Recent studies of this device have highlighted good repeatability in the measurement of walking gait parameters such as cadence, walking speed, and stride length [
21,
22,
23]. However, the validation of the extracted parameters during running is a prerequisite for extending their use in this activity.
For this reason, the aim of this present study was to evaluate the test–retest reliability and criterion validity (i.e., the extent to which scores for volunteers who have not changed are the same for repeated measurements over time and the degree to which the scores of Patient-Reported Outcome Measures (PROMs) are an adequate reflection of a “gold standard”, respectively [
24]), of the running parameters measured using embedded insoles in comparison to those obtained from the gold-standard system, i.e., a motion analysis system.
3. Results
3.1. Population Characteristics
The population was composed of 14 females and 16 males. Female population characteristics were age 27.6 ± 5.2 years, height 165.2 ± 5.3 cm, body mass 61.0 ± 9.2 kg and shoe size 39.2 ± 1.8. Male population characteristics were age 28.2 ± 6.1 years, height 180.3 ± 5.3 cm, body mass 74.4 ± 8.6 kg and shoe size 43.4 ± 1.5.
3.2. Running Gait Cycle
For overground running, 2896 gait cycles were recorded (comfortable speed: 1640 cycles and fast speed: 1256 cycles). For treadmill running, 18,945 gait cycles were recorded (comfortable: 9317 cycles and fast: 9628 cycles).
3.3. Test–Retest Reliability
For overground running, the mean and standard deviation of each parameter, ICC values, and MDC are presented in
Table 1 for motion capture and
Table 2 for insoles.
For motion capture, ICC values were considered excellent for swing time, impact force, and leg stiffness at a comfortable speed and for flight time, stride length, and impact force at a fast speed. Speed, stance time, propulsion time, stride length, and stride height at comfortable speed had a moderate ICC value. The ICC values for the other parameters were considered to be good. Concerning fast speed, the ICC values for speed, swing time, and stride height were good. A lower ICC value was found for propulsion time (ICC = 0.499, p = 0.004). The other parameters had moderate ICC values.
For insole devices, ICC values were considered excellent for impact force and leg stiffness at a comfortable speed and for flight time, impact force, and leg stiffness at a fast speed. For comfortable speed, the following parameters had ICC values higher than 0.75: flight time, stance time, swing time, and flexion angle foot in. Fast speed, swing time, stride length, and stride height also had ICC values higher than 0.75. Regardless of speed conditions, the other parameters had moderate ICC values.
Concerning treadmill running, the mean and standard deviation of each parameter, ICC values, and MDC are presented in
Table 3 for motion capture and
Table 4 for insoles. For motion capture, parameters were significantly correlated between sessions with good to excellent ICC values (0.853 to 0.999,
p < 0.001). For insole devices, parameters were significantly correlated between sessions with good to excellent ICC values (0.875 to 0.995,
p < 0.001) regardless of speed.
The absolute and relative error between the two sessions of each parameter extracted with motion capture and with insoles are presented in
Tables S2 and S3. No significant difference was found, except for stride height during treadmill running (fast condition,
p = 0.002).
3.4. Criterion Validity
Mean, standard deviation, and ICC values for running parameters are presented in
Table 5 for the overground and
Table 6 for the treadmill. Additionally, Bland–Altman plots for running parameters are presented in
Figure 2 for the overground and
Figure 3 for the treadmill. CCC values and plots are presented in
Table S4 and Figures S1 and S2.
Concerning intraclass correlation, ICC values from 0.729 to 0.987 (p < 0.001) were found for all parameters in overground running (comfortable and fast speed). For fast speed, leg stiffness had a moderate intraclass correlation (ICC < 0.75, p < 0.001). The other parameters showed good and excellent intraclass correlations (ICC > 0.75, p < 0.001). For treadmill running, ICC values from 0.899 to 0.999 (p < 0.001) were found for all parameters (comfortable and fast speed).
Concerning Lin concordance coefficient, CCC values were considered excellent (higher than 0.8) for all parameters except for plantar flexion foot during all speeds in overground and during treadmill running (CCC = [0.358–0.500]) as well as loading time (CCC = 0.467), propulsion time (CCC = 0.453) and stride length (CCC = 0.690) for fast speed during overground running.
Based on Bland–Altman plots, the following temporal parameters were found to have a mean bias close to zero seconds: flight time, stride time, stance time, swing time, loading time, and propulsion time. The following spatial parameters also have a mean bias close to zero: speed, stride cadence, stride length, and stride height. These spatial parameters also showed an increase in error heterogeneity with speed. The mean bias for plantar flexion angle foot was close to 6°. During overground running, the mean bias for dynamic parameters (impact force and leg stiffness) was close to −0.1 kN and 0 kN/m. Moreover, LoA was found to be lower than 15% of mean values for all parameters except plantar flexion foot in (20.5% and 25.2%, respectively, for comfortable and fast speed) and leg stiffness (27.9% for fast speed) for overground running. For treadmill running, flight time had an LoA of 26.6% for comfortable speed and 18.2% for fast speed. Plantar flexion foot in had an LoA of 20.6% for comfortable speed and 21.8% for fast speed.
4. Discussion
The aim of the present study was to evaluate the test–retest reliability and criterion validity of running parameters measured using embedded insoles in comparison to the values obtained from the gold standard motion-capture system. Based on the motion capture of 30 participants during overground and treadmill running, the present results highlight the relevance of DSPro® insoles in a sports context. Furthermore, the evaluation of two distinct speeds for both overground and treadmill running introduced a novel dataset, enhancing the precision and comprehensiveness of the validation process for these insoles.
Test–retest reliability was measured to quantify the degree of agreement of parameters for treadmill and overground running. For overground running, ICC values varied between moderate and excellent reliability. The moderate ICC values were explained by the high variability in running between two specific sessions overground because, unlike treadmill running, participants could choose their speed during the second session, thus modifying the gait parameters. However, the results obtained for insoles and motion capture on the treadmill highlight good to excellent reliability due to the identical speed setting between the two sessions. Nevertheless, propulsion time for motion capture shows a moderate and poor ICC value for comfortable and fast speed, respectively. Only a few previous studies have examined the reliability of measurements derived from IMUs during running [
29], but our results are comparable to previous research, regardless of the conditions. For example, Deflandre et al. [
30] reported good to excellent reliability for stance time and step frequency with a portable accelerometer during treadmill running. Using textile socks incorporating IMU, Mason et al. [
31] also highlighted good to excellent test–retest agreement for stride and swing time during treadmill running and moderate to good during overground running. Additionally, the good results of this present study were reinforced by the absence of significant differences in absolute and relative errors between the two sessions (except for stride height during treadmill running in the fast condition), and all these values were lower than the MDC.
For criterion validity, good results were obtained with the ICC analysis. Spatiotemporal parameter measurements had good to excellent agreement between insoles and the motion-capture system regardless of speed (except for leg stiffness at fast speed). Additionally, these results were observed for treadmill conditions with ICC values higher than 0.80 for all parameters. The good reliability of DSPro
® is consistent with other validation studies of IMU against a 3D motion analysis system during a running task. For instance, Uno et al. [
32] found excellent relative validity (ICC > 0.900) for stride length, stride duration, stride frequency, stride speed, vertical height, stance phase duration, swing phase duration, and sagittal angle. Brahms et al. [
33] found an ICC value of 0.955 for stride length during running. Moreover, in a recent meta-analysis, Horsley et al. [
34] reported similar ICC values for stance phase time (ICC = 0.81–0.97), swing phase time (ICC = 0.56–0.81), stride duration (ICC = 0.55–0.99), stride frequency (ICC = 0.96–0.99), stride length (ICC = 0.75–0.99), and flight time (ICC = 0.81–0.86). Additionally, CCC values were in accordance with ICC values except for loading time (fast speed—overground running), propulsion time (fast speed—overground running), and plantar flexion foot in (comfortable and fast speed—overground and treadmill running). Differences between these two coefficients could be linked to deviations of the point cloud from the identity line. Finally, the Bland–Altman plots show that there is low heterogeneity in the measurement bias, with a mean bias close to 0 for spatiotemporal parameters, impact force, and leg stiffness in overground and treadmill conditions. Moreover, a bias of 6° was observed for the plantar flexion angle foot in but not influenced by the speed of running, and a bias of about 20 cm was observed for stride length during fast speed. Additionally, a linear error was observed for speed, stride cadence, stride time, propulsion time, and stride length during overground running. All these biases/errors could be partially explained by the hardware calibration and the algorithm used (fusion filters and calculation method) between motion capture and embedded IMU insoles. Additionally, LoA were not negligible but seem satisfactory with values lower than 15% compared to the mean. However, due to LoA higher than 20%, plantar flexion foot in was only partially validated for all conditions, flight time for treadmill condition, and leg stiffness for overground running at fast speed. However, these variables had moderate to excellent ICC values.
Furthermore, this study validates the use of IMU embedded insoles during overground and treadmill running and, since running was performed in different conditions and at different speeds, indicates that it is a clinically relevant means of assessing gait performance. IMUs are low-cost systems and could easily be used to follow different running parameters for professionals or individuals. Additionally, considering the limitations of motion capture (dedicated space requirements, technical expertise requirements, and high costs) [
9], running analysis could be easily performed only with insoles in a clinical context.
This study has some limitations. First, running experiments were conducted in a laboratory, which can modify kinematics and temporal variables compared to outdoor running. Many studies have highlighted the importance of measuring running outside of the laboratory in natural training and/or competition environments [
35]. Moreover, the treadmill is the most common system used to evaluate and quantify running gait [
17]. This approach has the advantage of providing a standardized and reproducible environment. However, treadmill running only partially reflects natural running behavior, and there are differences in kinematic and kinetic patterns compared to overground running [
15,
17,
36]. Additionally, without an instrumented treadmill and due to the lack of international consensus definition, the loading time parameter was not measured for this condition. However, depending on the attack of the foot during treadmill running, there is a potential for considerable variability, and comparison with data from insoles seems difficult. Additionally, a choice has been made not to estimate leg stiffness and impact force with literature models (e.g., presented by Morin et al. [
37]) in order to validate insoles with real measurements. Second, experiments involved young (mean 27.6 ± 5.2 years) and healthy participants. Showing good accuracy for this population, these results could be modified for older or injured populations. Further investigations are warranted in patients with locomotor disabilities (e.g., with a higher risk of falls) to confirm these good psychometric properties of the insoles in care settings (neurologic rehabilitation, orthopedic surgery, etc.).