**4. Discussion**

Before comparing Elite and DYKIMOT measurements we recall their main features in Table 4.

**Table 4.** Summary of Elite and DYSKIMOT main features relative to the present study.


The two-way ANOVA revealed that angle and average angular velocity were significantly different between Elite and DYSKIMOT systems. The difference in angle (1.76◦) between the two systems is lower than the standard clinical angle evaluation of 5◦ reported via classical goniometry [9]. Such difference between the two systems is not clinically relevant, as an error of 2◦ is acceptable in most clinical situations [35]. Concerning the average angular velocity, the difference of −5.50 ◦s<sup>−</sup><sup>1</sup> may come from the errors induced by the derivation of the Elite position. This result is lower than the difference necessary to detect significant differences (7.1 ◦s<sup>−</sup>1) between adults and children [28]. Nevertheless, the clinical significance of a such difference is currently unknown. Apart from these differences, two other ANOVA results may be noted. First, as in [36], no signifiant difference between DP and NDP were observed for all variables studied. Second, the interactions effects (System x Status) did not induce significant differences. At this stage, DYSKIMOT and Elite give broadly similar results, but the computed parameters do not allow to distinguish between DP and NDP, at least in our population. Another point preventing the separation of DP and NDP is that the differences of the means were generally larger for the system factor than for the status factor: The accuracy of the DYSKIMOT device has to be improved, e.g., by appropriate filtering of the raw data and a better integration of gyrometer data, to reduce these discrepancies and improve the diagnostic ability of the sensor.

In a classic way [37–39], we have previously used Bland and Altman's method to evaluate the agreemen<sup>t</sup> between DYSKIMOT and Elite [40]. It appeared that the Bland and Altman's plots did not show a trend with the mean values of the measurements. The Bland and Altman plot for the angle parameter showed more points outside or close to the limits of agreemen<sup>t</sup> than the other plots, which is an indication that agreemen<sup>t</sup> between both systems is less obvious for the range of motion than for other parameters [40]. Since this method does not provide any quantified results on the comparison and leaves the user to decide whether this agreemen<sup>t</sup> is clinically acceptable or not, we analyzed the agreemen<sup>t</sup> between DYSKIMOT and Elite using Passing–Bablok regressions. The Passing–Bablok regression method is a non-parametric method for estimating the slope and the intercept of the linear relationship between two compared [34]. These two parameters are valued by medians and are less sensitive to extreme data and not making assumptions about errors distribution [41]. In our results, the Passing–Bablok indicated that the link between same parameters computed from both systems was well compatible with a linear shape (*r* = 0.694 to 0.922) for all parameters but angle, for which Pearson's coefficient was rather weak (*r* = 0.431) [42]. Nonzero offsets were observed but the 95% confidence intervals were large and always contained 0 value, while the slopes were close to 1 (up to 10% accuracy) with 95% confidence intervals always containing the value 1. Another advantage to this method is that by assuming that Elite results are gold-standard values, the Passing–Bablok regressions could be used to convert measured parameters with DYSKIMOT into "exact values" which are the Elite ones.

Although DTW has been known in the field of acoustic signal comparison [43], it has also been proposed for the purposes of similarity analysis during the functional pattern of gait [44], but never to compare motion neck signals obtained by two different devices. DTW is, by definition, sensitive for measuring two sequences with different lengths using dynamic programming [45]. In this work, the DTW distance between Elite and DYSKIMOT curves was adopted as an indicator of the similarity (up to an affine transformation) between the curves. In other words, the question was: Do both systems measures the same qualitative behaviors in position, angular velocity, and angular acceleration? Although angle measurements displayed a poor agreemen<sup>t</sup> between both systems, the DTW distance between DYSKIMOT and Elite angle was minimal: This result was expected since the structure of angle was simpler than angular velocity and angular acceleration. The DTW distance then increases between the angular velocity and the angular acceleration of the DYSKIMOT and Elite systems. This mostly results in the noise induced by the successive derivations, showing that qualitative features of these curves, especially the angular acceleration, should be interpreted carefully and might be artefacts of the sensor used.

The identification of particular kinematic events is relevant for the clinical assessment of patients, but the global shape of time series may contain more information of clinical interest. In our case for example, it is known that patients with neck pain have poorer sensory-motor control with open eyes, characterized by an increase in joint positioning error and a decrease in speed and acceleration during all movements [12]. The absence of difference in our kinematics data between patients and participants could seem unexpected as previous studies showed significant differences in terms of kinematics [5,46]. However this absence of difference could be explained by our sample size, resulted in low power, and by the difficulty for the DidRen laser to discriminate between such groups [47,48].

An obvious limitation of the present study is that we restricted our comparison of Elite and DYSKIMOT to cervical movements, while potential clinical applications may involve any other joints. Another limitation is that we used "naïve" drift correction following data acquisitions, which had to be implemented in real-time in the software. The Arduino prevented us to reach the desired frequency of 100 Hz with real-time complex filters like Kalman or Mahony. A future development would be the replacement of the Arduino by a slightly more expensive controller (ARM, 30 €), that will allow for real-time filtering and eventually for real-time angular data visualization without entailing too much the low-cost aspect of the DYSKIMOT project. It is therefore obvious that the presented experiments were carried out with a non-user-friendly interface, particularly because of the drift-related problems. However, as the goal of this study was to evaluate the accuracy of a device that could be used by clinicians in clinical practice, we have chosen to leave this concern for future works. A user-friendly interface is currently under development.

In conclusion, the DYSKIMOT-based analysis system compares fairly well to a gold-standard optoelectronic system (Elite) up to linear errors. This ultra-low-cost sensor is recommended for clinical use as it provides more accurate information than the commonly used systems in clinical practice.

**Author Contributions:** Conceptualization, F.B., R.H. and F.D.; methodology, F.B., F.D. and R.H.; software, F.B., C.D., L.J., R.H. and W.E.; validation, W.E., L.J. and R.H.; formal analysis, F.B., R.H., F.D. and C.D.; investigation, R.H.; resources, R.H.; data curation, R.H. and F.B.; writing—original draft preparation, F.B., R.H. and F.D.; writing—review and editing, F.B., R.H., F.D. and C.D.; visualization, F.B.; supervision, L.P., C.D.; project administration, L.P., C.D.; funding acquisition, F.B. and F.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** CeREF acknowledges financial support from the First Haute-Ecole programme, project n◦ 1610401, DYSKIMOT, in partnership with OMT-Skills (http://omtskills.be/), https://www.cerisic.be/technique/projet-cerisic/ developpement-dun-systeme-multitaches-immersif-et-low-cost-denregistrement-et-analyse-de-donneescinematiques-en-vue-de-levaluation-de-dyskinesies-motrices-et-de-leur-prise/.

**Acknowledgments:** We thank Stanley Lognoul and Alexandre Simeoni for their help in taking the measurements. We also thank Jean-Michel Brismée for his careful reading of the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
