Next Article in Journal
Optimization of Cladding Diameter for Refractive Index Sensing in Tilted Fiber Bragg Gratings
Next Article in Special Issue
3D Kinect Camera Scheme with Time-Series Deep-Learning Algorithms for Classification and Prediction of Lung Tumor Motility
Previous Article in Journal
High-Resolution Representations Network for Single Image Dehazing
Previous Article in Special Issue
The Reliability of the Microsoft Kinect and Ambulatory Sensor-Based Motion Tracking Devices to Measure Shoulder Range-of-Motion: A Systematic Review and Meta-Analysis
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Kinect v2-Assisted Semi-Automated Method to Assess Upper Limb Motor Performance in Children

by
Celia Francisco-Martínez
1,
José A. Padilla-Medina
1,
Juan Prado-Olivarez
1,*,
Francisco J. Pérez-Pinal
1,
Alejandro I. Barranco-Gutiérrez
1 and
Juan J. Martínez-Nolasco
2
1
Electronics Engineering Department, National Technology of Mexico in Celaya, Celaya 38010, Mexico
2
Mechatronics Engineering Department, National Technology of Mexico in Celaya, Celaya 38010, Mexico
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(6), 2258; https://doi.org/10.3390/s22062258
Submission received: 4 February 2022 / Revised: 2 March 2022 / Accepted: 11 March 2022 / Published: 15 March 2022
(This article belongs to the Special Issue Kinect Sensor and Its Application)

Abstract

:
The interruption of rehabilitation activities caused by the COVID-19 lockdown has significant health negative consequences for the population with physical disabilities. Thus, measuring the range of motion (ROM) using remotely taken photographs, which are then sent to specialists for formal assessment, has been recommended. Currently, low-cost Kinect motion capture sensors with a natural user interface are the most feasible implementations for upper limb motion analysis. An active range of motion (AROM) measuring system based on a Kinect v2 sensor for upper limb motion analysis using Fugl-Meyer Assessment (FMA) scoring is described in this paper. Two test groups of children, each having eighteen participants, were analyzed in the experimental stage, where upper limbs’ AROM and motor performance were assessed using FMA. Participants in the control group (mean age of 7.83 ± 2.54 years) had no cognitive impairment or upper limb musculoskeletal problems. The study test group comprised children aged 8.28 ± 2.32 years with spastic hemiparesis. A total of 30 samples of elbow flexion and 30 samples of shoulder abduction of both limbs for each participant were analyzed using the Kinect v2 sensor at 30 Hz. In both upper limbs, no significant differences (p < 0.05) in the measured angles and FMA assessments were observed between those obtained using the described Kinect v2-based system and those obtained directly using a universal goniometer. The measurement error achieved by the proposed system was less than ±1° compared to the specialist’s measurements. According to the obtained results, the developed measuring system is a good alternative and an effective tool for FMA assessment of AROM and motor performance of upper limbs, while avoiding direct contact in both healthy children and children with spastic hemiparesis.

1. Introduction

Effective public health and social measures are fundamental strategies to combat Coronavirus (COVID-19). Avoiding crowds and maintaining a physical distance from others is essential to reduce the rate of infection [1]. The negative consequences of the interruption of rehabilitation activities are significant for the population with physical disabilities [2]. The World Health Organization reported that more than 15% of the world’s population has a physical limitation, such as cerebral palsy (CP) [3]. The Gross Motor Function Classification System (GMFCS) scale is essential for determining physical movement skills in patients with CP, e.g., the ability to sit up [4]. Similarly, the Manual Ability Classification System (MACS) scale determines fine motor skills, specifically of the hand, e.g., activities that the patient can perform daily, from combing hair to tying shoelaces [5]. Motor treatment is essential, and the effectiveness of cerebral palsy treatment depends on many factors, including optimal and systematic monitoring of patients’ progress [6]. Home therapy with active video game technologies and low-cost virtual reality is a favorable solution to minimize the impact of rehabilitation interruptions [2].
Recently, telemedicine has increased exponentially due to the COVID-19 pandemic. The authors Ben-Pazi et al. [7], in their various recommendations, mention the possibility of a range of motion (ROM) measurement based on photographs of the patient that are remotely taken and sent to specialists. Clinical ROM measurements are essential metrics for monitoring motor rehabilitation progress. In general terms, this provides objective and reliable information in the diagnosis and clinical monitoring of the physical condition evolution [8]. The goniometer is one of the most common instruments for measuring ROM [9]. Sophisticated instruments are also used such as: electro-goniometers [10], laser goniometers [11], optoelectronic devices [12], wearable sensors [13,14], smartphone applications [15], Kinect sensors [16], and Xsens systems [17].
In this context, the Kinect v2 sensor (Microsoft, Redmond, WA, USA) has a color camera and a depth measurement system based on active illumination (infrared camera and an infrared projector) [18,19]. Hence, color, depth, and infrared image detection are possible, which turns the Kinect into a low-cost three-dimensional RGB-D camera [20]. Time-of-Flight depth-sensing technology allows object distance measurements for each image pixel according to the output data. Depth maps correspond to 16-bit encoded 2D images, containing the measurement information stored in a pixel array; from this 2D data, it is possible to infer 3D coordinates (X, Y, Z) [18]. The natural user interface (NUI) presents a markerless human pose estimation algorithm; its main objective is interpreting human positions and gestures by processing vector data.
Several researchers have evaluated the reliability of the Kinect v2 system in the analysis of upper limb functionality and found significant measurement accuracy [16,21,22,23,24,25,26,27]. The authors Cai et al. [21] found that the mean square error of the shoulder joint angle (adduction and abduction) was less than six degrees between the Kinect v2 (Microsoft, Redmond, WA, USA) and Vicon (Oxford Metrics Group, Vicon Motion Systems Ltd., Oxford, UK). After a stroke, the FMA scale is used to assess the motor performance of the upper limb. This scale is composed of an assessment of motor and sensory abilities, balance, joint range of motion, and joint pain [28]. For example, the Kinect sensor has been used in FMA tests, mainly because it allows the measurement of ROM [29,30,31,32]. The authors Kim et al. [30] found a significant correlation between FMA scores and scores using the Kinect v1 sensor (Microsoft, Redmond, WA, USA). They claim that the use of FMA with Kinect v1 is a reliable way to assess upper limb motor performance. Recently, Lee et al. [33] proposed an FMA scoring algorithm with fuzzy logic and Kinect v2; the authors argue that this method presents a high correlation with the scores given by a specialist.
A comparison of upper limbs’ angular measurements using a Kinect v2 sensor (Microsoft, Redmond, WA, USA) and a universal goniometer (Richardson Products Inc, Gulfstream Rd, Frankfort, KY, USA) was undertaken in the study described in this paper. In addition, a semi-automated FMA clinical scoring algorithm was implemented. The main study goal was to assess upper limbs’ motor performance using the proposed semi-automated method assisted by the Kinect v2 sensor, from which the results were contrasted with specialist’s measurements.

2. Materials and Methods

An experimental study using a low-cost natural user interface (NUI) system to quantitatively measure AROM and clinically FMA assess the upper limb motor skills in children was conducted. Firstly, the measuring system were implemented using the selected components. Then the designed graphical user interface was programmed and finally validated in the experimental stage.

2.1. Participants

Two test groups, each having eighteen participant children, were analyzed in the experimental stage, where upper limbs’ AROM was measured and motor performance was assessed using FMA. In the control group, participants (mean age of 7.83 ± 2.54 years) had no cognitive impairment or upper limb musculoskeletal problems. The study test group comprised children aged 8.28 ± 2.32 years with spastic hemiparesis. A total of 30 samples of elbow flexion and 30 samples of shoulder abduction of both limbs for each participant were analyzed using the Kinect v2 sensor. In addition, 30 FMA assessments tests were performed [28]. All participants gave informed consent. The inclusion and rejection criteria for both groups are presented in Table 1.

Measurement Protocol

The participant remained seated and performed a voluntary movement within the flexor synergies according to the FMA scoring; Fugl-Meyer (1975) [28]. The measurement conditions were the same as those of the preliminary test (Section 2.3).
A physiotherapist performed upper limbs’ AROM measurement with a goniometer and motor performance assessment using FMA scoring. The cognitive characteristics for each participant were assessed by a psychologist using the Bender Koppitz test [34]. It is relevant to mention that, of the sample of 27 candidates initially participating in the study group, nine were excluded: three who did not want to participate in the evaluations, and six due to severe mental age lags detected by the Bender Koppitz test [34]. However, following the psychologist’s suggestion, participants with mental age deficits capable of following directions were accepted as participants in the study because, in most cases, the chronological mismatch was due to hand manipulation problems in performing the activities required by the test (Table 2).

2.2. Graphical User Interface and Programming

A Kinect v2 sensor and an HP Intel® Core™ i7-9750H 9750H 2.60 GHz laptop with NVIDIA GeForce GTX 1650 graphics card was used to integrate the AROM measurement and FMA assessment system. The Kinect v2 sensor (Microsoft, Redmond, WA, USA) was employed to assess children’s limb movements based on angles formed by the limb joints.
The general participant data, such as name, age, weight, height, shoulder/elbow AROM, and performance level, are registered in the frontal panel of the graphical user interface panel (Figure 1). Thirty control parameters related to upper limb activity covering items I, III, IV, V of section A, B, C, and D of the FMA scoring are also available in this panel; Fugl-Meyer (1975) [28]. These parameters were manually selected by the therapist according to the movements of rotation, pronation, supination, and retraction of the arm, which cannot be acquired by the Kinect v2 sensor, and for activities observed during the test, such as the grip type and hand sensitivity. Section A-II “Voluntary movement within synergies without gravitational assistance”, corresponds to the automatized part of the programming as the shoulder angles in abduction, adduction, flexion, and extension movements of the elbow are acquired through the Kinect v2 sensor. The upper limbs motor performance is shown in the “Total A-D” label. The highest FMA score corresponds to the best motor functioning state of the patient’s limb. An FMA score ranging from 0 to 22 represents no upper limb motor capacity; scores of 23 to 31 represent low capacity; scores of 32 to 47 represent limited capacity; finally, scores from 53 to 66 represent total upper limb capacity [35]. The interface allows visualization of the measurement environment.
The graphical user interface was developed using LabVIEW software (14.0, NI, Austin, TX, USA). The Kinect for Windows SDK 2.0 software development kit was used, and MakerHub Interface for Microsoft Kinect One was employed to acquire color, depth, infrared, and sensor skeleton tracking data. The measurement system programming process is shown as a block diagram in Figure 2. In the programming, the motor evaluation is performed by following Fugl-Meyer scale algorithm, where the left and right shoulder and elbow angles are the input data; these vector points are acquired by the Kinect sensor. The process involves the AROM measurement, and the evaluation results are stored in a Microsoft Access database (Microsoft, Redmond, WA, USA).
The AROM calculation of the forward shoulder abduction/adduction in the frontal plane and elbow flexion/extension in the sagittal plane are based on the angle formula between two vectors given by Equation (1) [36]:
Cos   θ = V d · V p | V d | | V p |
where θ is the shoulder/elbow angle, V d is a distal longitudinal vector, and V p is a proximal segment vector.
The segments used for the shoulder angle calculation are summarized in Table 3, selected from the 25 joint points provided by the Kinect v2 sensor [19]. The recommendations established by the International Society of Biomechanics (ISB) were followed [37], but applied only to forearm movements (elbow angle) because of the location of the joint points provided by the sensor.

2.3. Experimental Setup

The Kinect v2 sensor (Microsoft, Redmond, WA, USA) was placed on a uniform base at a height of 80 cm and at a distance of 2 m from the participant according to the recommendation of [21]. The Kinect v2 sensor was turned on and run for 25 min before starting the assessments as recommended in [38]. Measurements were carried out in a 3 × 3 m space considering the operational measurement range [19,23,39] with an illumination intensity of 73 Lux (lx).
In order to verify the distance and illumination, 30 angle measurements of the arm at 90 degrees located at a distance of 1 m, 2 m, and 3 m were made. The illumination was measured using a Ut382 Digital luxmeter (UNI-T, Hong Kong, China ). Two illumination values were defined by the minimum (7 lx) and maximum (73 lx) values provided by Salenec (Sentul, Ciudad de México, Mexico) power fixed LED lamp, which by a slide dimmer (Leviton México, Ciudad de México, Mexico) power fixed LED dimmable lamp. The obtained measured angles are shown in Table 4, where optimum performance is observed at 2 m for both 7 and 73 lx, with low variability and an angle difference of about ±3°.
Fixed angles were marked on a paper background using a universal goniometer (Richardson Products Inc, Gulfstream Rd, Frankfort, KY, USA) as shown in Figure 3. AROM angles were obtained by the Kinect v2 based measurement system. A set of 30 measurements was taken at different angles (30°, 60°, 90°, and 120°) of the left and right shoulder of one participant; also, a set of 30 measurements was taken (45°, 90°, 135°, and 180°) of the elbow of the left and right side, and the standard deviation of the measured angles was ±0.2° on both sides (Table 5).

2.4. Statistical Analysis

In order to verify the joint angle differences between those obtained by the Kinect-based measurement system and those obtained by the goniometer, a Wilcoxon rank-sum test equivalent to the Mann–Whitney U-test was carried out for the data resulting from the control and study groups. The statistical analysis was undertaken using MATLAB (R2017b, MathWorks, Natick, MA, USA). The significance level used was α = 0.05 .

3. Results

Table 5 shows the results obtained from the 30 samples using the Kinect at different angles and compared with those using the universal goniometer. The standard deviation of the measured angles was ±0.2° on both sides. The errors obtained are minimal, with an average difference of ± 1 ° between the actual value and the value estimated by the Kinect-based measurement system.
The measurements of elbow flexion and shoulder abduction movement are presented in Table 6. The difference between the angles obtained by both groups was not significant (p < 0.05). Similarly, there was no significant difference between the manual FMA measurement and FMA with NUI (p < 0.05). The error of the measurement system with NUI is less than ±1°.
The box plot in Figure 4 shows minimal differences between the angular means of the measurements recorded with the Kinect v2 and the goniometer for the right and left limb. As can be observed, the variability between the two devices is not significant. The literal “R” represents the assessment of the right side, and “L” represents the measurements of the left side.
The shoulder abduction and elbow flexion movement measurements for the control and study groups obtained with the NUI system are summarized in Table 7. The difference between the elbow angles obtained in the two groups was not significant (p < 0.05). Similarly, there were considerable differences between the FMA and shoulder abduction measurements between the two groups (p < 0.05).
The box plot in Figure 5 shows significant differences between the angular means of the abduction movement recorded with the Kinect v2 between the control group and the study group; the study group presents difficulties in reaching a complete AROM in the abduction movement. As can be seen, in the elbow flexion movement, the variability between the two groups is not representative; both groups can flex the elbow. The literal “Abd_Ctrl” represents the abduction movement for the control group, “Abd_St” is the abduction movement in the study group. Similarly, the label “Flx_Ctrl” represents the flexion movement in the control group, and “Flx_St” is the flexion movement in the study group.
Similarly, as shown in Figure 6, according to the FMA scale, the motor performance, of the control group and the study group shows significant variability. The literal “FMA-C” represents the FMA score of the control group, and “FMA-S” is the score of the study group.

4. Discussion

The present study enabled angular measurements of the upper limb using the NUI of the Kinect v2 sensor. Data from a universal goniometer were used as a reference to compare upper limb movement analysis measurements.
Concerning the measurements, it is observed in Table 4 that, at one meter, there was a higher variation. The sensor was very close to the participant, and the camera’s field of view did not capture the whole scene, generating flying pixels. Similarly, in the measurement of 3 m, flying pixels were also present. However, the viewing range was higher. Optimum performance was at 2 m at both 7 and 73 lx, showing a minor variability with a difference of ±3°. Our results for distance and height coincide with the work of Cai et al. [21], where they recommend a height of 0.80 cm and a distance of 2 m from the sensor to the participant. However, we added lighting parameters where, according to the characteristics of the infrared camera, RGB camera, and the experiment performed, the proposed system can operate from 7 to 73 lx.
Several studies have shown that upper limb angle measurements based on the Kinect sensor and other reference instruments, such as goniometers or wearable sensors, have a high degree of validity [16,22,40]. The authors Beshara et al. [22] also compared measurements with a goniometer and claimed that the use of wearable inertial sensors in conjunction with Kinect v2 is a reliable and valid way to assess active shoulder flexion and abduction, with minimal measurement errors (2–4°), which represents high reliability. The results of this study show a significant validity of the measurements with the Kinect v2 sensor with an error of ±1° compared to the specialist’s measurements. Our results match the authors’ claims [16] about the reliability of measures with the Kinect sensor, where their study demonstrated an accuracy of ±5°.
Concerning the validity of the measurement system, no significant differences were found (Table 6, p > 0.005) between flexion and abduction movements of both shoulder and elbow. Therefore, the AROM measurements of the elbow and shoulder are reliable. Overall, these results indicate that the algorithm programmed with the FMA scale is an effective and alternative tool for determining upper limb motor performance.
By comparison, as shown in Table 7, significant differences were found (p > 0.005) between the control group and the study group. The study group, representing a particular case of hemiparesis in the upper extremity, presented a lower motor performance than the control group. A share of 78% showed a total performance, according to the FMA scoring (52–66), 11% a notable capacity (47–52), and 11% a limited capacity (31–47). However, 83% of the cases presented difficulties in performing the activities during the test, mainly in the hand, wrist, and type of grip. Finally, 11% did not achieve a maximum score due to a lack of coordination and speed. This system has the advantage of obtaining angular measurements at 2 m, i.e., without direct contact with the participant. It also semi-automatically measures a level of upper limb motor skills. Unfortunately, in 2017, Kinect v2 sensors (Microsoft, Redmond, WA, USA) were discontinued. However, a viable replacement may be the Azure Kinect (Microsoft, Redmond, WA, USA) [17].
It should be noted that this study was carried out with a small sample size. However, this population was appropriate, particularly when eliminating cognitive and functional deficits that can intervene in the evaluation. The developed assessment system is semi-automatic because it only acquires real-time angles involved in the FMA assessment; the other parameters, such as sensitivity or pain, are entered into the system manually by the specialist. The assessment of motor performance with the FMA scale manually requires approximately 30 min per patient; however, with the semi-automatic method, the time is considerably reduced, to around 5–10 min, depending on the GMFCS and MACS level, and the cognitive level of the patient. For this reason, it is essential to consider these visuomotor factors; otherwise, it can complicate the assessment, as not all patients can follow instructions and perform active movements. Nevertheless, one of the benefits for specialists using the semi-automatic method is quantitative and distance measurement, with less angular measurement error than using a goniometer, as the accuracy of this instrument depends on the experience of use between each evaluator. The implementation of this measurement system is relatively simple, first requiring an exclusive area of 3 × 3 m, with illumination of 7–73 lx, and the relevant equipment; at the time of the evaluation, a distance of 2 m between the sensor and the patient must be considered. Regarding the large-scale implementation of the system, training for technicians is required. For the correct use of the evaluation system, it is also necessary to provide the relevant indications to the specialist or person in charge of the rehabilitation area. This training is left as future work, considering a video tutorial, and providing a system manual for the end-user. It is worth mentioning that the assessment instructions are available at the top of the graphical interface. Similarly, the controls of the latest developed system version indicate the labels of the scale, which a specialist will be familiar with due to familiarity with this type of clinical scale for evaluation of motor performance, i.e., in this case, FMA. Future work will also include Constraint-Induced Movement Therapy (CIMT) in the study group. The results obtained will be used as a basis for comparing measurements before and after the CIMT rehabilitation process to effectively and quantitatively assess progress.

5. Conclusions

In both groups, this study showed a significant agreement between the measurements of the angles (abduction and flexion) of the upper limbs (shoulder and elbow) performed with the Kinect v2 sensor and those made manually with a universal goniometer. Our system differed by ±1° compared to the measurements made by the specialist, which represents a small difference. However, the proposed system provides greater accuracy and a more reliable FMA score, allowing the specialist to focus on specific parts of the upper limb for effective rehabilitation.
The proposed system is an effective alternative clinical measurement tool for the analysis of upper limb motor performance. It allows the therapist to analyze the AROM and, at the same time, determine the upper limb motor performance level of the FMA patient. We recommend conducting the evaluations in a controlled area of 3 × 3 m, considering the illumination factor of 73 lx, a distance between the sensor and the participant of 2 m, a sensor height of 0.80 m, and pre-warming of the Kinect, as mentioned above. However, we suggest implementing this evaluation methodology with the available Kinect sensors or with the new versions of Kinect (Azure Kinect (Microsoft, Redmond, WA, USA)).
Finally, the novelty of this system lies in the assessment of AROM and motor performance in children with spastic hemiparesis assisted with the NUI of the Kinect v2 sensor. Tests were conducted with a universal goniometer and supported by clinical scales, such as MACS, GFMCS, FMA, and the Bender Koppitz test. The system also allows distancing strategies to be implemented, i.e., the motor assessment is carried out without direct contact with the patient, thus reducing the probability of infection with the COVID-19 virus.

Author Contributions

Conceptualization, C.F.-M. and J.P.-O., methodology, C.F.-M., J.P.-O. and J.A.P.-M.; Software, C.F.-M.; validation, C.F.-M., J.P.-O., J.A.P.-M., A.I.B.-G., F.J.P.-P. and J.J.M.-N.; formal analysis, C.F.-M., J.P.-O., J.A.P.-M., F.J.P.-P., A.I.B.-G. and J.J.M.-N.; investigation, C.F.-M.; writing—original draft preparation, C.F.-M.; writing—review and editing, C.F.-M., J.P.-O. and J.A.P.-M.; visualization, J.A.P.-M., A.I.B.-G., F.J.P.-P. and J.J.M.-N. All authors have read and agreed to the published version of the manuscript.

Funding

MII. Celia Francisco received a PhD scholarship from the Consejo Nacional de Ciencia y Tecnología (Conacyt), N: 005620, CVU: 791523. The funder had no role in the design of the study or collection, analysis, or interpretation of data or in writing the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to acknowledge Karla S. Morales Soto, Coordinator of Early Stimulation and Occupational Therapy Department of Association for People with Cerebral Palsy (APAC for its Spanish acronym), for her valuable support in the assessments in both studies groups of this research. We acknowledge Lourdes S. Alva Álvarez and Yolanda López Aranda for their participation in the measurements carried out in this study. We acknowledge the Manuscript Writing Training Team (CEMAI for its Spanish acronym) of Conacyt for their help with reviews and constructive criticism of the research papers.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

AROMActive range of motion
PROMPassive range of motion
ROMRange of motion
FMAFugl-Meyer Assessment
NUINatural user interface

References

  1. World Health Organization. COVID-19 Strategic Preparedness and Response Plan (SPRP 2021). Available online: https://www.who.int/publications/i/item/WHO-WHE-2021.02 (accessed on 3 March 2021).
  2. Demers, M.; Martinie, O.; Winstein, C.; Robert, M.T. Active Video Games and Low-Cost Virtual Reality: An Ideal Therapeutic Modality for Children With Physical Disabilities During a Global Pandemic. Front. Neurol. 2020, 11, 601898. [Google Scholar] [CrossRef] [PubMed]
  3. World Health Organization. Disability and Health. Available online: https://www.who.int/news-room/fact-sheets/detail/disability-and-health (accessed on 8 March 2021).
  4. Pavão, S.L.; Lima, C.R.G.; Rocha, N. Association between sensory processing and activity performance in children with cerebral palsy levels I-II on the gross motor function classification system. Braz. J. Phys. Ther. 2021, 25, 194–202. [Google Scholar] [CrossRef] [PubMed]
  5. Compagnone, E.; Maniglio, J.; Camposeo, S.; Vespino, T.; Losito, L.; De Rinaldis, M.; Gennaro, L.; Trabacca, A. Functional classifications for cerebral palsy: Correlations between the gross motor function classification system (GMFCS), the manual ability classification system (MACS) and the communication function classification system (CFCS). Res. Dev. Disabil. 2014, 35, 2651–2657. [Google Scholar] [CrossRef] [PubMed]
  6. Krasowicz, K.; Michoński, J.; Liberadzki, P.; Sitnik, R. Monitoring Improvement in Infantile Cerebral Palsy Patients Using the 4DBODY System—A Preliminary Study. Sensors 2020, 20, 3232. [Google Scholar] [CrossRef] [PubMed]
  7. Ben-Pazi, H.; Beni-Adani, L.; Lamdan, R. Accelerating Telemedicine for Cerebral Palsy During the COVID-19 Pandemic and Beyond. Front. Neurol. 2020, 11, 746. [Google Scholar] [CrossRef] [PubMed]
  8. Norkin, C.C.; White, D.J. Measurement of Joint Motion: A Guide to Goniometry, 5th ed.; F.A. Davis Company: Philadelphia, PA, USA, 2016. [Google Scholar]
  9. Pérez-de la Cruz, S.; de León, Ó.A.; Mallada, N.P.; Rodríguez, A.V. Validity and intra-examiner reliability of the Hawk goniometer versus the universal goniometer for the measurement of range of motion of the glenohumeral joint. Med. Eng. Phys. 2021, 89, 7–11. [Google Scholar] [CrossRef] [PubMed]
  10. Fard, M.K.; Fallah, A.; Maleki, A. The compensation of biomechanical errors in electrogoniometric measurements of the upper extremity kinematics. Sens. Actuators A Phys. 2020, 315, 112170. [Google Scholar] [CrossRef]
  11. Elgendy, M.H.; El-Khalek, W.O.A.A. Validity and intra-rater reliability of laser goniometer versus electro-goniometer in measuring shoulder range of motion. Int. J. Physiother. 2019, 6, 169–176. [Google Scholar] [CrossRef]
  12. Agustín, M.S.; García-Vidal, J.A.; Cánovas-Ambit, G.; Vecchia, A.D.; López-Nicolás, M.; Medina-Mirapeix, F. Validity and Reliability of a New Optoelectronic System for Measuring Active Range of Motion of Upper Limb Joints in Asymptomatic and Symptomatic Subjects. J. Clin. Med. 2019, 8, 1851. [Google Scholar] [CrossRef] [Green Version]
  13. Poitras, I.; Dupuis, F.; Bielmann, M.; Campeau-Lecours, A.; Mercier, C.; Bouyer, L.J.; Roy, J.-S. Validity and Reliability of Wearable Sensors for Joint Angle Estimation: A Systematic Review. Sensors 2019, 19, 1555. [Google Scholar] [CrossRef] [Green Version]
  14. Rigoni, M.; Gill, S.; Babazadeh, S.; Elsewaisy, O.; Gillies, H.; Nguyen, N.; Pathirana, P.N.; Page, R. Assessment of Shoulder Range of Motion Using a Wireless Inertial Motion Capture Device-A Validation Study. Sensors 2019, 19, 1781. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Mourcou, Q.; Fleury, A.; Diot, B.; Franco, C.; Vuillerme, N. Mobile Phone-Based Joint Angle Measurement for Functional Assessment and Rehabilitation of Proprioception. BioMed Res. Int. 2015, 2015, 328142. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Çubukçu, B.; Yüzgeç, U.; Zileli, R.; Zileli, A. Reliability and validity analyzes of Kinect V2 based measurement system for shoulder motions. Med. Eng. Phys. 2020, 76, 20–31. [Google Scholar] [CrossRef]
  17. Francisco-Martínez, C.; Prado-Olivarez, J.; Padilla-Medina, J.A.; Díaz-Carmona, J.; Pérez-Pinal, F.J.; Barranco-Gutiérrez, A.I.; Martínez-Nolasco, J.J. Upper Limb Movement Measurement Systems for Cerebral Palsy: A Systematic Literature Review. Sensors 2021, 21, 7884. [Google Scholar] [CrossRef]
  18. Lachat, E.; Macher, H.; Landes, T.; Grussenmeyer, P. Assessment and calibration of a RGB-D camera (Kinect v2 sensor) towards a potential use for close-range 3D modeling. Remote Sens. 2015, 7, 13070–13097. [Google Scholar] [CrossRef] [Green Version]
  19. Guzsvinecz, T.; Szucs, V.; Sik-Lanyi, C. Suitability of the Kinect Sensor and Leap Motion Controller—A Literature Review. Sensors 2019, 19, 1072. [Google Scholar] [CrossRef] [Green Version]
  20. Timmi, A.; Coates, G.; Fortin, K.; Ackland, D.; Bryant, A.L.; Gordon, I.; Pivonka, P. Accuracy of a novel marker tracking approach based on the low-cost Microsoft Kinect v2 sensor. Med. Eng. Phys. 2018, 59, 63–69. [Google Scholar] [CrossRef]
  21. Cai, L.; Ma, Y.; Xiong, S.; Zhang, Y. Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor. Appl. Bionics Biomech. 2019, 2019, 7175240. [Google Scholar] [CrossRef] [Green Version]
  22. Beshara, P.; Chen, J.F.; Read, A.C.; Lagadec, P.; Wang, T.; Walsh, W.R. The Reliability and Validity of Wearable Inertial Sensors Coupled with the Microsoft Kinect to Measure Shoulder Range-of-Motion. Sensors 2020, 20, 7238. [Google Scholar] [CrossRef]
  23. Bonnechère, B.; Jansen, B.; Salvia, P.; Bouzahouene, H.; Omelina, L.; Moiseev, F.; Sholukha, V.; Cornelis, J.; Rooze, M.; van Sint Jan, S. Validity and reliability of the Kinect within functional assessment activities: Comparison with standard stereophotogrammetry. Gait Posture 2014, 39, 593–598. [Google Scholar] [CrossRef]
  24. Scano, A.; Mira, R.M.; Cerveri, P.; Molinari Tosatti, L.; Sacco, M. Analysis of Upper-Limb and Trunk Kinematic Variability: Accuracy and Reliability of an RGB-D Sensor. Multimodal Technol. Interact. 2020, 4, 14. [Google Scholar] [CrossRef]
  25. Wilson, J.D.; Khan-Perez, J.; Marley, D.; Buttress, S.; Walton, M.; Li, B.; Roy, B. Can shoulder range of movement be measured accurately using the Microsoft Kinect sensor plus Medical Interactive Recovery Assistant (MIRA) software? J. Shoulder Elb. Surg. 2017, 26, e382–e389. [Google Scholar] [CrossRef]
  26. Da Cunha Neto, J.S.; Filho, P.P.R.; da Silva, G.P.F.; da Cunha Olegario, N.B.; Duarte, J.B.F.; de Albuquerque, V.H.C. Dynamic Evaluation and Treatment of the Movement Amplitude Using Kinect Sensor. IEEE Access 2018, 6, 17292–17305. [Google Scholar] [CrossRef]
  27. Huber, M.; Seitz, A.; Leeser, M.; Sternad, D. Validity and reliability of Kinect skeleton for measuring shoulder joint angles: A feasibility study. Physiotherapy 2015, 101, 389–393. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Rech, K.D.; Salazar, A.P.; Marchese, R.R.; Schifino, G.; Cimolin, V.; Pagnussat, A.S. Fugl-Meyer Assessment Scores Are Related With Kinematic Measures in People with Chronic Hemiparesis after Stroke. J. Stroke Cereb. Dis. 2020, 29, 104463. [Google Scholar] [CrossRef] [Green Version]
  29. Olesh, E.V.; Yakovenko, S.; Gritsenko, V. Automated Assessment of Upper Extremity Movement Impairment due to Stroke. PLoS ONE 2014, 9, e104487. [Google Scholar] [CrossRef]
  30. Kim, W.-S.; Cho, S.; Baek, D.; Bang, H.; Paik, N.-J. Upper Extremity Functional Evaluation by Fugl-Meyer Assessment Scoring Using Depth-Sensing Camera in Hemiplegic Stroke Patients. PLoS ONE 2016, 11, e0158640. [Google Scholar] [CrossRef]
  31. Otten, P.; Kim, J.; Son, S.H. A Framework to Automate Assessment of Upper-Limb Motor Function Impairment: A Feasibility Study. Sensors 2015, 15, 20097–20114. [Google Scholar] [CrossRef] [Green Version]
  32. Lee, S.; Lee, Y.S.; Kim, J. Automated Evaluation of Upper-Limb Motor Function Impairment Using Fugl-Meyer Assessment. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 125–134. [Google Scholar] [CrossRef]
  33. Lee, S.-H.; Hwang, Y.-J.; Lee, H.-J.; Kim, Y.-H.; Ogrinc, M.; Burdet, E.; Kim, J.-H. Proof-of-Concept of a Sensor-Based Evaluation Method for Better Sensitivity of Upper-Extremity Motor Function Assessment. Sensors 2021, 21, 5926. [Google Scholar] [CrossRef]
  34. Tafti, M.A.; Azizi, Z.R.; Mohamadzadeh, S. A comparison of the diagnostic power of FEATS and Bender-Gestalt test in identifying the problems of students with and without specific learning disorders. Arts Psychother. 2021, 73, 101760. [Google Scholar] [CrossRef]
  35. Hoonhorst, M.H.; Nijland, R.H.; van den Berg, J.S.; Emmelot, C.H.; Kollen, B.J.; Kwakkel, G. How Do Fugl-Meyer Arm Motor Scores Relate to Dexterity According to the Action Research Arm Test at 6 Months Poststroke? Arch. Phys. Med. Rehabil. 2015, 96, 1845–1849. [Google Scholar] [CrossRef] [PubMed]
  36. Hwang, S.; Tsai, C.Y.; Koontz, A.M. Feasibility study of using a Microsoft Kinect for virtual coaching of wheelchair transfer techniques. Biomed. Tech. 2017, 62, 307–313. [Google Scholar] [CrossRef] [PubMed]
  37. Wu, G.; van der Helm, F.C.T.; Veeger, H.E.J.; Makhsous, M.; van Roy, P.; Anglin, C.; Nagels, J.; Karduna, A.R.; McQuade, K.; Wang, X.; et al. ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion—Part II: Shoulder, elbow, wrist and hand. J. Biomech. 2005, 38, 981–992. [Google Scholar] [CrossRef] [PubMed]
  38. Wasenmüller, O.; Stricker, D. Comparison of kinect v1 and v2 depth images in terms of accuracy and precision. In Proceedings of the Computer Vision—ACCV 2016 Workshops, Taipei, Taiwan, 20–24 November 2016; Springer: Cham, Switzerland, 2017; pp. 34–45. [Google Scholar]
  39. Jiao, J.; Yuan, L.; Tang, W.; Deng, Z.; Wu, Q. A Post-Rectification Approach of Depth Images of Kinect v2 for 3D Reconstruction of Indoor Scenes. ISPRS Int. J. Geo-Inf. 2017, 6, 349. [Google Scholar] [CrossRef] [Green Version]
  40. Xu, X.; McGorry, R.W. The validity of the first and second generation Microsoft Kinect™ for identifying joint center locations during static postures. Appl. Ergon. 2015, 49, 47–54. [Google Scholar] [CrossRef]
Figure 1. Graphical user interface.
Figure 1. Graphical user interface.
Sensors 22 02258 g001
Figure 2. Programming process block diagram.
Figure 2. Programming process block diagram.
Sensors 22 02258 g002
Figure 3. Angle setting.
Figure 3. Angle setting.
Sensors 22 02258 g003
Figure 4. Variability in abduction movement.
Figure 4. Variability in abduction movement.
Sensors 22 02258 g004
Figure 5. AROM measurements.
Figure 5. AROM measurements.
Sensors 22 02258 g005
Figure 6. FMA measurements.
Figure 6. FMA measurements.
Sensors 22 02258 g006
Table 1. Selection criteria.
Table 1. Selection criteria.
Control groupInclusion criteriaChildrenHealthy
Age4–12 years
Cognitive profileAcceptable (Bender Koppitz Test)
Informed consentAuthorized
Study groupInclusion criteriaChildrenDiagnosis of spastic hemiparesis due to CP
Age4–12 years
Cognitive profileAcceptable (Bender Koppitz Test)
GMFCSLevel I–II
MACSLevel I–II
Botulinum toxinWithout or more than six months after application
Both groupsExclusion criteriaParticipants with visual, hearing, and severe cognitive impairment.
Rejection criteriaUnauthorized informed consent. Participants with general ailments; undergoing pharmacological, medical treatment, and musculoskeletal injuries.
GMFCS: Gross Motor Function Classification System; MACS: Manual Ability Classification System; CP: cerebral palsy.
Table 2. Characteristics of the study population.
Table 2. Characteristics of the study population.
CGP123456789101112131415161718
FMA656465666665646666666363636366666666
CA4455567888999911111112
MA4455567888999911111112
SGFMA576054506365396452535742656562663158
CA4556688899999910111212
MA-44466575-75845789
MACSIIIIIIIIIIIIIIIIIIIIIIIIII
GMFCSIIIIIIIIIIIIIIIIIIIIIIIIIIII
CG: control group; SG: study control; P: participants; CA: chronological age; MA: mental age—able to follow instructions.
Table 3. Joints for AROM calculation.
Table 3. Joints for AROM calculation.
JointMovementSegments
ShoulderAbduction Hip, Shoulder, Elbow
AdductionHand, Shoulder, Hip
ElbowFlexionShoulder, Elbow, Hand
ExtensionHand, Elbow y Shoulder
Table 4. Variability in distance and illumination.
Table 4. Variability in distance and illumination.
Illumination 7 lx
Meters1 m2 m3 m
Mean   ±   SD 94.19   ± 5.64 89.74   ± 2.94 89.33   ± 4.77
Illumination 73 lx
Mean   ±   SD 111.32   ± 4.01 91.65   ± 2.06 95.96   ± 1.96
SD: standard deviation.
Table 5. Results of the averages for the right and left limbs.
Table 5. Results of the averages for the right and left limbs.
Assessment VariableDegrees °MeanAbsolute ErrorRelative Error (%)
RightShoulder
(Abduction)
3030.5490.5491.83
6059.9940.0050.00
9090.4250.4250.47
120120.5460.5460.45
4545.6640.6641.47
Elbow (Flexion)9090.6240.6240.69
135135.4490.4490.33
180180.5480.5480.30
LeftShoulder
(Abduction)
3030.2570.2570.85
6060.5900.5900.98
9090.5140.5140.57
120120.8760.8760.73
Elbow (Flexion)4545.5400.5401.20
9090.5040.5040.56
135135.5110.5110.37
180180.6030.6030.33
Table 6. Comparison of joint angles of the upper limb.
Table 6. Comparison of joint angles of the upper limb.
GroupVariableROMSystem Mean   ±   SD H0p-Value
ControlShoulder
abduction
Right AROMNUI 108.51   ± 19.67 01
Right PROMGoniometer 108.50   ± 19.62
Left AROMNUI 104.01   ± 23.44 00.98
Left PROMGoniometer 104.05   ± 23.44
Elbow
flexion
Right AROMNUI 100.85   ± 37.88 00.98
Right PROMGoniometer 100.77   ± 37.88
Left AROMNUI 94.25   ± 34.73 00.96
Left PROMGoniometer 94.27   ±   34.70
FMA NUI 65   ± 1 00.12
Goniometer 65   ± 2
StudyShoulder
abduction
AROMNUI 81.60   ± 26.94 00.21
PROMGoniometer 81   ± 27.57
Elbow
flexion
AROMNUI 116.15   ± 44.09 00.19
PROMGoniometer 115.50 ± 43.88
FMA NUI 56   ± 10 00.78
Goniometer 56   ± 9
AROM: active range of motion; PROM: passive range of motion; NUI: natural user interface; H0 = 0: There are no differences between the measurements of the two instruments.
Table 7. Control group vs. study group measurements.
Table 7. Control group vs. study group measurements.
VariableGroup Mean   ±   SD H0p-Value
Shoulder
abduction
Control 108.51   ± 19.67 10.0043
Study 81.60   ± 26.94
Elbow
flexion
Control 100.85   ± 37.88 00.2145
Study 116.15   ± 44.09
FMAControl 65   ± 1 10.0004
Study 56   ± 9
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Francisco-Martínez, C.; Padilla-Medina, J.A.; Prado-Olivarez, J.; Pérez-Pinal, F.J.; Barranco-Gutiérrez, A.I.; Martínez-Nolasco, J.J. Kinect v2-Assisted Semi-Automated Method to Assess Upper Limb Motor Performance in Children. Sensors 2022, 22, 2258. https://doi.org/10.3390/s22062258

AMA Style

Francisco-Martínez C, Padilla-Medina JA, Prado-Olivarez J, Pérez-Pinal FJ, Barranco-Gutiérrez AI, Martínez-Nolasco JJ. Kinect v2-Assisted Semi-Automated Method to Assess Upper Limb Motor Performance in Children. Sensors. 2022; 22(6):2258. https://doi.org/10.3390/s22062258

Chicago/Turabian Style

Francisco-Martínez, Celia, José A. Padilla-Medina, Juan Prado-Olivarez, Francisco J. Pérez-Pinal, Alejandro I. Barranco-Gutiérrez, and Juan J. Martínez-Nolasco. 2022. "Kinect v2-Assisted Semi-Automated Method to Assess Upper Limb Motor Performance in Children" Sensors 22, no. 6: 2258. https://doi.org/10.3390/s22062258

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop