Biometric Image-Analysis Techniques for Monitoring Chronic Neck Pain

Abstract

The term “mechanical neck pain” is a generic term used to define neck pain in people with neck injuries, neck dysfunction, or shoulder and neck pain. Several factors must be considered during the physical-therapy evaluation of cervical disorders, including changes in the visual systems and postural and proprioceptive balance. Currently, the Cervicocephalic Relocation Test (CRT) is used by physiotherapists to detect changes in cervical proprioception. This procedure requires precise equipment, customized installation in a dedicated area and, above all, a significant amount of time post-treatment for the doctor to make the diagnosis. An innovative system composed of Google’s MediaPipe library combined with a personal laptop and camera is proposed and evaluated. The system architecture was developed, and a user interface was designed with the goal of allowing the system to be used more easily, more quickly, and more effectively by the healthcare practitioner. The tool is presented in this paper and tested in a use case, and the results are presented. The final user report, containing the visualization of the results of the CRT, which are ready for analysis by the physical therapist, can be exported from the developed tool.

1. Introduction

In the context of cervical proprioceptive dysfunction, lower back pain and neck pain are among the most common health-related reasons for performing medical exams.

As Zennaro et al. [1] and Moley [2] noted, injuries and wear and tear can often occur because the neck is flexible and supports the head. Rix and Bagust [3] emphasized that greater attention has been given to this dysfunction, which is known as cervical proprioceptive syndrome. According to Pinsault et al. [4], the Cervicocephalic Relocation Test (CRT) [5] is used as a tool to detect related pathologies in cervical proprioception. It consists of positioning the head neutrally, blindfolding the patient, and then performing a rotation of the head to the left (and then to the right) a set number of times, with a subsequent return to the starting position. This procedure allows the evaluator to recover data relating to the initial and final positions of the movements, providing a measure of cervical proprioceptive acuity.

This evaluation of patients’ cervical proprioception has been proposed in several research studies. In [6], researchers used the target head-repositioning technique with elderly individuals with chronic neck pain (CNP). The objectives were to (a) compare the cervical proprioception and functional balance between patients with CNP and asymptomatic individuals, (b) investigate the relationship between cervical proprioception and functional balance ability in individuals with CNP. In this study, a cervical range of motion (CROM) instrument was utilized to evaluate cervical proprioception in the flexion, extension, and right and left rotation directions, as follows:

-

Step 1: examiner guides the participant’s neck from the starting position to the target position;

-

Step 2: patient’s neck returns to the starting position after the patient has memorized the target position;

-

Step 3: active repositioning to the target position by the participant.

CROM is a valid and reliable instrument for measuring cervical range of motion and proprioception in individuals with and without neck symptoms [7,8]. The unit is helmet-shaped and includes three inclinometers to measure the range of motion in sagittal and transverse planes.

Other studies measured the joint repositioning error [9] and cervical range of motion (ROM) [10]. In the first paper, the purpose was to compare the effects of specific neck-muscle training and general neck-shoulder exercises on neck proprioception, pain, and disability in patients. Twenty-five patients with chronic non-specific neck pain were recruited into a preliminary single-blinded randomized clinical trial. Joint-repositioning error for neck rotation was measured based on the methods first described by Revel et al. [5]. Participants were invited to sit on a chair located 90 cm from a white wall. A laser-beam pointer was fastened by an elastic strap to their heads. They were asked to keep their heads in a relaxed and neutral position while they looked straight forward at the wall (Figure 1). The laser light on the wall was marked as the reference point. Then, patients were asked to turn their head to the right and to the left to the end of available range, performing each movement once, and return to the original position with eyes open, attempting to align the laser light with the reference point. Next, their eyes were covered by a blindfold and they were asked to repeat the procedure with their eyes closed. While returning their heads to the original position, participants were asked to inform the examiner when they reached the original position. The new laser point on the wall was marked as the target point. Joint position error was calculated in degrees as the arctangent of the distance between the target and reference point in cm divided by 90 cm.

In the second paper, authors described using an iPhone 7 (Apple Inc., Cupertino, CA, USA) fixed on the patient’s head for a test of Cervical Range of Motion in lateral rotation [11]. The Smartphone Inclinometer Application was used to quantify Cervical spine mobility. Twenty-three individuals were asked to perform neck maximal (end-range) movements (frontal flex-extension and left-right side flexion in the sitting position, left-right rotation in the prone position). This study confirmed that the tested smartphone can provide a valid and reliable measure of ACROM on the frontal and the sagittal planes; furthermore, it also demonstrated its usefulness for rotational-movement analysis using the inclinometric application. This research illustrates the tendency to use new technology in the medical field for diagnosis.

The CRT is essential for determining the condition of people with cervical proprioceptive dysfunction. Although the CRT is a worldwide reference test, it requires setup and preparation of the experiment with the patient, the use of excessive space, a helmet and laser technology, and, above all, significant time post-treatment to allow the specialist to generate results. This research presents a low-cost and easy-to-use tool that can become a new support for specialists who have to evaluate dysfunction of cervical proprioception via the CRT.

The technology presented in this document was developed by considering proposals such as those by Dror et al. [12], Shao et al. [13], and Li et al. [14], which discuss the importance of “the ability to involve the physical body in an interaction with technology”. According to this idea, they proposed a noninvasive system based on Microsoft’s Kinect (Microsoft, Redmond, WA, USA) depth sensor and using a new tracking algorithm based on motion capture (MoCap). They proposed several applications to measure and record the position and orientation of the human body in motion.

Other authors, such as Castro et al. [15] and McDuff [16], have described other applications of MoCap technology, such as detecting heart rate, detecting an individual’s balance, identifying seizures, and analyzing disorders resulting from neurodegenerative diseases. In particular, this work was inspired by the proposals of Hanke et al. [17] and Brook et al. [18], who discussed the possibility of objectively assessing the gaits of patients with neurodegenerative diseases in controlled environments to quantify these anomalies using Kinect.

Proposals of low-cost solutions for treatment by specialists in monitoring cervical proprioceptive acuity involve the use of motion sensors in combination with new techniques, such as artificial intelligence [19,20]. Researchers have developed solutions to carry out low-cost, efficient, and accurate analyses that aid in the treatment of various dysfunctions. Applying these methods can further improve the ability to analyze complex multimodal imaging data and increase the efficiency of these diagnoses [11,21].

The analysis of the studies mentioned above shows that there are opportunities to use low-cost motion sensors for measuring and evaluating patients’ performances for specific evaluations that otherwise require expensive facilities. These new solutions can provide reliable support for experts. For other patient evaluations that are based on qualitative observation or that require complex installations, some of the testing equipment developed using new technologies is not only cheap, but also easy to install and handle. Most of the time, the new testing approaches also make it easier to quantify assessments and process data after the test.

For the CRT, the equipment used is not very expensive: a helmet, one target on the wall, one laser, a pen for marking points, and a dedicated room. The main issue arises during post-processing of the information collected. The innovative tool allows the practitioner to immediately obtain the data needed to make his diagnosis. Thus, this article presents materials and methods used to develop such an innovation dedicated to the CRT. The architecture of the proposition and the calculation model are presented, too. The formalization of the results aligns with the physicians’ needs to allow the diagnosis of cervical proprioceptive syndrome.

2. Materials and Methods

This section introduces the main ideas and concepts used to develop the tool proposed for analyzing cervical proprioceptive dysfunction.

2.1. Medical Protocol

As mentioned in the introduction, a clinical technique known as CRT is used to assess sensory-motor control disorders in people with mechanical neck pain or cervical kinesthesia. The patient undergoing the test must wear a cycling helmet with a laser pointer attached to the top. He is fitted with a mask to block his vision and has to stare at a target 90 cm in front of him. The target is made of graph paper (40 × 40 cm²), with the horizontal (x) and vertical (y) axes drawn so that origin of the axes divides the paper into four equal quadrants. After he has positioned the laser beam on the target, the patient is asked to memorize this position, perform repeated rotations to the left, and return to the starting position as accurately as possible. After 2 min of rest, the procedure is carried out with rotations to the right. At each stopping point, the beam indicated on the target is marked with a pen and labeled with the current number of repetitions and the direction of rotation.

This procedure can be time-consuming and prone to errors. To offer an option for achieving better results with greater efficiency, this paper proposes modifying the equipment by integrating a laptop and camera to make it easier to carry out the CRT analysis while maintaining the operating procedure, which has been validated by the specialist community at international level. In this way, the procedure can be performed with simple equipment, facilitating its use by various specialists and may speed up data-analysis procedures in real time cases. Because it is run with the support of a computer, the application provides instant analysis of the stored and collected data.

2.2. Motion Sensors

Stereoscopic image-detection technologies use sensors with at least two cameras to calculate depth. These sensors can visualize, understand, interact with, and learn from their environment in 3D.

The tool presented in this article is a version that evolved from the initial use of a Kinect sensor and that required various studies and analyses. Devices similar to the Kinect, such as Microsoft Azure Connecta (Microsoft, Redmond, WA, USA), Intel RealSense (Intel, Santa Clara, CA, USA) and ASUS Xtion (Asus, Taipei, Taiwan), which appeared after Kinect, enable detailed and accurate data to be retrieved. These devices, which are relatively similar in their technologies, are easy to install, enable data to be analyzed and interpreted quickly, and offer support for different operating systems and programming languages.

Tests using these sensors were carried out while a laptop camera was being used. Analyses of the data from the various cameras and consultations with the experts carrying out the CRTs led to the choice of the laptop camera for use in the development of our innovative solution.

2.3. Computer Vision and MediaPipe

Computer vision is one of the most innovative and fascinating areas of modernization, as it allows us, for example, to create applications with the ability to perceive the world around us. These applications involve using sensors and developing algorithms and machine-learning models to perceive an environment and capture it. As [13,22] stated, computer vision captures, processes, and interprets light rays from the outside world.

Image-analysis applications have shown great potential in the medical field, as they help significantly in the diagnosis of various diseases. Their use can further enhance the ability to analyze complex multimodal image data and improve the efficiency of diagnosis [21]. The possibilities are even greater when one can perform “dynamic analysis” of images involving gestural navigation or the ability to identify and track the human body. These methods involve carrying out basic machine-learning (ML) tasks, such as manual tracking, which consumes considerable time and resources.

In 2015, Google developers created a tool called TensorFlow Lite [22], an open-source library for ML applications on mobile, embedded, and IoT devices and computers. It facilitates computer vision, allowing developers to run their trained models on devices. This tool facilitated the development of MediaPipe by Google in 2019.

MediaPipe is an open-source, high-precision framework that offers access to various machine-learning models, allowing data capture with hardware acceleration and model optimization [23,24]. It enables the creation of pipelines with ML applied to multiple modalities (audio, video). It has an extensive collection of models for detection and tracking of the human body, wherein the points are normalized in three dimensions. With this approach, inference can be performed on arbitrary data, including perception with modular graphical components, model inference, media-processing algorithms, and data transformation [25]. It can be integrated into any tool using programs in different languages, such as Python, Java, and C/C++. In August 2020, Google provided open access to the code, which attracted the attention of developers of devices that use depth cameras in their applications.

The framework integrates a Face Mesh solution, which performs facial-landmark detection. It detects, in real time, an individual’s face in the input image and estimates 468 3D facial landmarks (Figure 2) as face-blend shapes. The reference at each point comprises data from the x, y and z axes. The values of x and y are normalized [0.0, 1.0] for the image width and height; z represents its depth, and the closer the object is to the camera, the smaller this value z will be.

As reported by [26,27], MediaPipe has been applied as a sensor for face detection in mobile devices. Advances in technologies applied to cameras have enabled an increase in their integration into wearable platforms, demonstrating a great potential for exploration. The tool proposed in this article is one of the possible uses of MediaPipe, combined with a laptop camera. The laptop camera has an HD resolution of 1280 × 720 and the ability to capture video at 30 fps. MediaPipe has a maximum resolution of 640 × 420 at 30 fps, so testing with the laptop camera was reasonable and proved suitable for the system under development.

2.4. Data Analysis

The measurement process carried out with the proposed tool follows the principles of the CRT. An individual wearing a helmet with a laser attached is positioned in front of a laptop with a sheet of graph paper positioned above the laptop screen, as shown in Figure 3. To meet the system’s objectives, experiments and tests were carried out to determine the best distance between the camera and the individual, and a distance of 80 cm was chosen.

As shown in Figure 2, MediaPipe Face Mesh detects an individual’s face in an input image and estimates 468 3D facial reference points. In the application developed, only point 94 (Figure 4), which corresponds to the center point of the nose (Nose Bottom) and is mentioned in [25], is used.

3. The Developed Tool

This section presents the tool that was developed. It shows the process it uses, one of its screens, and the procedure for collecting data.

3.1. System Architecture

Figure 5 shows the working principle of the tool developed with a standard RGB HD notebook camera and the program designed in JavaScript. Initially, an image is captured by the laptop camera; subsequently, using the MediaPipe application, the face is identified and the points are captured. The software then processes these data and indicates the distance between the first and second readings. These obtained values are visualized and presented by the software in the form of data and graphs immediately after the position of the nose has been captured. They can also be stored in a file to be analyzed later.

3.2. Current Version of the Tool

The system home screen shows the options “Home”, “Search-FaceMesh”, and “FaceMesh”. The “Home” option always returns to the home screen. The “FaceMesh” option gives access to the screen shown in Figure 6. Only the “Get Camera” option (shown on the left) is displayed initially. After the camera has been selected, the Panel screen (shown on the right) appears. From there, one can define the direction of rotation (the “Selected Test” option) and the number of readings (the “Average Points” option) that must be performed. One can access the data-storage and reading-configuration options on the same screen. Once the direction of rotation and the number of readings have been defined, the “Start” button must be activated for the readings to be taken.

To implement the test, an individual is asked to wear a helmet (Figure 3) and to position the light beam at a point on the sheet of graph paper; this position is subsequently stored. Then, the individual is asked to move their head to position the light beam approximately 5 cm away from the starting point (to the left or right—as chosen), and the new point is captured. The process is repeated until all the measurements have been taken.

A distance of 5 cm was defined based on references [3,5], which noted that an angle greater than 4 degrees (obtained with a 5 cm displacement) indicates that the individual may have problems with cervicocephalic kinesthetic sensitivity. Although the tests carried out in the articles use a distance of 90 cm from the sheet of graph paper, after tests were carried out with MediaPipe, a distance of 80 cm from the face to the camera was chosen, ensuring an angular displacement of 4 degrees.

To better understand the calculations made with the tool, Figure 7 presents starting point A and ending point B on the sheet of graph paper. These points are used in the calculations performed by the tool. Thus, the $\overrightarrow{AB}$ vector represents the distance covered by the laser point on the target during one movement of the patient (to the left or to the right). From the analysis of the model, Equation (1) is introduced, as follows:

b² = (x_B − x_A)² + (y_B − y_A)²

(1)

where b = distance (cm), x_A = starting point, x_B = end point, y_A = starting point, y_B = end point.

The length b indicates the distance between the readings taken when the subject’s head moves, capturing the initial and the final point after the movement. As the points can be captured at different positions, for each of the measurements taken, the distance between the first and second points is calculated (x_A and y_A indicate the reading of the first point, and x_B and y_B indicate the reading of the second point) to enable analyses to be performed with the CRT.

With the defined value of the hypotenuse (h value), one can calculate the sine of the angle of displacement (sin(α) = (b)/(h)) and then the angle (Figure 8).

After the readings have been finished, the “Report” button can be used to generate a report on the screen with graphs and tables showing information vital to the specialist. The software performs the calculations and displays the values on a screen. The collected data are presented with an indication, for each test, of the distance between the movements and the read points, the variation on the x and y axes, the value of the sine, and the angle of these points in relation to the nose.

4. Results

The

Table 1. data collected from the CRT test for 10 movements of the patient’s head.

Movement Number	Point Number	Points Values		b Value	c Value (cm)	b Value (cm)	h Value	Tan (Alpha) in Degrees
		94X	94Y
1	$(x_A,{y_A)}_1$	0.5149	0.6739	0.00450018	80	4.50017777	80.1265	3.22641877
	$(x_B,{y_B)}_1$	0.5104	0.6739
2	$(x_A,{y_A)}_2$	0.5144	0.6742	0.00440454	80	4.40454311	80.1212	3.15771289
	$(x_B,{y_B)}_2$	0.51	0.674
3	$(x_A,{y_A)}_3$	0.5147	0.6738	0.00450587	80	4.50587394	80.1268	3.2305113
	$(x_B,{y_B)}_3$	0.5102	0.6736
4	$(x_A,{y_A)}_4$	0.5139	0.6745	0.004219	80	4.21900462	80.1112	3.02444391
	$(x_B,{y_B)}_4$	0.5097	0.6749
5	$(x_A,{y_A)}_5$	0.5128	0.6742	0.00460109	80	4.60108683	80.1322	3.29892391
	$(x_B,{y_B)}_5$	0.5082	0.6743
6	$(x_A,{y_A)}_6$	0.5123	0.6743	0.00444072	80	4.44072066	80.1232	3.1837025
	$(x_B,{y_B)}_6$	0.5079	0.6749
7	$(x_A,{y_A)}_7$	0.513	0.6746	0.00503587	80	5.03587132	80.1583	3.61144855
	$(x_B,{y_B)}_7$	0.508	0.6752
8	$(x_A,{y_A)}_8$	0.5113	0.674	0.00465601	80	4.65600687	80.1354	3.33838946
	$(x_B,{y_B)}_8$	0.5067	0.6747
9	$(x_A,{y_A)}_9$	0.5102	0.674	0.0043195	80	4.31950229	80.1165	3.09662544
	$(x_B,{y_B)}_9$	0.5059	0.6744
10	$(x_A,{y_A)}_{10}$	0.5098	0.674	0.00433133	80	4.33132774	80.1172	3.10511961
	$(x_B,{y_B)}_{10}$	0.5055	0.6745

shows the values of each movement performed. The values refer to point 94 identified by MediaPipe (Figure 4) and correspond to the position of the subject’s nose in relation to the laptop camera.

Figure 9 and Figure 10 indicate the variation in the readings on the x and y axes for each reading obtained according to the movement of the face.

Using these graphs, it is easy to show the specialist the deviation in the replacement head position of the patient. Based on these data, he is able to immediately propose a diagnosis of cervical proprioceptive dysfunction. In this situation, one can observe a more or less consistent shift of the patient’s head to the left (decrease in the x value in Figure 9) and a deviation of the patient’s head upward (increase in the y value in Figure 10).

Figure 11 shows the final results presented to the specialist. The image is a digital representation of the graph paper indicating the result of the CRT applied to the patient. It is the equivalent of the square sheet of paper used in a standard CRT. On the target, a curve composed of 10 points can be observed. These points are the one represented in the Table 1, coordinates $(x_A,{y_A)}_1$ to $(x_A,{y_A)}_{10}$.

This representation provides access to measurements of the displacements that occur during 20 head movements. To allow an easier understanding and absolute measurement of deviation, the proposition was to define the first point of the test at coordinates $\left(x_A,y_A\right)=\left(\mathrm{0,0}\right)$. All other coordinate values are positioned relative to these.

From this stage of the program, it is possible to store the obtained points in a file using the “Export” button. If the specialist has identified a problem during data collection, it is possible to restart the process using the “Reset” button. If the reading information is satisfactory, a spreadsheet (.csv file) can be generated with the data obtained.

To generate the file name, information about the individual (name, age, and sex) to whom the test was applied is requested. These data and a code for the meaning of the reading performed make up the file name in the form name-age-sex-codxx.csv. The user can also enter more information that is necessary to identify the reading and the individual. With the stored data, it is possible to search the file names and review the measurements taken when necessary for a new analysis or comparison.

It is essential to evaluate the quality of measurement of our system compared to the traditional CRT method. Figure 12 shows the results of an experiment wherein ten measures were compared using a 5 cm reference for the traditional CRT method and our system at the same time. The proposed solution systematically yielded a distance less than 5 cm. A mean difference of 10% between the two methods was calculated.

5. Discussion

These initial results are very encouraging, but there is still room for improvement.

Comparative tests between the different types of camera can be carried out in greater depth to assess the advantages and disadvantages of each. As the specialists are very keen on simplicity and ease of understanding with regard to the equipment made available for their patient assessments, the use of a laptop camera was highly favored. Nevertheless, it could have been interesting to analyze how different camera qualities on laptops impact the accuracy of the system.

Extensive tests were carried out to verify the impact of the color of the backdrop behind the patient. No influence of color was detected. No tests have yet been carried out to verify the impact of brightness or that of patients with peculiar facial features. Such tests can be done in future studies

According to the data obtained, it was verified that the method can identify the position of the face between the initial and final movements in the same way this can be done when a CRT is applied to a patient in a controlled environment.

The manner in which these data should be analyzed will be defined by the expert. However, as seen, the tool facilitates both the application of the CRT and the formalization of the captured data and then stores the data in a simple location on the computer, facilitating the work of physical therapists, which was one of the project’s goals. For the CRT procedure, the installation of the laptop is very easy and the measurements can be carried out in the practitioner’s office. The practitioner will need only one measurement tool to properly position the patient at a good distance from the camera, or, better, a tool for measuring the distance between the camera and the patient can be configured on our system. Thus, a very easy-to-understand visual indicator can be used by the practitioner and the patient. With their eyes closed or while wearing a blindfold, the patient will be able to make head movements, sitting on a chair, listening to the instructions given by the healthcare professional or issued automatically by our system.

Point 94 of the MediaPipe Face Mesh reference points is used for this research. In future tests using the present system, the researchers plan to use at least three points to evaluate the quality of the proposal.

6. Conclusions

This article presents a tool to help physical therapists work with people who experience chronic neck pain due to problems with cervical proprioceptive dysfunction and describes how the tool was designed and developed. This paper describes how the face positioning is captured and which calculations are implemented to interpret the data captured by the tool. The research and development carried out and briefly presented here were fundamental to developing the studies that determined the sensor and framework best suited to the tool.

The tools currently used for the CRT are not very expensive. The improvements proposed, compared to the current system, are better not only in terms of simplicity of implementation (no helmet, no more targets on the wall, no laser, no more marking with a pen, no need for a dedicated room), but also and above all improve post-processing of the information collected. The innovative tool allows the practitioner to immediately obtain the data needed to make his diagnosis.

It is also essential to continue research using Google MediaPipe, for example, as an analysis tool for measuring human joint angles. Research in this direction could facilitate the creation of new tools for analyzing parts of the human body.

In addition to using the tool in CRTs, another possible application is aiding people who need physical therapy, for example, after a hand or arm surgery. A new version of the tool could help determine joint angles and thus be used to identify the presence or absence of dysfunctions and quantify joint-angle limitations.

In [28], the aim of the study was to investigate the effect of supra-threshold electrotherapy on pain level, subjective feeling of disability, and spinal mobility in patients with chronic pain in the spinal cord. In that study, 11 men and 24 women with a mean age of 49 years were randomly divided into three groups. Cervical and lumbar range of motion (ROM), as well as disability in daily life, were investigated for all the patients before and after the electrotherapy sessions. Only questionnaires were used for evaluation after the supra-threshold electrotherapy sessions. With such an intuitive and effective tool to evaluate cervical and lumbar range of motion, researchers would be able to add quantitative data to the study.