Evaluating a Kinematic Data Glove with Pressure Sensors to Automatically Differentiate Free Motion from Product Manipulation

Abstract

When studying hand kinematics, it is key to differentiate between free motion and manipulation. This differentiation can be achieved using pressure sensors or through visual analysis in the absence of sensors. Certain data gloves, such as the CyberGlove II, allow recording hand kinematics with good accuracy when properly calibrated. Other gloves, such as the Virtual Motion Glove 30 (VMG30), are also equipped with pressure sensors to detect object contact. The aim of this study is to perform a technical validation to evaluate the feasibility of using virtual reality gloves with pressure sensors such as the VMG30 for hand kinematics characterization during product manipulation, testing its accuracy for motion recording when compared with CyberGlove as well as its ability to differentiate between free motion and manipulation using its pressure sensors in comparison to visual analysis. Firstly, both data gloves were calibrated using a specific protocol developed by the research group. Then, the active ranges of motion of 16 hand joints angles were recorded in three participants using both gloves and compared using repeated measures ANOVAs. The detection capability of pressure sensors was compared to visual analysis in two participants while performing six tasks involving product manipulation. The results revealed that kinematic data recordings from the VMG30 were less accurate than those from the CyberGlove. Furthermore, the pressure sensors did not provide additional precision with respect to the visual analysis technique. In fact, several pressure sensors were rarely activated, and the distribution of pressure sensors within the glove was questioned. Current available gloves such as the VMG30 would require design improvements to fit the requirements for kinematics characterization during product manipulation. The pressure sensors should have higher sensitivity, the pressure sensor’s location should comprise the palm, glove fit should be improved, and its overall stiffness should be reduced.

1. Introduction

Electromechanical data gloves are among the most commonly used systems for recording hand kinematics. These gloves are equipped with strain gauges, which are placed over the joints under study. The signal provided by each gauge, measured in mV, varies depending on the degree of flexion. Electromechanical data gloves are the best choice for recordings involving manipulation, as they do not present problems such as the occlusions that can occur in optical systems or data interferences caused by the proximity of ferromagnetic objects, unlike electromagnetic systems. Furthermore, their setup is significantly easier compared to other systems. Optical systems with markers require initial calibration for each session and marker location, and electromagnetic systems such as Polhemus Fastrak or Polhemus Viper require sensor location in specific hand segments. Data gloves only require an initial calibration process when acquired [1]. CyberGlove (CyberGlove Systems, San Jose, CA, USA) is one of the most used data gloves in biomechanics to record hand kinematics [2,3,4,5,6,7]. It has the capability to record either 18 or 22 degrees of freedom (DoFs), depending on the glove model. Its main body is made of elastic fabric, with a thicker section on the back of the hand and a thinner section on the palm contributing to achieve a better fit. To prevent interfering in manipulation, all the wiring and strain gauges are positioned on the back of the hand, and each gauge’s position is fixed with seams. This glove has a thin and well distributed wiring facilitated by the slim design of the strain gauge sensors.

When analyzing human hand kinematics during activities involving product use, it is key to differentiate between reaching, manipulation, and release phases. However, in recordings using the CyberGlove, a visual analysis is required to distinguish these different phases. This visual analysis involves recording the time stamp of specific events using the data acquisition software. Time stamps allow the segmentation of recordings in several parts at convenience (e.g., in reach, grasp and release phases) or the identification of specific events (e.g., the performance of a specific grasp). Nevertheless, acquiring time stamps though visual analysis requires a person observing all the performed tasks, which also introduces an error associated with their reaction time. An alternative to this visual analysis could be automating the procedure by incorporating pressure sensors to all the products (which may be expensive and time-consuming to install) or using a glove that includes pressure sensors.

There are various commercially available data gloves that allow hand posture recording in addition to CyberGlove, such as Rokoko (Rokoko, København, Denmark), Xsens Manus Prime II (Xsens Technologies, Enschede, The Netherlands), or MoCap Pro Gloves (Stretchsense, Auckland, New Zealand), among others. Nevertheless, as observed in recent review studies investigating available commercial data gloves [8] and robotic gloves for remote control [9], there are few commercial gloves that record hand posture and are also equipped with pressure sensors. This low availability of commercial kinematic data gloves with pressure sensors has boosted the development of custom gloves by research groups.

Table 1. Kinematic data gloves with pressure sensors, along with their sensor characteristics and commercial availability. Joints: IP1 for thumb interphalangeal joint, PIP for proximal interphalangeal joints of fingers, MCP for metacarpophalangeal joints, CMC1 for carpometacarpal joint of thumb, CMC5 for palmar arch resulting from flexion/extension of carpometacarpal joints of ring and little fingers. Nomenclature: _F for flexion, _A for abduction; 1 to 5, digits from thumb to little.

Glove Model	Manufacturer/ Reference	Kinematic DoFs	Pressure Sensors	Commercially Available
CaptoGlove	CaptoGlove LLC, Shalimar, FL, USA	10 (IP1_F, MCP1_F, PIP2-5_F, DIP2-5)	1 (thumb fingertip)	Yes
VMG30	Virtual Motion Labs, Dallas, TX, USA	16 (CMC1_F, CMC1_A, IP1_F, PIP2-5_F, MCP1-5_F, MCP2-4_A, CMC5_F)	5 (fingertips)	Yes
VMG35	Virtual Motion Labs	16 (CMC1_F, CMC1_A, IP1_F, PIP2-5_F, MCP1-5_F, MCP2-4_A, CMC5_F)	5 (fingertips)	Yes
SenGlove	[10]	15 (CMC1_F, CMC1_A, IP1_F, PIP2-5_F, MCP1-5_F, MCP2-4_A, CMC5_F)	5 (fingertips)	No
Smart Glove	[11]	5 (IP1_F, PIP2-5_F)	5 (fingertips)	No
Smart Glove	[12]	1 (IP1_F)	1 (thumb fingertip)	No
Smart Glove	[13]	5 (IP1_F, PIP2-5_F)	1 (palm)	No
Smart Glove	[14]	20 (CMC1_F, CMC1_A, IP1_F, PIP2-5_F, DIP2-5_F, MCP1-5_F, MCP2-4_A, CMC5_F)	10 (4 on fingertips, 6 on index and little phalanxes and metacarpals, none on thumb)	No

presents a summary of kinematic data gloves with pressure sensors, both commercially available and custom-made, detailing the number of kinematic DoFs, number of pressure sensors, and commercial availability.

One of the commercially available gloves listed in Table 1 that presented the best characteristics regarding the number of DoFs for kinematics recording and the number of pressure sensors was VMG30, which was produced by Virtual Motion Labs. For this reason, a VMG30 glove (Figure 1) was acquired by our research group but customized with five extra pressure sensors on the palm and the proximal phalanx of the middle finger. The customization of the glove and the location of the five extra sensors was motivated by the results obtained in previous studies [15], where the study of hand contact pressure during 21 activities of daily living revealed high implication of specific palm spots. In fact, one of the main purposes of customizing the VMG30 glove was initially analyzing hand kinematics and pressure distribution in a combined way.

Figure 1 shows the location of all the sensors, with the additional ones (s6 to s10) marked in red. The main body of this glove is composed of elastic fabric, but the fit is not as optimal as that of the CyberGlove due to the presence of bulky embedded wiring and the glove tailoring (see Figure 2). The glove is fixed to the wrist with a Velcro strap, and the fingertips are covered. In this glove, the wiring of the gauges is positioned on the backside of the hand, while the pressure sensors and their wiring are located in the palm and fingers.

The aim of this communication is to evaluate the feasibility of using virtual reality gloves with pressure sensors, such as the VMG30, for hand kinematics characterization during product manipulation, testing its accuracy for motion recording when compared with CyberGlove as well as its ability to differentiate between free motion and manipulation using its pressure sensors in comparison to visual analysis. The main advantages of validating the VMG30 glove would be: (1) automatic distinction between free motion and manipulation without the human error associated with visual analysis, (2) the possibility of using a more affordable commercially available data glove, and (3) the option of recording pressure applied to specific hand spots during grasps.

2. Materials and Methods

Experiments were performed in two different phases that consisted of testing the kinematic accuracy of VMG30 recordings (Phase I) and studying VMG30 detection sensitivity when contacting objects (Phase II) while performing a set of tasks. VMG30 and CyberGlove II (both right gloves with 16 DoFs and 18 DoFs, respectively) were used to acquire data during the experiment.

Both gloves recorded exactly the same 16 DoFs, according to the manufacturers, but CyberGlove also recorded flexion and deviation of the wrist. For this reason, this 16 DoFs present in both gloves were calibrated using the same validated across-subject calibration protocol [1] developed by our research group. This calibration protocol only has to be performed when acquiring the glove, contrarily to the calibration protocol recommended by the manufacturers, which has to be performed for each participant recorded. This protocol consisted of 44 recordings: static postures controlled with wood accessories (to obtain gauge gains) and controlled movements (to apply kinematic cross coupling corrections). Both gloves were calibrated by recruiting six participants with hand sizes between 160 mm and 200 mm, selected to achieve a sample representative of the average adult hand length. Participants were not the same for both gloves; therefore, a total of 12 participants took part in the calibration experiments, with a mean hand length of 177.2 (±12.3) mm for CyberGlove trials and 179.8 (±15.3) mm for VMG30 trials. The calibration procedure was performed three times for each participant, and the mean gains and coefficients from the calibration were defined as the mean values obtained across all the participants and trials for each glove.

All the experiments were approved by the Research Ethics Committee with Human Beings (formerly Deontology Committee) of Universitat Jaume I (Spain), reference number CD/31/2019. All the participants were previously informed about the characteristics of the experiment and gave their written consent to participate.

2.1. Phase I

Three participants with different hand sizes and ages (Participant 1: 174 mm, 49 years; Participant 2: 196 mm, 48 years; Participant 3: 176 mm, 44 years) volunteered to participate in Phase I. Since the aim of Phase I was to compare both glove kinematic recordings in extreme postures but not extract normative kinematic data, a larger sample of participants was considered unnecessary. In this phase, in order to assess the kinematic accuracy of VMG30, the active ranges of motion (AROMs) of 16 hand joints angles were recorded with both gloves, recording a total of 27 AROMs per participant, corresponding to maximum and minimum values of the 16 hand joint angles (except carpometacarpal joint flexion/extension and finger abductions/adductions, where only maximum flexion and maximum abduction values were considered, see Figure 1 and

Table 2. Active range of motion assessed in each recording presented in Figure 3. Joints: IP1 for thumb interphalangeal joint, PIP for proximal interphalangeal joints of fingers, MCP for metacarpophalangeal joints, CMC1 for carpometacarpal joint of thumb, CMC5 for palmar arch resulting from flexion/extension of carpometacarpal joints of ring and little fingers. Nomenclature: _F for flexion, _E for extension, _A for abduction; 1 to 5, digits from thumb to little.

Posture	Active Range of Motion Assessed
A	Max. thumb CMC flexion (CMC1_F)
B	Max. thumb CMC extension (CMC1_E)
C	Max. thumb CMC abduction (CMC1_A)
D	Max. thumb interphalangeal joint (IP1_F) and metacarpophalangeal joint flexion (MCP1_F)
D	Max. proximal interphalangeal joint flexion of index to little fingers (PIP2_F-PIP5_F)
E	Max. metacarpophalangeal joint flexion of index to little fingers (MCP2_F-MCP5_F)
F	Max. metacarpophalangeal joint and proximal interphalangeal joint extension of index to little fingers (PIP2_E-PIP5_E) (MCP2_E-MCP5_E)
G1	Max. abduction between index and middle fingers (MCP2-3A)
G2	Max. abduction between middle and ring fingers (MCP3-4_A)
G3	Max. abduction between ring and little fingers (MCP4-5_A)
H	Max. flexion of carpometacarpal joint (CMC5_F)

). To do so, participants were firstly asked to perform a reference posture with their flat hand lying on a table, with the fingers and thumb together. Joint angles for a given posture were computed as rotated angles from those of this reference posture. Then, participants were asked to perform ten static postures that forced participants to achieve the active extreme postures of groups of joints, therefore allowing the recording of their AROMs (Figure 3 and Table 2). After the AROMs recording, joint angles were computed using the calibration coefficients obtained for each glove, according to a previous study [1], and using a custom angle calculation code [16]. Then, the joint angles obtained from both gloves were filtered using a 2nd order two-way low pass Butterworth filter, with a cut-off frequency of 5 Hz. The AROM for each participant and joint was computed from these filtered angles as the maximum values obtained for each DoF across all recorded postures. Then, AROMs obtained for each joint and participant when using the VMG30 and CyberGlove were compared by performing 27 repeated measures ANOVAs (Sig. ≤ 0.01), one per recorded AROM.

2.2. Phase II

Participants 1 and 2 from Phase I volunteered to participate in Phase II. Since the aim of Phase II was solely to study sensor detection, a larger sample of participants was considered unnecessary. Instead, participants with different hand lengths (174 mm and 196 mm) were selected, as hand length could potentially impact pressure sensor activation. These hand sizes represented the 41st percentile for the US women adult population and the 53rd percentile for the US male adult population, according to normative data from PeopleSize 2008 Professional v1.10 software (OpenErgonomics Ltd., Wroxham, UK). In this phase, in order to assess the manipulation detection capabilities, six activities (A1 to A6, Figure 4) representative of several grasp types and involving different objects were performed by two participants while wearing the VMG30. Some tasks (A1, A2, A5 and A6) were inspired by the Sollerman Hand Function Test [17], a standardized hand function test.

Table 3. Activities performed in the experiment, grasp type required and description.

Task	Grasp	Description
A1	Oblique/Cylindrical	Grab the door handle, turning it and releasing it.
A2	Pulp pinch/Lateral pinch	Taking the key from the table, introducing it into the lock, turning it and releasing it, leaving the key inside the lock.
A3	Cylindrical	Taking a bottle of water from the table, moving it between two points drawn in the table and releasing it.
A4	Five-finger pinch/cylindrical	Taking a glass of water from the table, holding it for few seconds and leaving it again on the table.
A5	Tripod pinch	Taking a pen from the table, writing the word “BIOMECÁNICA” in a sheet of paper and leaving it again on the table.
A6	Intermediate	Taking a knife and a fork from the table, cutting play dough (simulating a sausage) and leaving them again on the table.

details all the tasks performed.

The person in charge of the experiments recorded the time stamp of the instants where grasp and release of the objects occurred in each activity (using the data acquisition software) as part of the visual analysis. After this, time stamps collected in visual analyses were compared to those from the activation of any of the pressure sensors. Pressure sensors raw data signal varied from 100 to 0, with a precision of 0.1, with 100 the value for non-contact. Therefore, sensor activation was considered when the raw data value was below 100.

3. Results and Discussion

Figure 5 presents a box and whiskers plot with the AROMs (in degrees) measured in Phase I of the experiment. The 27 performed repeated measures ANOVAs (one per recorded AROM) revealed statistically significant differences (Sig. ≤ 0.01) between gloves in 10 out of 27 AROMs.

Table 4. Mean values of AROMs, as well as reported values in literature [18]. Statistically significant differences (Sig. ≤ 0.01) after the repeated measures ANOVAs between the values obtained using CyberGlove and VMG30 are shaded in grey.

JOINT	CyberGlove AROM	VMG30 AROM	Values Reported in Literature [18]
IP1_F	82.88°	115.80°	102.1°
IP1_E	−32.54°	8.01°	−12.4°
MCP1_F	28.16°	47.65°	26.1°
MCP1_E	−24,13°	−19.39°	−21.0°
CMC1_F	30.14°	19.34°	42.10°
CMC1_E	−42.66°	−6.41°	−26.10°
PIP2_F	112.71°	88.52°	108.80°
PIP2_E	−1.69°	−0.02°	−3.80°
MCP2_F	75.26°	127.89°	70.60°
MCP2_E	−19.53°	−9.26°	−25.30°
PIP3_F	98.45°	86.32°	96.60°
PIP3_E	−6.32°	0.69°	−6.70°
MCP3_F	81.72°	103.88°	73.60°
MCP3_E	−32.61°	0.10°	−23.10°
PIP4_F	105.28°	105.84°	102.80°
PIP4_E	−8.08°	−1.48°	−9.90°
MCP4_F	75.11°	223.05°	68.40°
MCP4_E	−35.76°	−2.14°	−21.90°
PIP5_F	102.62°	99.58°	89.90°
PIP5_E	−7.16°	−2.66°	−7.80°
MCP5_F	69.38°	166.25°	68.40°
MCP5_E	−38.59°	−31.93°	−21.90°
CMC5_F	34.95°	12.93°	29.60°
CMC1_A	20.05°	23.91°	19.70°
MCP2-3_A	35.50°	34.71°	35.20°
MCP3-4_A	26.99°	19.48°	25.70°
MCP4-5_A	31.13°	−51.17°	28.4°

details the AROMs and the mean values for each data glove and values reported in the literature [18] for each specific AROMs in order to compare them.

The statistically significant differences obtained are high, except in MCP4_A, where the difference is only 7.51°, which is close to the glove sensitivity [1]. Most of the obtained AROM values using the VMG30 are far from the healthy hand AROMs, as observed in Table 4, and are present mainly in MCP flexion and extension. These differences may be attributable to the coefficients obtained from the calibration process in these joints, which may not be appropriate. We experienced several problems when placing the initial calibration accessories on the hand dorsum because of the wiring of the glove, which is bulky and not uniformly distributed, which might have affected the performance of the entire calibration. Figure 6 shows some accessories used to calibrate the data gloves, according to a previously validated protocol [1].

Figure 7 shows the differences between automatic detection (activation/deactivation of sensors) and visual analysis. Positive values mean delay of sensors regarding visual analysis, while negative values mean anticipation of sensors. The figure only provides difference values for the pressure sensors that were first activated in each task (to detect grasp) and the sensor that was last deactivated (to detect release). Sensors activated in between, and therefore not important to detect grasp or release, are shaded in grey. Sensors not activated during the task are not shaded (white cells). Differences lower than 0.5 s (absolute value) are shaded in green. Differences higher than 0.5 s (absolute value) are shaded in red.

Differences in visual analysis were higher than 1 s on three occasions: when opening a door (A1), grasping a pen (A5), and releasing a knife (A6), due to delay/anticipation (A5 and A6) or activation without any contact (A1). Differences were higher than 0.5 s on eight occasions. These differences are high, taking into account that the obtained response time for visual stimuli in the literature varies between 200 ms and 300 ms [19,20], and considering that in these experiments the observer is previewing the full task and can anticipate events, therefore reducing its reaction time.

Furthermore, some sensors were frequently activated (s1, s2, s3), while others were rarely or not activated at all (s4, s10) (Figure 7). The only tasks where grasp and release were correctly detected in both participants were using a key (A2) and taking a glass (A4).

The sensor most used to detect grasp was s1 (located at thumb fingertip) in both participants. The most used to detect release was also s1 for participant 1 (hand length = 174 mm), but it was s2 (located at index fingertip) for participant 2 (hand length = 196 mm). It may be attributable to a glove bad fit, which may not allow correctly locating fingertips below the pressure sensor location. It can also be observed that sensors s6 (thenar area of palm), s7 (index and middle interdigital area of palm), s8 (middle finger proximal phalanx), and s9 (ring and little interdigital area of palm) were mainly activated only in participants with larger hands. These results allow drawing important conclusions regarding the sensors’ location within the glove. On the one hand, they highlight the importance of thumb, index, and middle fingertips pressure sensors to distinguish free motion from manipulation. On the other hand, they reveal that the additional sensors s8 to s9 located in the palm also provide key information, except S10. Therefore, palm sensors should not be underestimated when designing data gloves with pressure sensors. In fact, palm is the part with the first and last contact with the objects when performing power grasps. This can be observed in sensor activation in tasks A1, A3, and A4 (all requiring power grasps), where additional palm sensors detected grasp or release. These results are consistent with previous studies analyzing hand pressure distribution in activities of daily living [15]. The only important difference regarding the results in the literature was the inactivity observed for pressure sensor s10. This palm spot was observed to present a higher activation of pressure sensors in [15] but this may be attributable to the pressure sensor size, as the used in previous studies was almost thrice the size of VMG30 sensors. The second spot observed to have higher activation in the aforementioned study was distal phalanx of index finger, which is consistent with the results here obtained using VMG30.

The sensor that presented the most differences in contact detection was s2 (located at index fingertip), implying an anticipation of deactivation in release detection in tasks A5 and A6, both implying performing grasp types where the index finger was key (tripod pinch and intermediate, respectively). To better illustrate the results, Figure 8 presents a fusion of raw kinematic data, raw pressure sensors data, and visual analysis data while participant 1 was performing task A6, which was the one that presented the most differences between pressure sensors detection and visual analysis. It can be observed that s1 deactivation occurs long before the stabilization of the kinematic signal.

Moreover, it can be observed that the most differences in reaching mainly implied a delay in sensors detection (positive values), while in release they implied an anticipation in their deactivation (negative values). This led us to suspect that pressure sensors sensitivity is not enough to automatically differentiate product manipulation from release in ADLs.

The contact detection clearly failed, and it can be attributable to different factors such as sensor sensitivity or bad fit of glove in small hands and the selected activities/products. In fact, participants reported some difficulties with the completion of some activities due the thickness and stiffness of the VMG30.

4. Conclusions

A data glove with pressure sensors would allow automatic differentiation between free motion and manipulation, avoiding human reaction-time errors. Nevertheless, owing to the results obtained in this experiment, currently available gloves such as the VMG30 would require design improvements to fit the requirements for kinematics characterization during product manipulation.

Firstly, pressure sensors included in these types of data gloves should have higher sensitivity for more accurate automatic detection of manipulation. This conclusion can be drawn after observing that the sensors did not offer additional precision regarding the visual analysis technique (which also implies human error). Moreover, the pressure sensor distribution within the glove should consider the palm, as this part is the first and last to be in contact with objects when performing power grasps, such as cylindrical or oblique, as observed in pressure sensor activation.

Another important weakness of the glove is the wiring on the dorsum of the hand, which should be modified in order to reduce glove stiffness and bulkiness. This conclusion is motivated by the concerns raised by participants during the experiments and the conviction that the bulkiness reduction would favor the placement of accessories during the initial calibration procedure if necessary. Nevertheless, it could be solved by developing a calibration protocol to control posture using an optical motion tracking system instead of wooden accessories. Apart from favoring glove calibration, less bulky wiring and reduced stiffness would contribute to more realistic hand motion. Bulkiness reduction could be accomplished by using a lighter wiring technology, such as textile circuit boards. Another alternative would be to reduce the number of pressure sensors on the glove.

Moreover, it was observed that sensor detection varied depending on hand size. It can be attributable to the small area covered by sensors (Ø = 20 mm) combined with a bad fit of the glove, which contributes to locating pressure sensors in areas of the hand where they are not intended to be. The covered area of pressure sensors could be extended by using other sensor technologies, such as textile pressure sensors, that cover larger areas. This improvement should be accompanied by an improved glove fit. Correct sizing is a key factor determining data accuracy, as studied in previous studies in the literature [21], and the importance of manufacturing different glove sizes is evident.

In general, improving glove fit and reducing glove stiffness would also have an effect on manipulation dexterity, as observed in previous studies [22]. A holistic analysis of glove usability, considering aspects such as overall flexibility, loss of manipulation dexterity, or glove weight, among others, needs to be addressed.

With the current characteristics of each type of glove and taking all the outcomes into account, a glove without pressure sensors, such as the CyberGlove, is still the best choice for experiments requiring product manipulation. Nevertheless, in experiments requiring precision in phase distinction, the best choice may be sensor products in order to avoid human error in detection and sensor error associated with a bad glove fit. In the same way, CyberGlove could also be improved by lightening its wiring and offering several glove sizes, especially in their versions measuring 22 DoFs, which include sensors to record distal interphalangeal joint kinematics and do not fit small/medium hand sizes [21].

The main limitation of this study is the low sample size in Phase I and Phase II, as they were intended to be a technical validation prior to the main research experiments using data gloves. Another limitation may be the human error associated with the visual analysis and detection technique.