Accuracy and Inter-Unit Reliability of Ultra-Wide-Band Tracking System in Indoor Exercise

Featured Application

The use of the present technology to monitor movement patterns has been shown to be accurate and reliable in indoor conditions.

Abstract

The purpose of this study was to assess the accuracy of positional data and the inter-unit reliability of an ultra-wide-band (UWB) tracking system. Four well-trained males performed five courses designed for the analysis of x- and y-coordinate accuracy analysis, specifically related to the positional distance variation between the UWB data and the fixed reference lines of a basketball court. This was achieved using geographic information system (GIS) mapping software that calculated, for each interval and participant, the distance from the main axis of displacement and from the opposite side of the court each 0.5 s (x and y coordinate). The accuracy of the results was satisfactory, with a mean absolute error of all estimations for the x-position of 5.2 ± 3.1 cm and for the y-position of 5.8 ± 2.3 cm. Regarding inter-unit reliability, the intra-class correlation coefficient (ICC) value was high for the x-coordinate (0.65) and very high for the y-coordinate (0.85). The main findings of the study were: (i) The accuracy of UWB tracking systems can be considered suitable for practical applications in sport analyses; (ii) position estimations are very precise and acceptable for tactical analyses; (iii) the error of the position estimations does not change significantly across different courses; and (iv) the use of different devices does not significantly affect the measurement error.

1. Introduction

The use of wearable devices for quantifying and monitoring exercise is a basic concern when designing exercise physiology and sport performance studies [1]. In contemporary competitive sports, positional tracking systems are currently used to perform time-motion and tactical analyses [1]. These different systems can be categorized according to the type of technology employed. These technologies include wearable radiofrequency systems, such as global navigation satellite systems (GNSS) [2,3], local positioning systems (LPS) [4,5], ultra-wide-band systems (UWB) [6,7] and optical tracking systems, such as multiple camera video-based computerized tracking [3,8]. Each class of system has specific advantages and disadvantages [3,9,10]. The accuracy of each system is one distinguishing characteristic. At present, tracking systems are mainly used for time–motion analyses (e.g., distance covered, running speed), which provide important information for the assessment of external loads [11]. For such applications, all systems have sufficient accuracy [4,12,13]. GNSS has been the most studied system. Overall, it has been proven to be a valid and reliable instrument for distance and speed profile measurements during typical team sports actions [14], although, in short actions of high intensity and with changes of direction, limitations of accuracy have been reported [14]. In addition, some additional considerations should be taken into account, such as the sampling frequency of the devices, the quality of the satellite connection or the horizontal dilution of precision (see more in Malone et al. [15]). UWB-based tracking systems have been proven to be accurate systems for distance and speed measurements [6,7,12]. Even so, a previous study established that GPS and UWB are similar (not significant difference between them) in accuracy and inter-unit reliability for time-motion analysis [12].

The majority of these studies have assessed the accuracy of distance covered and speed using predefined test courses and time gates as references [4,16,17]. A different method should be used to evaluate the accuracy of the position of players on the court or to estimate instantaneous velocities (real-time analysis). Only three studies have evaluated position data, and these studies used LPS-based technology with video analysis as a reference [5,18,19]. LPS and video analysis are costly and complex to use. In particular, the reference system is based on fixed cameras and can therefore only acquire data in restricted areas [20]. In this sense, no studies have been found that have evaluated the accuracy of a tracking system in a complete field of play, where instead they only exist in confined and controlled spaces [3,5,18,19,21]. The capture volume is dependent on the maximum number of cameras and the field of view of each camera. A high number of cameras results in significant practical difficulties regarding cost, portability, calibration, synchronization, labor and set-up [22].

For this reason, the present study developed an alternative accuracy assessment method that does not require any instrument other than a device with software including a GIS (geographic information system) application. Moreover, UWB technology is the ideal radio frequency-based candidate to provide positioning information in any environment (indoor and outdoor) [23], but especially for indoor conditions, where GPS cannot operate, and others radiofrequency systems have problems in accuracy (e.g., interference with the radio frequency signal) because of the different architecture and buildings used for different courts. These differences refer to the elements and their arrangement, such as distance from the court to the walls, marker position, etc. Therefore, the accuracy of the system can change, and each system should be evaluated in the court where it will be used. For this purpose, a different and easy low-cost assessment of position data accuracy was developed. The aim of this study is to assess the accuracy of position data and the inter-unit reliability of an UWB tracking system in indoor conditions.

2. Materials and Methods

2.1. Participants

Four healthy and well-trained males (age: 38.43 ± 5.45 years, mass: 76.34 ± 7.65 kg, height 1.85 ± 0.056 m) volunteered to participate in the current investigation. The participants did not present any physical limitations or musculoskeletal injuries that could affect testing. Subject height was measured using a stadiometer (SECA, Hamburg, Germany). Body mass was obtained using a scale (TANITA BC-601, Tokyo, Japan). The study was conducted according to the Declaration of Helsinki and was approved by the Bioethics Commission of the University of Murcia (ID: 2061/2018). Participants were informed of the risks and provided informed written consent.

2.2. Equipment

Eight WIMU PRO^TM inertial devices (RealTrack Systems, Almeria, Spain) were used in this research (Figure 1). Each device had its own internal microprocessor, 2 GB flash memory and a high-speed USB interface, to record, store and upload data. The devices were powered by an internal battery with 4 h of life, had a total weight of 70 g and dimensions of 81 × 45 × 16 mm. The devices were composed of different sensors (four accelerometers, a gyroscope, a magnetometer and a GNSS and UWB chipset, among others [24]). S PRO^TM software (RealTrack System, Almeria, Spain) was used to analyze and export the data of the x- and y-position coordinates.

This UWB system was designed to alleviate any satellite reference problems by using time-based positioning techniques, in which the signal propagates from the transmitter (antenna) to the receiver (device). The transmitter in an UWB system is a set of antennas (4, 6, 8 or more) that transmit the radiofrequency signal almost under the same principle as the GPS system [24], except in this case, this reference system is fixed and is not affected by environmental conditions [25]. The ultra-wide band system occupies a very large frequency band, at least 0.5 GHz, as opposed to more traditional radio communications which operate in much smaller frequency bands. On the other hand, since UWB is only allowed to transmit at very low power, its spectrum becomes very noise-like and it can coexist with other services without influencing them.

Thus, if all nodes (transmitters and receivers) have a common clock, the receiving node can determine the time of arrival (TOA) of the incoming signal and directly calculate its distance from the transmitter, thus multiplying the estimated TOA by the speed of light allows the drawing of a circle with the reference node at its center and a radius equal to the estimated range. By collecting at least three measurements (trilateration 2D) and intersecting the defined circles, it is possible to determine the position of the receiver node/target (Figure 2). If the target receives UWB signals from more than three references, in normal conditions, the accuracy of the position measurement increases [26].

The UWB system is composed of two sub-systems: (i) The reference system and (ii) the devices tracked (carried by the participants). The reference system is composed of antennas that are transmitters and receivers of the radio frequency signal (see Figure 3). The antennas (mainly the master antenna) perform the computerizing of the position of the devices that are in their area of performance, while the devices receive that calculation.

2.3. Procedures

The UWB reference system was installed on the field as follows (Figure 3A): (1) Six antennae with UWB technology were fixed 5 m from the perimeter line of the field, except for those located at the middle line of the field, which were fixed 7 m from the perimeter. This way, the antennae formed a hexagon for better signal emission and reception. All of the antennae were positioned at a height of 3 m and held by a tripod. (2) Once installed, they were switched on one-by-one, with the master antenna turned on last, and a process of autocalibration of the antennae was carried out for 5 min, where the master synchronized all antennas with a common clock. Each time position coordinates were calculated (18 Hz), the master sent a synchronization in time to the rest of antennas [27]. (3), After, the tracking devices were switched on and a process of recognition and automatic communication with the antennae was carried out for 1 min. Each participant was equipped with two lightweight (70 g) inertial devices, each measuring 81 × 45 × 16 mm. (4) Four devices were placed in the corners of the basketball court on a tripod with a height of 1.85 m (mean height of participants). Data obtained in the previous step were used to establish the perimeter of the court and project it in the SPRO software later. (5) Finally, two inertial devices were placed in a custom vest located on the back of the upper torso, fitted tightly to the body, as is typically used in games. In the custom vest, the devices were placed in parallel (with a separation of 2 cm) and at the same height.

Data acquisition in the current study was carried out at the “Sinan Erdem Dome” Sport Arena (Istanbul, Turkey), measuring 28 × 15 m. Five different types of displacement were realized six times for each participant (n = 120 trials performed and 8344 samples) on an official basketball court to represent different directions of movement and different distances, as shown in Figure 4: (b) Perimeter markings of court; (c) middle line of court; (d) exterior perimeter of the painted lines; (e) center circle and (f) 6.75 m line. The actual distances were measured with a trundle wheel (Mini Rolfix, BMI, Hersbruck, Germany). All the tasks started from a standing position. Each task was performed three times for each direction with an interval of at least 5 min of rest between trials. The participants made the different movements shown (Figure 4) according to two criteria: (i) To move only on the lines marked on the basketball court, and (ii) to carry out the movements reaching a speed of >15 km/h when the course allowed it. However, for the center circle (c) and 6.75 m line (e), it was not possible to reach 15 km/h due to the centrifugal force generated by the curvilinear trajectory. Before beginning the protocols, the athletes performed a standardized 5-min warm-up at aerobic intensity (RPE 5/10) and a 5-min protocol composed of a simulation of the different movements that were to be performed later. In addition, all of the designed courses were practiced during the warm-up. A 10-point Likert-type scale was employed, with 0 being the minimum effort and 10 the maximum of the scale. The warm-up period and the tests were monitored in real time by the S PRO^TM software to verify that the devices were performing correctly, and that the participants achieved the necessary speed in each trial. When the participants finished the protocols, they performed 5 min of recovery running.

2.4. Data Processing

Data was collected during the Adidas Next Generation Basketball Tournament, part of the 2016–2017 season, at the “Sinan Erdem Dome” Sport Arena (Istanbul, Turkey). To investigate the accuracy of the UWB system for monitoring players’ positions on the court, the data were transformed into raw position data (x and y coordinates) using software (S PRO, RealTrack Systems, Almeria, Spain). The reference system to compare the results was projected in the software using a GIS mapping application. To do this, the set of points obtained by step 4 in the procedure were downloaded into the software. The centroid of each set of points was calculated as the point of reference (the four corners) to project onto the template of the court (Figure 3B). The standard deviation of the centroids calculated in all sets was ± 2.2 cm and ± 2.9 cm for coordinate x and y, respectively.

The GIS mapping application allows representation of geometrical shapes, such as polygons or circles, with millimeter accuracy. In this way, the routes selected with their real measurements (measured by trundle wheel) were introduced on the template created before. And then, the x and y coordinate data of the UWB system were introduced and compared. The distance errors of each axis (coordinate) were reported. Of all the data entered, only those that corresponded to the execution of the routes were selected, according to registers obtained using ANT+ technology and photocells at the beginning and end of each test. The protocol to register the beginning and the end of the test was described in a previous study (see that study for more information) [28].

In routes (b), (c) and (d), a number was assigned to each edge of the projected rectangle, so in each test the software automatically calculated the distance of the participant’s position between the court line where the displacement occurred (y coordinate) and the opposite court line (x coordinate). In lanes (e) and (f), the centroid of the projected circle was assigned, so in each test the software automatically calculated the distance of the position of the participant from the indicated centroid (coordinate x). The calculation of the distance of the participant’s position according to the reference element was made every 0.5 s, obtaining a total of 8344 samples. The reference distances were as follows:

(b)

Perimeter markings of court. (1) Baseline: 28 m with respect to the opposite baseline (coordinate x) and 0 m with respect to the displacement baseline (coordinate y). (2) Lateral line: 0 m with respect to the displacement lateral line (coordinate x) and 15 m with respect to the opposite lateral line (coordinate y).

(c)

Middle line court. 14 m with respect to the baseline (coordinate x) and 0 m with respect to the center line (coordinate y).

(d)

Exterior perimeter of the painted lines. (1) Baseline/front-line of the paint: 5.8 m with respect to the baseline or the front line of the paint (coordinate x) and 0 m with respect to the displacement line of the paint (coordinate y). (2) Lateral line of the paint: 0 m with respect to the displacement line of the paint (coordinate x) and 4.9 m with respect to the opposite lateral line of the paint.

(e)

Centre circle. 1.8 m with respect to the centroid of the center circle (coordinate x).

(f)

6.75 m line. 6.75 m with respect to the centroid of the three-point line (coordinate x).

The estimation of error was considered as the difference in the distance between the UWB devices’ tracking data (x and y coordinate) and the reference distance described previously.

2.5. Statistical Analysis

The distances from the two axis coordinates to the reference line were automatically calculated and downloaded using the S PRO^TM specialized software. The accuracy of the position data was calculated as the mean differences and percentage of differences of the x- and y-position coordinates, with respect to their reference line on the course. A Mann–Whitney U test was performed to compare differences in the differently designed routes and between devices. The significance level was established at p < 0.05. Inter-unit reliability was determined using Hopkins’s reliability spreadsheet [29] to calculate the percentage typical error of measurement (% TEM) and the intra-class correlation coefficient (ICC) values. This involved comparing the reported values for the two units used in the present investigation. These tests assisted with understanding the degree of error and the amount of variation between the units. The magnitudes of the TEM included poor (>10%), moderate (5–10%) or good (<5%) (Hopkins, spreadsheet). The strength of the ICC scores ranged from trivial (0.0), small (0.1), moderate (0.3), large (0.5), very large (0.7), nearly perfect (0.9) and perfect (1.0) (Hopkins, spreadsheet).

3. Results

3.1. Accuracy

Table 1. Accuracy in mean differences (cm) and percentage of differences of x- and y-position coordinates.

Designed Travel	Device	Differences		Percentage of Differences
		X	Y	X	Y
Perimeter of court	1	5.8	6.2	0.39%	0.41%
	2	5.1	5.1	0.34%	0.34%
Center line of the court	1	7.4	7.2	0.53%	0.51%
	2	11	10.8	0.79%	0.77%
Perimeter of the paint	1	0.3	4.4	0.05%	0.76%
	2	3.3	3.6	0.57%	0.62%
6.75 m line	1	1.9	-	0.28%	-
	2	8.6	-	1.27%	-
Center circle	1	5	5.9	3.03%	3.58%
	2	4.1	3.9	2.48%	2.36%
Mean ± SD		5.2 ± 3.1	5.8 ± 2.3	0.97 ± 1	0.94 ± 1.14
LOA (L to U)		2.1 to 8.3	3.5 to 8.2	−0.03 to 1.98	−0.21 to 2.08

Note. LOA: Limits of agreement (L: Lower; U: Upper); SD: Standard deviation.

summarizes the mean differences of the position estimation error in different courses. The mean absolute error of all position estimations (n = 8344 samples) was 5.2 ± 3.1 cm for the x-position and 5.8 ± 2.3 cm for the y-position. This represents percentage of differences of 0.97% ± 1% for the x-coordinate and 0.94% ± 1.14% for the y-coordinate. The limits of agreement in the estimation of errors were established between 2.1 to 8.3 cm for the x-coordinate and 3.5 to 8.2 cm for the y-coordinate. No significant differences were found between the designed routes and between devices.

3.2. Inter-Unit Reliability

A large ICC for the x-coordinate (0.65), a very large ICC for the y-coordinate (0.88) and a good % TEM (2%) was reported for the error agreement between the two devices assessed (

Table 2. Inter-unit reliability of x- and y-position coordinates.

Axis	Unit 1 (Mean ± SD)	Unit 2 (Mean ± SD)	ICC	90% CI	% TEM
x	4.1 ± 2.9	6.4 ± 3.3	0.65	−0.15 to 0.94	2
y	5.9 ± 1.2	5.9 ± 3.4	0.88	0.26 to 0.99	2

Note. ICC: Intra-class correlation coefficient; CI: confidence interval; TEM: Typical error of measurement; SD: Standard deviation.

). The 90% confidence interval (CI) was reported as −0.15 to 0.94 for the x-coordinate and 0.26 to 0.99 for the y-coordinate.

4. Discussion

To our knowledge, this study was the first to investigate the accuracy and the inter-unit reliability of an UWB system for monitoring the positions of players in indoor conditions. The results confirmed that this UWB model (WIMU PRO^TM) was a suitable system to monitor position data with high accuracy (x = 5.2 ± 3.1 cm; y = 5.8 ± 2.3 cm) and good inter-unit reliability, with a large ICC for the x-coordinate (0.65), a very large ICC for the y-coordinate (0.85) and a good % TEM (2%).

There is limited comparability of the accuracy and reliability with studies on other sport-related positioning techniques. Nevertheless, the results of average speed and the distance covered measurements reported [2,17,30,31,32,33] allow the conclusion that this UWB model outperforms GNSS results. LPS reported better accuracy results than GNSS [5], with a mean absolute error of 23.4 ± 20.7 cm, which is nonetheless outperformed by the present results. Considering that position measurement systems are mainly used for time-motion analyses in sport [1], this result seems acceptable [5]. It has been discussed that for tactical analyses, the estimation error of position sensing should be below the natural sway of the body´s center of gravity (15 to 20 cm) in the observed movements [1]. Therefore, the accuracy results in this study seem to be sufficient for such applications.

Most sport applications of location systems focus on the analyses of time-motion parameters, such as covered distances or speeds [1]. The validation methods used in earlier studies, using standard courses and time gates, are sufficient for such analyses [17,19,30,34]. Yet, other implications of position measurement systems (e.g., tactical analysis) require more accurate data. As a consequence, other studies have applied video analysis systems to satisfy these requirements for accuracy [5,21,31,35,36]. However, these video analysis systems are expensive and time consuming. Moreover, given that each court venue should be evaluated for accuracy, these methods are not appropriate. For this main reason, an extended validation method seemed to be necessary. The accuracy validation method applied in this paper tries to satisfy these requirements. Although reducing the ecological/internal validity, the five courses used in this study replicate the typical movements of a team sport: (i) Linear courses, (ii) curvilinear movements, (iii) changes of direction, (iv) accelerations and decelerations and (v) high speeds (>15 km/h). The considerations used in the method of this study were based on basketball requirements (e.g., to carry out movements reaching a >15 km/h speed, considering that the maximum speed in basketball usually does not exceed 20 km/h). In conclusion, the error of the position estimations does not change significantly among the different courses tested, and the use of different devices does not significantly affect the measurement error, due to their good inter-unit reliability. Hence, the accuracy of the UWB system (WIMU PRO^TM) can be considered suitable for practical applications in exercise and sport performance analyses, resulting in position estimates that are precise and acceptable.

5. Study Limits

In the current study, raw positional data were examined. Nevertheless, not all systems provide unfiltered raw positional data for analysis. The current study reports insights into the raw positional data and the error in the acquisition technology, without the possible influence of the manufacturer’s software. Thus, it could be used as a more stable measure of accuracy than software-derived metrics.

On the other hand, eight devices were used simultaneously in the experimental protocols. The effect of using more devices using the same bandwidth could affect the results, although a previous study did not report any problems in UWB-based tracking system accuracy with 28 devices turned on [12].

The criterion method performed in the present study aims to alleviate some problems of the use of the gold standard method described previously. The main limitation was to ensure as much as possible that the participants moved along the marked lines. For this, we carried out a pilot study and a familiarization session with the experimental protocols to ensure that the execution of the displacement of the participants would take place along the marked lines.

6. Conclusions

The accuracy of an UWB system’s output is highly sensitive to relative positioning on the field of play. Measures of position from the present model can be used confidently in time-motion analyses and tactical analyses for indoor team sports. Although the use of the same device by each player is usually recommended, the high inter-unit reliability reported in the present study shows that it is not strictly necessary. Future studies should explore the effect of the relative influence of the characteristics of the court and the effect of antenna disposition/distribution on the accuracy of the system.