Viability of 2D Swimming Kinematical Analysis Using a Single Moving Camera

Featured Application

The proposed moving camera method allows for 2D kinematical swimming analysis with no field of view limitation and with better accuracy and precision than those using a single static camera.

Abstract

Despite its limitations, 2D kinematical analysis remains a simple and valid alternative for swimming analysis. This analysis is limited by the length of the field of view, and the reconstruction errors are proportional to the calibrated area. A possible solution for these problems is the usage of moving cameras that allow for tracking an object across a larger area without the concerns of the calibration area. The purpose of this study was to verify the viability of the utilization of moving cameras for underwater 2D kinematical analysis. Two calibration processes were evaluated: (i) obtaining the extrinsic parameters for every frame based on pool markers (M1) and; (ii) constraining the degrees of freedom of the camera’s movements and tracking translation based on the principal point (M2). M1 obtained better accuracy in reconstructing the coordinates of static markers (mean error: 12.1 against 14.2 mm from a static camera) and in the estimation of the rod length (−2.6 vs. 12.6 mm). M2 obtained better accuracy when evaluating the distance between the points (−0.3 mm) than that for position estimation (58.6 mm). The results indicate that moving cameras are a viable alternative for 2D underwater kinematic analysis, but M2 had limitations about real position extraction.

1. Introduction

Cameras have been used for swimming kinematic analysis for many years, since a single camera can obtain the bi-dimensional coordinates of a point on a calibrated plane in its field of view. In this way, a group of selected points of interest for a swimmer can be used to describe movement. However, none of the swimmer’s movements are truly planar, generating uncertainty in the bi-dimensional data [1,2]. Besides this limitation, 2D kinematics remains a viable alternative for some parameters due to its lower cost and easy implementation and analysis when compared with 3D methods [3,4].

Another problem with the use of a single camera is the length of the field of view. Front crawl swimmers usually have a stroke length longer than 1.5 m [5], being necessary a large field of view to analyze consecutive stroke cycles [6,7,8]. The further the swimmers are away from the camera, the longer the field of view is in the swimming plane, but it also reduces sensitivity and makes the digitalization process harder. Moreover, larger calibrated volumes present larger errors when compared with those of smaller volumes [9,10], and extrapolations beyond the calibrated volume also increase the reconstruction error as a function of distance from the initial frame position [11,12]. These limitations make the kinematical analysis of an entire swimming trial (25 or 50 m) almost impracticable with a single static camera.

The usage of a single moving camera could solve this problem, since it is possible to use a relatively small camera field of view with a small calibrated area that follows the swimmer’s displacement, allowing for the analysis of an entire swimming trial. Only one study evaluated 2D kinematics variables using moving cameras, and their calibration process was limited to using a rope with marks at every 100 cm to scale the images [13]. Moreover, the accuracy, sensitivity, and resolution were not reported.

The challenge of using a single moving camera is knowing the extrinsic parameters (i.e., position and orientation) at any time in the pool global reference, in addition to the intrinsic parameters. The aim of the current study was to verify the applicability of a single moving camera for underwater kinematical analysis by comparing its accuracy and precision with a static camera. One traditional extrinsic calibration method (where the extrinsic matrix was obtained based on the 3D real coordinates of a set of point in the camera field of view) and a method based on the real coordinates of the principal point (when the camera movement degrees of freedom are constrained) were tested.

2. Materials and Methods

2.1. Experimental Setup

Two GoPro Hero 6 cameras were used. A trolley car was positioned on a rail mounted at the side of a swimming pool (25 m), and one camera was fixed at the end of the arm of the trolley, positioned 12 cm below the water surface. The second camera was fixed on the right wall of the swimming pool 50 mm deeper than the moving camera, in such a way that, when the trolley was aligned with the static camera, the position of the cameras differed only by 5 cm in the vertical direction. Thirteen vertical rods were positioned in the swimming pool 2000 mm apart between the 2nd and 3rd lane markings. Each rod had two markers 1000 mm apart, and the highest one was 500 mm below the water surface. All the trials occurred above the second lane markings; the first lane rope was removed from the pool to not cover the camera’s field of view (Figure 1).

2.2. Calibration Process

The intrinsic and extrinsic parameters of each camera were calculated using the Matlab (2016a, The MathWorks Inc., Natick, MA, USA) camera calibration toolbox [14,15]. A 1500 × 1000 mm chessboard (15 × 10 100 mm squares) was recorded underwater in different positions using both cameras at 120 fps with a 1920 × 1080 resolution, wide window. The focal length, the principal point coordinates, the pixel skew, three radial distortion coefficients (2nd, 4th, and 6th orders), and two tangential distortion coefficients were estimated based on 81 frames extracted from the footages recorded with each camera [16]. Afterward, the extrinsic parameters were obtained simultaneously for each camera using a rectangular rigid calibration body with 6 marks (1500 × 1000 mm). Markers were placed horizontally at 0, 750, and 1500 mm and vertically at 0 and 1000 mm. The calibration body was positioned above the second lane, with the upper edge on the water surface, and its center was aligned with a 12 m marker in the pool. In this procedure, the trolley was aligned with the static camera.

These procedures conclude the calibration of the static camera, but represents only the first two steps in moving camera calibration. For the first method applied for moving camera calibration, the previous extrinsic parameter calculated simultaneously with the static camera was not used. Instead, the extrinsic parameters (camera position and orientation, 6 DOF) were calculated frame by frame based on the pool reference frame determined by the pool markers (Figure 1) in each one of the experimental protocols (M1).

The second method applied for the moving camera calibration is based on the camera principal point (M2). The principal point is the point on the image plane onto which the perspective center is projected, and in this way, it is the point that provides the best symmetry of distortions [17]. If the camera rotations and translations in the direction perpendicular to the image plane were constrained (using a trolley rail for instance), the translation of the real coordinates of the principal point maps camera translation. With this information, it is possible to translate the camera coordinates obtained by first using extrinsic parameter estimation (using the 6 marks rectangular rigid calibration body) to obtain the pool coordinates. In this method, the more reference points there are available, the better the results, since the camera position is only known when the principal point is aligned with the real marker., while in the other frames, the camera position is interpolated. In the pool where the test was performed, the separation between the tiles was perfectly visible to the camera, which provided us with reference points every 190 mm.

2.3. Experimental Protocols

To evaluate the methods, three experimental protocols were performed: (I) recording a set of static markers in known positions; (II) recording a moving rod with known dimensions and; (III) a front crawl swimming trial. The first two protocols provided information about the method’s accuracy and precision, while the third one allowed for comparing the methods in a more realistic situation.

First, 15 static markers were positioned in the calibrated plane on pre-defined real coordinates, as presented in Figure 2. This trial was also simultaneously recorded using both cameras; the moving camera started the video at the position of approximately −2500 mm, and finished at approximately 2500 mm. The reconstruction errors of each camera were evaluated by the Euclidean distance from the real coordinates. As mentioned before, extrapolations beyond the calibrated area also increase the reconstruction error; therefore, the data from the moving camera were evaluated, considering the full field of view (F_FOV), and also subdivided into the coordinates evaluated at the horizontal center of the field of view (C_FOV) and the coordinates evaluated out of this zone based on a ±500 mm threshold from the center of the frame. Correlation between the estimated errors and the absolute horizontal pixel coordinate of the point were also evaluated. The F_FOV data from the moving camera were also compared with those the obtained using the static camera.

In the second protocol, one evaluator moves a 600 mm rod across the calibrated plane at 0.462 ± 0.048 m/s, while rotating it in the calibrated plane within the range of ±30°; the center of the rod was kept about 500 mm under the surface in such way that the rod was always completely underwater. This trial was simultaneously recorded by both cameras, and the trolley operator was instructed to try keeping the camera and rod always aligned. The video was analyzed when the rod was in the space within ±1500 mm, the edges of the rod were digitized, and its coordinates at each frame as well as the evaluated rod length in each frame for each camera were compared. In this protocol, the effect of the rod position in the static camera field of view was evaluated by subdividing the coordinates evaluated at the full field of view (F_FOV), the horizontal center of the field of view (C_FOV), and coordinates evaluated out of this zone. The correlation between the estimated length errors and the absolute horizontal pixel coordinate of the point was also evaluated. The data from the moving camera were also compared with the F_FOV data from the static camera.

The third protocol consisted of a 25 m front crawl swim. The trial was simultaneously recorded by both cameras, and the trolley operator was instructed to try keeping the camera and the swimmer’s hip always aligned. Markers were fixed on the right iliac crest (hip), the right wrist, and the right foot of the swimmer. These coordinates were extracted from each camera, and the iliac crest marker was also used to calculate the swimmer’s velocity on each frame. The most central stroke cycle was analyzed, and the difference between the vertical coordinates of the wrist and foot points and the swimmer’s hip position and velocity were calculated between the cameras. The 0.95 and 0.93 intraclass correlation coefficients (fixed-effect, 2-way ANOVA model) demonstrate the high reliability of coordinate digitalization for the static and moving cameras, respectively.

In summary, the first protocol evaluates the accuracy and precision of the models in measuring the absolute position of the markers in the swimming pool reference based on a set of static markers; the second protocol evaluates the accuracy and precision of the models in measuring the relative positions of two points based on the rod length; and the third protocol exemplifies the practical application of the methods.

2.4. Statistical Analysis

In all the conditions, the moving camera data were treated with both calibration methods based on the pool markers (M1) and based on the principal point (M2). The accuracy and precision parameters used in previously mentioned comparisons were the mean error, the standard deviation of the error, and the absolute mean error. Bland–Altman analyses of all the comparisons were also applied between the data of both the static and the moving camera [18].

3. Results

Table 1. Reconstruction errors (mm) in the static markers protocol and comparison with the static camera.

	Real Coordinate Comparisons			Static Camera Comparisons
	Static	M1 F_FOV (C_FOV)	M2 F_FOV (C_FOV)	M1 mm	M2 mm
Mean Error	14.2	17.4 (12.1)	25.7 (18.6)	3.2	11.4
SD Error	10.2	11.5 (8.5)	12.6 (11.0)	5.6	11.7
Mean Abs Error	-	-	-	4.9	11.8

shows the reconstruction errors of the 15 static markers. When only the points out of the center of the frame were considered, the reconstruction errors for M1 were 18.1 ± 11.9 mm and 34.6 ± 15.2 mm for M2. The correlation coefficient between the error and the absolute horizontal pixel coordinate was 0.70 for the static camera (p < 0.01), 0.66 for M1 (p < 0.01), and 0.79 for M2 (p < 0.01).

Table 2. Length results (mm) in the rod protocol and comparison with the static camera.

	Real Length Comparisons			Static Camera Comparisons
	Static F_FOV (C_FOV)	M1	M2	M1	M2
Mean Error	12.6 (−9.8)	−2.1	−0.3	−14.6	−12.5
SD Error	34 (7.4)	26.1	12.4	25.3	25.7
Mean Abs Error	29.3 (14.8)	19.9	10.3	23.7	22.5

shows the errors of the estimated rod length. The estimated length error was 24.9 ± 37.8 mm when the rod was out of the center. The correlation coefficient between the length error and the horizontal pixel coordinate was 0.77 (p < 0.01). Considering the rod position, the error between the estimate of M1 and the static camera was 29.4 ± 20.0 mm and 58.6 ± 83.3 mm for the comparison between M2 and the static camera.

Table 3. Markers position comparison with the static camera in the swimming protocol.

	Wrist—Vertical		Foot—Vertical		Hip—Horizontal
	M1 (mm)	M2 (mm)	M1 (mm)	M2 (mm)	M1 (mm)	M2 (mm)
Mean Error	9.5	−5.5	2.3	13.4	−18.4	33.6
SD Error	14.5	57.4	17	19.6	78	81.3
Mean Abs Error	14.3	49.2	12.7	17.3	53.1	77.5

shows the comparison between the static and the moving camera for the vertical coordinates of wrist and foot and the horizontal position of the hip in front crawl swimming. The leg kick amplitude was estimated as 331.7, 339.9, and 328.1 mm for the static camera, M1, and M2, respectively. The lowest wrist point during the stroke cycle was 715.0, 699.1, and 634.3 mm for the static camera, M1, and M2, respectively.

Table 4. Velocity results in the swimming protocol and comparison with the static camera.

	Evaluated Velocity			Static Camera Comparison
	Static (m/s)	M1 (m/s)	M2 (m/s)	M1 (m/s)	M2 (m/s)
Mean	1.33	1.33	1.37	<0.01	0.04
SD	0.17	0.17	0.23	0.08	0.1
Mean Abs	-	-	-	0.06	0.08

shows the evaluated swimming velocity and the comparison between the static and the moving cameras. Figure 3 shows the instantaneous velocity of one stroke cycle evaluated by the three methods, and Figure 4 contains Bland–Altman plots.

4. Discussion

The errors evaluated were similar to the reported in the literature for similar experimental conditions, but in 3D. With respect to the static markers, M1 presents a 12.1 mm mean error, while de Jesus et al. [10] report values between 10 and 13 mm. For the rod protocols, M1 presents a mean error of −2.1 mm, while Silvatti et al. [19] report values between −1.12 and 6.09 mm. The single moving camera approach was similar or better than the static camera results in both the validation protocols and also allows for a bigger area of analysis, which ensures its viability as a method to evaluate 2D underwater kinematics.

The single moving camera calibrated frame by frame based on the pool markers (M1) showed the lowest errors in the static markers protocol when only the central markers were evaluated, and also showed a smaller error compared to that of the static camera in the rod protocol, even when only the central markers of the static camera were considered. In this way, the single moving camera not only allows for covering all the swimming pool, but also showed the best accuracy and precision for 2D underwater kinematic analysis.

The accuracy of the single moving camera calibrated based on the principal point (M2) depends on the ability to constrain the camera movement degrees of freedom as well as of the number of reference markers in the pool. Even using a trolley car mounted on a rail and having reference points at each 190 mm, the largest errors in evaluating the positions of the markers occur for M2, which is approximately 18 mm for the static marker protocol (which could be acceptable precision for some applications) and about 58 mm for the rod position compared to that of the static position, which is possibly a more severe error. However, for the rod length, the estimated error was lower than 0.1%, which makes sense since the possible errors in the extrinsic camera parameter equally affect all the pixels in the same frame. Considering that M2 made a rough estimate of the extrinsic parameters, while M1 looks for the optimal solution for each frame, M2 precision should be, at best, equal to that of M1. In fact, the difference of 1.8 mm between the methods for the rod length is lower than data sensitivity since each pixel corresponds to approximately 2.5 mm on average based on the field of view size and the camera resolution.

In the static marker and rod protocols, the magnitude of the error was correlated with the marker distance from the center of the view. Moreover, the errors evaluated considering only the markers in the central positions were lower for all the conditions. These findings were expected [9,10,11,12] and indicate another advantage of the use of a single moving camera: the possibility of keeping the target object always in the center of the field of view, reducing the reconstruction errors.

On several kinematic applications, the relation between the markers is more important than the real position and can be used to evaluate the distances, or the markers can be used to represent a body segment for instance. Considering that M2 also seems to be a viable alternative for some underwater 2D kinematics applications, in the cases where estimating the real position of a marker is relevant, M2 is not the best option. It is possible to estimate an offset value for M2 when trying to improve the position accuracy if there are some control markers with a known position visible during a trial; the mean error of these markers can be estimated and removed from the main data. However, since more reference markers in the pool are necessary for this approach, the usage of M1 is preferable, if possible.

M2 shows the same behavior in the swimming protocol, despite some larger errors in the positions of the markers, since the results of leg kick amplitude were quite similar between the methods. The mean error in terms of velocity should be 0.8% and 1.5% for M1 and M2, respectively, considering the cycle duration and mean error of the hip coordinate compared with those of the static camera. The use of a static camera is the most common method in 2D kinematic analysis, and this is why M1 and M2 were compared to it. However, it is important to remember that the static camera is not a gold standard. In fact, based on the results of the two validation protocols, the results of M1 are more likely to be closer to the real value. In this way, a larger difference between cameras does not necessarily mean a larger error.

Considering swimming velocity, M2 presents a slightly higher mean velocity with larger variability. Observing Figure 3, M2 evaluated a higher peak velocity and a lower minimal velocity, but the shape of the curves were quite similar: a single peak in the first arm stroke and a double peak in the second one, where all the peaks and minimal data were recorded approximately at the same moment for every method.

The Bland–Altman plots reveal an acceptable number of outliers (about 5%) for all the comparisons, and the distributions of the differences appear uniform across the estimated values for most of the cases, except for the comparison between M2 and the static camera for the rod and swimming protocols. The behavior in these situations seems to be contradictory, while in the swimming protocol, M2 improves the results (overestimated the higher values and underestimated the lower values), the opposite occurs in the rod protocol. The translation of the moving camera in the direction perpendicular to the image plane could generate this behavior, but this was probably the most constrained degree of freedom of the camera caused by the trolley rail.

5. Conclusions

In conclusion, a single moving camera is a viable option for underwater 2D kinematics since it allows for a larger capture area with better accuracy and precision compared to those of the current approach (static camera), probably due to the possibility to keep the target object in the center of the field of view. While the calibration process based on the pool marker (M1) presented the best results for all the tested conditions, the principal point method (M2) presented results similar or better than those of the static camera only when the relationship between the points is the parameter of interest and not their real position.