Next Article in Journal
Simultaneous Temperature and Relative Humidity Measurement Using Machine Learning in Rayleigh-Based Optical Frequency Domain Reflectometry
Next Article in Special Issue
Analyzing and Modeling the Dynamic Electrical Characteristics of Nanocomposite Large-Range Strain Gauges
Previous Article in Journal
Multi-Channel Fusion Decision-Making Online Detection Network for Surface Defects in Automotive Pipelines Based on Transfer Learning VGG16 Network
Previous Article in Special Issue
Assessing the Reliability and Validity of Inertial Measurement Units to Measure Three-Dimensional Spine and Hip Kinematics During Clinical Movement Tasks
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 24
Issue 24
10.3390/s24247912
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Measurement Properties of Wearable Kinematic-Based Data Collection Systems to Evaluate Ball Kicking in Soccer: A Systematic Review with Evidence Gap Map †

by
Luiz H. Palucci Vieira
1,*,‡,
Filipe M. Clemente
2,3,4,
Rui M. Silva
2,3,
Kelly R. Vargas-Villafuerte
5 and
Felipe P. Carpes
6
1
Grupo de Investigación en Tecnología Aplicada a la Seguridad Ocupacional, Desempeño y Calidad de Vida (GiTaSyC), Facultad de Ingeniería y Arquitectura, Campus Callao, Universidad César Vallejo (UCV), Callao 07001, Lima, Peru
2
Escola Superior Desporto e Lazer, Instituto Politécnico de Viana do Castelo, Rua Escola Industrial e Comercial de Nun’Álvares, 4900-347 Viana do Castelo, Portugal
3
Sport Physical Activity and Health Research & Innovation Center, 4900-347 Viana do Castelo, Portugal
4
Department of Biomechanics and Sport Engineering, Gdansk University of Physical Education and Sport, 80-336 Gdańsk, Poland
5
Facultad de Ciencias de la Salud, Campus Callao, Universidad César Vallejo (UCV), Callao 07001, Lima, Peru
6
Applied Neuromechanics Research Group, Laboratory of Neuromechanics, Federal University of Pampa (Unipampa), P.O. Box 118, Uruguaiana 97500-970, RS, Brazil
*
Author to whom correspondence should be addressed.
†
This article is a revised and expanded version of a conference paper entitled “Work in progress: reviewing measurement properties of wearables in computing ball kicking features”, which was presented as a poster at the 22nd LACCEI International Multi-Conference for Engineering, Education and Technology, Hybrid, 17–19 July 2024.
‡
Address: Av. Argentina 1795, Universidad César Vallejo, Campus Callao, Lima—Perú. Postal code: 07001.
Sensors 2024, 24(24), 7912; https://doi.org/10.3390/s24247912
Submission received: 14 November 2024 / Revised: 6 December 2024 / Accepted: 7 December 2024 / Published: 11 December 2024
(This article belongs to the Special Issue Advances in Sensor Technologies for Wearable Applications)
Browse Figures
Review Reports Versions Notes

Abstract

:
Kinematic assessment of ball kicking may require significant human effort (e.g., traditional vision-based tracking systems). Wearables offer a potential solution to reduce processing time. This systematic review collated measurement properties (validity, reliability, and/or accuracy) of wearable kinematic-based technology systems used to evaluate soccer kicking. Seven databases were searched for studies published on or before April 2024. The protocol was previously published and followed the PRISMA 2020 statement. The data items included any validity, reliability, and/or accuracy measurements extracted from the selected articles. Twelve articles (1011 participants) were included in the qualitative synthesis, showing generally (92%) moderate methodological quality. The authors claimed validity (e.g., concurrent) in seven of the eight studies found on the topic, reliability in two of three, and accuracy (event detection) in three of three studies. The synthesis method indicated moderate evidence for the concurrent validity of the MPU-9150/ICM-20649 InvenSense and PlayerMaker™ devices. However, limited to no evidence was identified across studies when considering wearable devices/systems, measurement properties, and specific outcome variables. To conclude, there is a knowledge base that may support the implementation of wearables to assess ball kicking in soccer practice, while future research should further evaluate the measurement properties to attempt to reach a strong evidence level.

1. Introduction

In the sports engineering field, particularly in the context of team ball sports, one of the main goals is to conceive monitoring tools that can be used in naturally occurring environments [1,2]. This is given the general poor construct validity of laboratory or even field-based tests outside game-play to evaluate skill-related performance, such as in the case of soccer kicking (i.e., passing or shooting) ability [3,4,5,6]. Testing and reporting the soccer kicking skill features (e.g., lower limb movement pattern and velocity) to coaching staff is of paramount importance since (a) it could help provide feedback aiming at perfecting players’ movements [1], (b) there are strong relationships between kicker mechanical parameters and kick outcomes [7,8,9,10], and (c) the number of injury occurrences (e.g., >10–15%) associated with ball-kicking events is not negligible [11,12,13], which in turn may be prevented with adequate monitoring of player motions under training or match conditions. In this sense, wearable technologies have been recognized to assist in biofeedback purposes [14]. Indeed, both the use/perception of the importance of wearables and the monitoring of key performance indicators derived from ball-kicking features (e.g., shooting or passing metrics) are frequent in soccer contexts [15,16].
To collect data on position, velocity, and acceleration during kicking movement, there are various electronic performance and tracking systems (EPTSs) currently commercially available, such as optical-based, inertial measurement units (IMUs), GNSS, and LPS [17,18,19]. High-speed semi-automatic three-dimensional optical systems are generally considered the “gold standard” given their lowest levels of reported measurement error [18,20]. However, these systems are limited in capture volumes and traditionally require reflective markers attached to players’ bodies [17] and can also be labor-intensive. Consequently, the use of this method during official competition is therefore limited. In this sense, artificial intelligence-assisted markerless video-based tracking could be a possible solution, representing a less invasive method for capturing kicking data [1,21]. Aside from the advantages of such a straightforward approach, the calibration of extended volumes is challenging, potential occlusion effects can still occur, and it is necessary to position the cameras in optimal locations (e.g., high). Once again, this can collectively constrain the use of optical-based systems when investigating in-game ball kicking in soccer.
Since the mid-2010s, the Fédération Internationale de Football Association (FIFA) has allowed the use of EPTSs [22]. Although biofeedback appears to be available in many of the current EPTSs, its definitive implementation in day-to-day practice has not yet been widely observed, as in the case of soccer kick testing/monitoring [23]. IMUs (e.g., Xsens Technologies B.V., Enschede, The Netherlands) represent a wearable type of EPTS that may allow kicking kinematics to be obtained [24]. In fact, wearables may assist in overcoming many of the limitations mentioned above associated with the use of video-based techniques, also helping in possibly providing real-time data [25]. On the other hand, not all of them are feasible to apply in competition settings. Examples included those multi-sensor apparatuses requiring unique clothing and/or reinforced attachment on various body locations, which may potentially constrain movement patterns. Starting from September 2023, a specific wearable foot-mounted apparatus that enables tracking lower extremity movements was the first of the category approved by FIFA, the so-called “PLAYERMAKER” system (Playermaker LTD) [26,27]. Of note is that there are recent criticisms regarding some aspects of FIFA’s quality performance reports for EPTSs [28,29].
While several previous literature reviews addressed methodological details of wearables applied to kinematic analysis in sports tasks [25,30,31,32,33,34,35,36,37,38,39], none have solely examined soccer kicking skills. Thus, a systematic analysis of the level of scientific evidence available concerning the specific measurement properties of wearables used in monitoring ball-kicking actions is still lacking. One particular concern when assessing soccer kicking movement kinematics refers to the potential distortion in time-series data caused by the impact of the foot with the ball [40,41]. Also, ball kicking may involve a very rapid change in speed values of the lower extremity from preparation to follow through. The concomitant demands upon velocity and accuracy constraints, which are known to elicit paradoxical central inputs of players [42], probably tend to contribute to producing a substantial degree of variability in the assessment of soccer kicking as well [43]. This emphasizes the need to choose the methods for collecting/processing kick data carefully. Taken together, there is an evident necessity to collate studies that have determined measurement errors of wearable technology devices used to measure biomechanical aspects of soccer kicking as this can help inform practitioners, researchers, and industry workers in the area. Therefore, the present study aims to synthesize evidence from scientific journal articles that provided data about measurement properties (validity, reliability, and/or accuracy) of wearable kinematic-based technology systems used to evaluate ball kicking in the soccer context through a systematic literature review.

2. Methods

The protocol established for the present systematic review was previously registered (OSF REGISTRIES—project ZM3J6) and is available in full elsewhere [44]. The Institutional Research Ethics Committee, from the Universidad César Vallejo Perú, granted permission to run this review study. The review protocol was established considering the items from the PRISMA 2020 statement (Table S1) [45].

2.1. Database Searches

Table 1 contains terms used for the searches across the databases selected. For the formulation of the search string, terms between the same column were combined with the Boolean operator OR, while the groups of terms between columns were combined with the Boolean operator AND. The searches were performed on 23 April 2024, through seven electronic databases: EBSCOHost, IEEE Xplore, Physical Therapy and Sports Medicine, ProQuest, PubMed, Scopus, and Web of Science.
The selected search terms attempted to respect the PICO headings method [46]. To assist in managing titles across the whole review process (i.e., from initial searches until the step of final inclusion), Zotero software was used in the present study (v6.0.30; https://www.zotero.org/; accessed on 5 February 2024). The researchers involved in each aspect of the development of all the review steps are described in the protocol study [44].

2.2. Eligibility Criteria

2.2.1. Inclusion Criteria

Studies were included (i) when published (even ahead of print) as original research articles (ii) in scientific peer-reviewed journals, (iii) having their abstracts available for screening in the respective databases, (iv) with full text available—in the English language—for download (i.e., full text download button available for the record in the respective database), and (v) describing aspects related to research ethics with human participants [47] or respective exemption to perform the investigation [48]. The inclusion criteria also included (vi) participants as soccer players and/or human data collected in a football-related environment, (vii) observation and/or experimental protocol including ball kicking (passing or shooting) while (viii) participants used at least one wearable kinematic-based device/system that was (ix) evaluated concerning its measurement properties (i.e., validity and/or reliability and/or accuracy aspects).

2.2.2. Exclusion Criteria

Studies were excluded if they (i) were published in the form of grey literature, (ii) were retracted, (iii) investigated only the application/feasibility aspects of wearables, (iv) assessed only movements pertaining to football codes other than ball kicking, (v) were explicitly conducted in football codes other than soccer, (vi) reported observations and/or experimental protocols where a ball was not kicked, (vii) reported results only concerning ball motion/flight, and (viii) were full texts with no information on the location to which the wearable was attached to the body of participants [44].

2.3. Data Extraction

Using custom-prepared Microsoft® Excel spreadsheets (v.Professional Plus 2016, Microsoft Corporation, Redmond, USA), the following indicators were extracted from the included studies, according to their specific aims [49,50]: (i) validity measures [e.g., correlation coefficient, root mean square difference (RMSD), and area under the receiver operating curve (AUC)]; (ii) reliability measures [e.g., intraclass correlation coefficient (ICC) and coefficient of variation (CV)], and/or accuracy measures (e.g., F1 score, prediction accuracy, and percentage of correctly classified activity). For all studies, the main characteristics of methods were also extracted, including the wearable (number of sensors, brand of the system/device used, location attached to the body, and acquisition frequency), metrics collected by the wearable, number of participants and sample profile (sex, age, playing level, and positional role), experimental protocol (aspects of testing/observation of ball kicking, duration, condition(s), and lower limb considered), form of data analysis, statistical outputs, and summary of findings/conclusions.

2.4. Methodological Quality and Risk of Bias Assessments

Here, methodological quality was assessed using the COSMIN checklist or the QUADAS-2 tool, depending on the measurement property addressed by each study. In particular, forms H and B from the COSMIN checklist were adopted to make judgments, respectively, on the quality of studies that evaluated validity and reliability aspects [51,52]. These studies were rated in each domain of the mentioned tool using a 4-point scale (excellent, good, fair, or poor) [53]. Subsequently, the ratings for each question were converted into values (excellent = 3 points, good = 2 points, fair = 1 point, and poor = 0 points), and then punctuations of all questions were summed for each study. For validity aspects, studies were deemed as having high quality (∑ = 14–18), moderate quality (∑ = 8–13), or low quality (∑ = 3–7). Likewise, for reliability aspects, studies were deemed as having high quality (∑ = 25–33), moderate quality (∑ = 14–24), or low quality (∑ = 3–13). For studies assessing accuracy aspects, the QUADAS-2 tool was adopted [54]. These studies were rated in each domain of the mentioned tool using a 3-point scale (low = 2 points, unclear = 1 point, high = 0 points) and then summed for each study. For accuracy aspects, studies were deemed as having high quality (∑ = 11–14), moderate quality (∑ = 8–10), or low quality (∑ = 0–7).
The risk of bias in results or inferences was determined using the RoBANS tool [55]. Each study was analyzed individually and rated as presenting a low, high, or unclear risk of bias concerning the selection of participants, confounding variables, measurement of exposure, blinding of outcome assessments, incomplete outcome data, and selective outcome reporting. RevMan software (v5.3; The Cochrane Collaboration, Denmark) was used to compute graphs of the resulting risk of bias. If any discrepancy existed in the rating between the two evaluators involved—in evaluating methodological quality and risk of bias in the included studies—a third (senior) researcher was then consulted until a consensus was reached.

2.5. Evidence Synthesis

To provide a synthesis of findings across the studies, the level of evidence was classified as (a) strong [consistent findings (≥75% of the studies reported results in the same direction) across multiple high-quality studies], (b) moderate (consistent findings across multiple moderate-quality and/or one high-quality study), (c) limited (consistent findings across one moderate-quality and/or only low-quality studies), (d) conflicting (<75% of studies reported results in the same direction), or (e) no evidence (no studies found). An evidence gap map was also constructed [44].

3. Results

A total of 676 title entries were initially identified when pooling results of the searches across all databases in addition to the titles manually identified. Following the elimination of duplications and low-relevant studies, the fields of title, abstract, and keywords of 80 studies were inspected. This procedure resulted in the exclusion of 44 studies. Finally, after reading the full texts of 36 studies assessed for eligibility and applying all the selection criteria, a total of 12 studies [56,57,58,59,60,61,62,63,64,65,66,67] addressing the measurement properties of wearable-based systems to evaluate soccer kicking features were included in the qualitative synthesis. Given the heterogeneity in methods observed across the included studies, a meta-analysis was not possible in the present review. Eight studies evaluated the respective validity aspects [56,57,61,62,63,64,66,67], three evaluated reliability [58,62,63], and three evaluated accuracy [59,60,65]. Figure 1 depicts the frequency of studies included according to the year of publication. Reasons for exclusions in the final eligibility stage included, for example, studies available only as conference proceedings, reporting results of kicking aggregated with other actions, or lacking a statement regarding an ethics committee (see Figure 2).

3.1. Characteristics of the Included Studies

The main aspects of the methods in the included studies are summarized in Table 2. The total number of participants was n = 1011 for all studies pooled (median = 11 participants). Seven studies included only male participants (58%) [56,57,59,62,63,64,66], four included both men and women (33%) [58,60,61,65], and in one, this information was unclear [67]. Eight studies were conducted with adult samples (67%) [56,57,58,59,62,63,64,66], two with youth (17%) [60,61], and one with both youth and adults [65]. One study included elite players [62], one both elite and sub-elite [56], and the remainder sub-elite, except two studies in which unclear information was provided [64,67].
Two studies reported the inclusion of samples of participants from all playing positions (pooled) [56,62], while the information related to the positional role of participants was unclear in all the remaining studies (83%). Common metrics collected by the wearables included velocity [pelvis [57] and foot (25%; [57,62,67])], acceleration [pelvis [59,66], thigh [59,66], shank [58,59,66], ankle [60,61], and foot (33%; [62,63,65,67])], angular displacement (pelvis, shank [57], and hip and knee [64]), angular velocity (pelvis [59,66], hip [64], knee [56,57,64], thigh [57,59,66], shank [57,59,66], ankle [62,63], and foot [65,67]), and angular acceleration of the hip and knee [56].
The number of sensors used in the data collections ranged from one sensor [67] to two (33%; [58,62,63,65]), four [60], five (42%; [56,59,61,64,66]), and seventeen sensors [57]. Placement locations included the waist [61], pelvis (33%; [57,59,64,66]), hip [60], thigh (42%; [56,57,59,64,66]), shank (50%; [56,57,58,59,64,66]), ankle (17%; [60,61]), and foot (42%; [57,62,63,65,67]). Approximately one-quarter of the studies analyzed preferred limbs [56,57,58], while half assessed both preferred and non-preferred limbs [59,62,63,64,65,66]. The information related to functional lateralization was unclear in three studies [60,61,67]. In addition, some studies also placed devices on the head, upper arm, forearm, sternum, shoulder, hand [57], wrist [61], and lower back [56].
Wearable devices/systems used in the included studies consisted of IMUs MPU-9150 (InvenSense) [56,59,62,63,66], ICM-20649 (InvenSense) [64,67], MVN Link (Xsens Technologies) [57], VICON Blue Trident (IMeasureU) [58], PlayerMaker™ [62,63], and accelerometers GT1M/GT3X (ActiGraph) [60] and GENEActiv (Activinsights) [61]. The information regarding the wearable devices/systems used was unclear in one study [65]. Acquisition frequencies ranged from 80 Hz [61], 100 Hz [67], 200 Hz [65], 240 Hz [57], 250 Hz [64], 500 Hz [56,59,66], 1000 Hz [63], to 1600 Hz [58]. Two studies presented unclear information relating to acquisition frequency [60,62].
The time of testing across studies varied from 5 min [58] to 10 min [56] and 15 min of warming-up, plus the experimental protocol [66]. One study provided 15 min of familiarization plus the experimental protocol [62]. Three studies provided descriptions of the duration of the protocol, consisting of 5 min [61], 20 min [60], and 35 ± 17 to 73 ± 38 min [65]. Unclear information about the total time duration of the warm-up and/or experimental protocol was observed in five studies (~42%; [57,59,63,64,67]), for which information on the composition of the experimental protocol, but not timing, was reported. The description of protocols adopted in all studies regarding the ball-kicking action is also presented in Table 2.
Ball kicking was collected as passing in eight studies (67%; [58,59,60,61,63,64,65,66]) and shooting in seven studies (58%; [56,57,59,64,65,66,67]). Unclear information about whether ball kicking was collected as a pass or shot was presented in one study [62]. Finally, most studies (92%) used protocols consisting of performing ball kicking as closed skill/constrained tasks [56,57,58,59,60,61,62,63,64,66,67]. There was only one exception in a study that considered data in both conditions, i.e., closed skill/constrained tasks and also open skill/unconstrained tasks format [65].

3.2. Measurement Properties

The main results regarding measurement properties (validity, reliability, and accuracy) presented by the wearable kinematic-based data collection systems to evaluate ball kicking in soccer are summarized in Table 3. The table also contains the summary—main conclusions—provided by the authors of each individual included study. In general, the validity of wearables was claimed by the authors in seven out of eight studies [56,57,62,63,64,66,67], reliability in two out of three studies [62,63], and accuracy in three out of three studies [59,60,65].

3.3. Methodological Quality and Risk of Bias in the Included Studies

Table 4 shows the results of the methodological quality assessment for the eight studies that addressed validity aspects. All studies showed moderate quality. Item 3 (sample size) presented a higher number of studies scoring “poor” (100%) followed by item 6 (38% of studies on this topic lacked correlations or AUC calculations).
Table 5 shows the results of the methodological quality assessment for the three studies that addressed reliability aspects. Once again, all studies showed moderate quality. Item 3 (sample size) presented a higher number of studies scoring “poor”, followed by item 11 (67% of studies on this topic lacked ICC calculations).
Table 6 shows the results of the methodological quality assessment for the three studies that addressed accuracy aspects. Two studies had moderate methodological quality while one study showed low methodological quality (33%). Item A from domain 1 (patient selection) showed the high number of studies rated with the poorest score (100% with a high risk of bias relating to the selection of participants).
Figure 3 and Figure 4 contain, respectively, the results of the risk of bias for individual studies and a summary of the results for all studies pooled. The item selection of participants presented a higher number of studies rated as high risk (17%). The item blinding of the outcome assessment showed a higher number of studies rated as unclear risk (75%).

3.4. Summary of Evidence

Figure 5 depicts the interaction between measurement properties and kinematic data collected by wearables in the studies included. Open circles are the number of studies. Thresholds for qualitative synthesis were applied according to those described previously (Section 2.5). Regardless of the outcome variable, taking into account the wearable devices/systems and the measurement properties (Figure 6), the MPU-9150 (InvenSense) showed moderate evidence for concurrent validity across studies [62,63,66], limited evidence for discriminative validity [56], within-session reliability [62], intra-unit reliability [63], and event detection accuracy [59], and no evidence for between-session reliability. The ICM-20649 (InvenSense) showed moderate evidence for concurrent validity across studies [64,67] and no evidence for the remainder of measurement properties. The PlayerMaker™ showed moderate evidence for concurrent validity across studies [62,63], limited evidence for within-session reliability [62] and intra-unit reliability [63], and no evidence for the remainder of the measurement properties. All the other devices/systems (MVN Link-Xsens Technologies [57], VICON Blue Trident-IMeasureU [58], GT1M/GT3X-ActiGraph [60], and GENEActiv-Activinsights [61]) showed limited to no evidence for the measurement properties. Finally, limited to no evidence was also identified across studies when considering wearable devices/systems, measurement properties, and specific outcome variables.

4. Discussion

The present systematic review aimed to synthesize the evidence on the measurement properties (i.e., validity, reliability, and/or accuracy) of wearable kinematic-based systems used for evaluating ball kicking in soccer. While a meta-analysis was not viable in the present investigation due to variations in various aspects of methods across the included studies and the generally reduced number of available published studies on the subject, the best evidence synthesis and gap map methods allowed us to draw some inferences. The main findings revealed moderate evidence supporting the concurrent validity of wearable systems, including the (i) MPU-9150, (ii) ICM-20649—both i and ii pertaining to InvenSense (San Jose, CA, USA)—and (iii) PlayerMaker™ (Tel Aviv, Israel), for soccer kicking analysis in controlled settings. However, evidence for between-session reliability and accuracy remains limited, particularly in real-game scenarios. The current study is a revised and expanded version of a previous conference paper [68].
The studies included predominantly focused on the validity of wearable systems, with a significant proportion showing its concurrent validity for different measures (e.g., joint angles, velocity, and acceleration). The moderate evidence supporting the concurrent validity of the wearable devices MPU-9150, ICM-20649, and PlayerMaker™ in controlled environments [62,63,64,67] suggests a great capability for these wearables to measure kinematic variables with a degree of accuracy that aligns with the gold-standard instruments for the same purpose. Regarding other wearables, such as the MVN Link system, Blair et al. [57] suggested that the MVN Link system has concurrent validity compared to a high-standard motion analysis system. However, the high error levels observed in faster kicking attempts indicate that while the system can detect movements, the precision of this system might be velocity-dependent. Moreover, limited evidence was found for devices like the GT1M/GT3X and GENEActiv. Their use in assessing kicking kinematics was less straightforward, showing moderate correlations for certain activities but not specifically highlighting their effectiveness for detailed kinematic kicking analysis [61]. The low cross-validity (i.e., for the GENEActiv) indicates that devices might be better suited for general activity recognition rather than precise kinematic measurements in particular movements or tasks. Most of these devices were not specifically designed for the high-speed, high-impact nature of soccer kicking, which may explain the limited evidence in kicking kinematic analysis.
The capability of these wearables to measure kinematic variables with a degree of accuracy that aligns with the gold-standard instruments is crucial as it benchmarks the actual technology used against established methods, providing a basis for further exploration into their practical utility in sporting settings [69]. However, the inherent variability in soccer kicking velocity and accuracy [43,70,71,72,73,74] requires wearables that measure and interpret these actions considering their context. This fact is not uniquely challenging to soccer; similar demands for contextual data interpretation are also needed in other sports where precision and variability in movement are paramount (e.g., tennis serves [75]). Furthermore, the distortion of time-series data due to the impact of the foot with the ball also complicates this scenario of contextual factors [40,41]. Indeed, the variability in validity outcomes of the included studies in the present systematic review exposes the complexity of collecting biomechanical data in real-world sports settings [14,30,76].
The still limited evidence on reliability (e.g., between-session) and the event detection accuracy showed an important gap in the practical application of wearable technologies in the context of soccer kicking assessment. This is critical as reliability over time and across different conditions is essential for wearables to be integrated effectively into the training process to improve decision-making. Another systematic review that addressed the use of IMUs in volleyball also identified reliability results that were worse than validity ones [77]. The limited between-session reliability and accuracy in real-game scenarios found in the present systematic review might be attributed to (i) the possible negative results from the (unpublished) research carried out [78]; (ii) on the occasion that (i) is true, then there might be a need to improve the existing data/equipment capture/processing methods and/or (iii) differences in sensor placement, the number of sensors used, or the specific measures being measured (e.g., velocity, acceleration). Burland et al. [58] explored the reliability of the VICON Blue Trident (IMeasureU) system, finding moderate to high ICC values during kicking actions. However, the lower reliability for ball kicking compared to other actions assessed (i.e., locomotion tasks without ball handling) suggests that the nature of kicking might introduce variability that affects the device’s consistency. This could be due to the placement of sensors or the sensitivity to rapid changes in acceleration. Regardless of the measurement property, replication studies are encouraged using an adequate/sufficient number of participants [79] given that the included studies of the present systematic review showed, in general, low sample sizes. The same problem of a small number of participants across the literature studies was identified in a systematic review that covered the topic of wearables in basketball [38].
Studies utilizing foot-mounted sensors like the PlayerMakerTM system have shown potential, yet the generalizability of these findings to other wearable configurations or all player positions remains uncertain. Specifically, the placement of sensors may directly affect data accuracy [80]. Foot-mounted sensors might excel in capturing kicking dynamics in controlled settings. Still, they might not generalize to real-world settings [81], such as the training context, where impacts and other inter-player contacts are very likely to occur. Thus, here, the presence of potential biases arising from experimental (e.g., controlled) conditions should be not overlooked. Using observations from well-defined ball kicking as closed skill/constrained tasks would possibly favor one outcome over others. Also, the need for regular calibration or updates to software algorithms to account for new data or different playing conditions is often overlooked, and reliability can be negatively affected [39]. Furthermore, as players fatigue, their biomechanics and outcomes change [82,83], potentially leading to different between-session data outputs from wearables [84]. This aspect of the soccer training context may explain the still limited evidence on the reliability and accuracy of data over multiple sessions. Moreover, accuracy in event detection, particularly for kicking, was not explicitly detailed for many of these devices. Notwithstanding, soccer kicking performance (e.g., ball release velocity) has been position-dependent in either youth/adult or men/women players [85,86,87]. In addition, in-game situations like ground duels could potentially create challenges for the signal processing of wearables in the soccer context and also vary according to playing position [88,89]. Thus, future analysis of measurement properties in wearables should consider the present results also separated by playing position. It is also advisable that the metrics included be justified in relation to their actual practical meaningfulness (e.g., importance to the monitoring of the performance of the kicking skill/overall player performance, age-related development, and/or injury risk mitigation).
The present systematic review has inherent limitations. Firstly, our focus was narrowed to the validity, reliability, and accuracy of wearable devices in soccer kicking actions, which might not capture the full capabilities of wearable technology applications in soccer or other sports. Although this seems to also be the case in other team sports (e.g., handball [90]), it is also necessary to recognize that in the analysis (i.e., qualitative synthesis) considering wearable devices/systems, measurement properties, and specific outcome variables, limited to no evidence was verified in the current literature. Another significant limitation is the demographic bias, where there was a predominance of men and adult participants in included studies, thereby limiting the generalizability of our findings across the broader populations. Using a traditional approach to systematically review the literature (e.g., excluding conference proceedings [91,92,93,94,95,96,97,98]) may have led to the overlooking of potentially worthwhile scientific findings in the synthesis presented. Our review’s lack of longitudinal studies highlights a gap in understanding how these technologies perform over time. It would be beneficial for future studies to adopt a broader inclusion criterion, encompassing studies that include other football codes to ensure that wearable technology’s benefits are applicable. For example, another potential limitation of the methods in the current review is that—in an attempt to collate specific evidence for the team sport of soccer—the realization of studies explicitly conducted in football codes other than soccer was established as an exclusion criterion. Also, ball-kicking outcomes are age-dependent [99,100,101,102]. In studies regarding the measurement property accuracy, only one used a sample of exclusively senior participants, while the other two used solely youth or mixed samples (adult and senior pooled). Extrapolating the measurement properties of studies with youth participants to seniors requires caution, and the opposite, too. The same thought can be derived regarding competitive standards; results encountered in non-elite levels may not be interchangeable with the elite due to the distinct kicking outputs according to playing standards [103,104,105]. In the present review, only two studies investigated the measurement properties of wearables in elite players, and their use in practice has become increasingly widespread in recent years among professional clubs. Moreover, the standardization of testing and calibration protocols for these devices could improve the consistency and comparability of the outcomes (e.g., using recommendations from the International Society of Biomechanics [106]). Finally, there is a need to test and report aspects related to the robustness of sensor solutions, their adaptation to player body segments, and resilience levels to cope with impacts and other inter-player contacts. Longitudinal research also considering the duration of official matches and training sessions should be prioritized to assess the long-term reliability and utility of wearable data.

5. Conclusions

From the present systematic review with the best evidence synthesis and evidence gap map methods, it is possible to conclude that there is a knowledge base that may support the implementation of wearables to assess ball kicking in practice. The wearable devices/systems that so far have the highest level of evidence—moderate—for the validity (i.e., concurrent) across the existing literature studies are the MPU-9150 (InvenSense), ICM-20649 (InvenSense), and PlayerMaker™ (Tel Aviv, Israel). In contrast, limited to no evidence is noted regardless of the wearable devices/systems considered regarding both reliability (e.g., between-session) and accuracy (event detection) aspects. Also, the conclusions are applicable to a greater extent to male adult players. Future research should further evaluate the measurement properties of wearables to attempt to reach a strong evidence level regarding their performance in assessing ball kicking in soccer, mainly considering the inclusion of a sufficient number of players, under-represented populations (e.g., female and youth players), and data collection approaches during actual training sessions and official matches.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/s24247912/s1, Table S1: PRISMA 2020 Checklist used in the current systematic review study.

Author Contributions

Methodology: L.H.P.V. and K.R.V.-V.; formal analysis and investigation: L.H.P.V., F.M.C., K.R.V.-V. and F.P.C.; conceptualization, data curation, visualization, project administration, and funding acquisition: L.H.P.V.; writing—original draft preparation: L.H.P.V. and R.M.S.; writing—review and editing: L.H.P.V., F.M.C., R.M.S., K.R.V.-V. and F.P.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Fondo de Apoyo a la Investigación Docente Investiga UCV (Proyecto de investigación básica “Validez de sistemas tecnológicos vestibles para medir las características del pateo en el contexto del fútbol: una revisión sistemática y síntesis de las mejores evidencias”—Convocatoria PROYECTO RENACYT UCV 2023—Campus CALLAO; Project (proyecto investigación docente) registered on the TRILCE plataform on the date of: 14/11/2023). This study was conducted as a part of working plan 2024 of the research group led by the first author (RESOLUCIÓN DE VICERRECTORADO DE INVESTIGACIÓN N°088-2024-VI-UCV). The APC was funded in part by Universidad César Vallejo.

Institutional Review Board Statement

Ethical review/approval was waived by the Comité de Ética en Investigación de las Escuelas Profesionales de Educación Inicial y Ciencias del Deporte de la Universidad César Vallejo (IdProyecto 2761, Report date: 12/03/2024). In addition, studies potentially eligible for our synthesis but lacking statements regarding ethics committees were therefore not considered in the current review results.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data supporting the findings of the current review are primarily derived from previously published studies, which have been cited when pertinent. Additional datasets also containing information that supports our findings are available without undue reservation in with The Open Science Framework (OSF) OSF (https://doi.org/10.17605/OSF.IO/28BDV). The percentage in the similarity report of the manuscript submitted and the latest versions were also made available (respectively 10%—https://doi.org/10.5281/zenodo.14053101 and 12%—https://doi.org/10.5281/zenodo.14286234). This article is a revised and expanded version of a conference paper entitled “Work in progress: reviewing measurement properties of wearables in computing ball kicking features”, which was presented as a poster at the 22nd LACCEI International Multi-Conference for Engineering, Education and Technology, Hybrid, 17–19 July 2024.

Acknowledgments

The authors would like to thank the technical support provided by Felipe Armando Chang Marquez and Walter Manuel Rea Olivares. We are also thankful for the information made available by the Centro de Información/Biblioteca Virtual MyLOFT-UCV.

Conflicts of Interest

The authors declare no conflicts of interest. The results of the present study do not constitute an endorsement of the products by the authors. In addition, the funder had no role in the design of this study; in the collection, analysis, or interpretation of data; in the writing of this manuscript; or in the decision to publish the results.

References

  1. Palucci Vieira, L.H.; Santiago, P.R.P.; Pinto, A.; Aquino, R.; Torres, R.d.S.; Barbieri, F.A. Automatic Markerless Motion Detector Method against Traditional Digitisation for 3-Dimensional Movement Kinematic Analysis of Ball Kicking in Soccer Field Context. Int. J. Environ. Res. Public Health 2022, 19, 1179. [Google Scholar] [CrossRef] [PubMed]
  2. Wixted, A.J.; Billing, D.; James, D.A. Validation of Trunk Mounted Inertial Sensors for Analysing Running Biomechanics under Field Conditions, Using Synchronously Collected Foot Contact Data. Sports Eng. 2010, 12, 207–212. [Google Scholar] [CrossRef]
  3. Palucci Vieira, L.H.; Santinelli, F.B.; Carling, C.; Kellis, E.; Santiago, P.R.P.; Barbieri, F.A. Acute Effects of Warm-Up, Exercise and Recovery-Related Strategies on Assessments of Soccer Kicking Performance: A Critical and Systematic Review. Sports Med. 2021, 51, 661–705. [Google Scholar] [CrossRef] [PubMed]
  4. Ré, A.H.N.; Cattuzzo, T.M.; Santos, F.M.C.; Monteiro, C.B.M. Anthropometric Characteristics, Field Test Scores and Match-Related Technical Performance in Youth Indoor Soccer Players with Different Playing Status. Int. J. Perform. Anal. Sport 2014, 14, 482–492. [Google Scholar] [CrossRef]
  5. Serpiello, F.R.; Cox, A.; Oppici, L.; Hopkins, W.G.; Varley, M.C. The Loughborough Soccer Passing Test Has Impractical Criterion Validity in Elite Youth Football. Sci. Med. Footb. 2017, 1, 60–64. [Google Scholar] [CrossRef]
  6. Wen, D.; Robertson, S.; Hu, G.; Song, B.; Chen, H. Measurement Properties and Feasibility of the Loughborough Soccer Passing Test: A Systematic Review. J. Sports Sci. 2018, 36, 1682–1694. [Google Scholar] [CrossRef]
  7. Fullenkamp, A.M.; Campbell, B.M.; Laurent, C.M.; Lane, A.P. The Contribution of Trunk Axial Kinematics to Poststrike Ball Velocity During Maximal Instep Soccer Kicking. J. Appl. Biomech. 2015, 31, 370–376. [Google Scholar] [CrossRef]
  8. Hunter, A.H.; Angilletta, M.J.; Wilson, R.S. Behaviors of Shooter and Goalkeeper Interact to Determine the Outcome of Soccer Penalties. Scand. J. Med. Sci. Sports 2018, 28, 2751–2759. [Google Scholar] [CrossRef]
  9. Palucci Vieira, L.H.; Barbieri, F.A.; Kellis, E.; Oliveira, L.; Aquino, R.; Cunha, S.; Bedo, B.; Manechini, J.; Santiago, P. Organisation of Instep Kicking in Young U11 to U20 Soccer Players. Sci. Med. Footb. 2021, 5, 111–120. [Google Scholar] [CrossRef]
  10. Secco Faquin, B.; Teixeira, L.A.; Coelho Candido, C.R.; Boari Coelho, D.; Bayeux Dascal, J.; Alves Okazaki, V.H. Prediction of Ball Direction in Soccer Penalty through Kinematic Analysis of the Kicker. J. Sports Sci. 2023, 41, 668–676. [Google Scholar] [CrossRef]
  11. Drummond, F.A.; Soares, D.d.S.; Silva, H.G.R.d.; Entrudo, D.; Younes, S.D.; Neves, V.N.d.S.; Medeiros, J.M.d.A.; Roza, P.R.d.S.; Pacheco, I. Incidence of Injuries in Soccer Players—Mappingfoot: A Prospective Cohort Study. Rev. Bras. Med. Esporte 2021, 27, 189–194. [Google Scholar] [CrossRef]
  12. Kerin, F.; Farrell, G.; Tierney, P.; Persson, U.M.; Vito, G.D.; Delahunt, E. Its Not All about Sprinting: Mechanisms of Acute Hamstring Strain Injuries in Professional Male Rugby Union—A Systematic Visual Video Analysis. Br. J. Sports Med. 2022, 56, 608–615. [Google Scholar] [CrossRef] [PubMed]
  13. Waldén, M.; Krosshaug, T.; Bjørneboe, J.; Andersen, T.E.; Faul, O.; Hägglund, M. Three Distinct Mechanisms Predominate in Non-Contact Anterior Cruciate Ligament Injuries in Male Professional Football Players: A Systematic Video Analysis of 39 Cases. Br. J. Sports Med. 2015, 49, 1452–1460. [Google Scholar] [CrossRef] [PubMed]
  14. McDevitt, S.; Hernandez, H.; Hicks, J.; Lowell, R.; Bentahaikt, H.; Burch, R.; Ball, J.; Chander, H.; Freeman, C.; Taylor, C.; et al. Wearables for Biomechanical Performance Optimization and Risk Assessment in Industrial and Sports Applications. Bioengineering 2022, 9, 33. [Google Scholar] [CrossRef]
  15. Herold, M.; Kempe, M.; Bauer, P.; Meyer, T. Attacking Key Performance Indicators in Soccer: Current Practice and Perceptions from the Elite to Youth Academy Level. J. Sports Sci. Med. 2021, 20, 158. [Google Scholar] [CrossRef]
  16. Davidson, T.-K.; Barrett, S.; Toner, J.; Towlson, C. Professional Soccer Practitioners’ Perceptions of Using Performance Analysis Technology to Monitor Technical and Tactical Player Characteristics within an Academy Environment: A Category 1 Club Case Study. PLoS ONE 2024, 19, e0298346. [Google Scholar] [CrossRef]
  17. Aughey, R.J.; Ball, K.; Robertson, S.J.; Duthie, G.M.; Serpiello, F.R.; Evans, N.; Spencer, B.; Ellens, S.; Cust, E.; Haycraft, J.; et al. Comparison of a Computer Vision System against Three-Dimensional Motion Capture for Tracking Football Movements in a Stadium Environment. Sports Eng. 2022, 25, 2. [Google Scholar] [CrossRef]
  18. Crang, Z.L.; Duthie, G.; Cole, M.H.; Weakley, J.; Hewitt, A.; Johnston, R.D. The Validity and Reliability of Wearable Microtechnology for Intermittent Team Sports: A Systematic Review. Sports Med. 2021, 51, 549–565. [Google Scholar] [CrossRef]
  19. Schmid, M.; Lames, M. Correction of Systematic Errors in Electronic Performance and Tracking Systems. Sports Eng. 2023, 26, 30. [Google Scholar] [CrossRef]
  20. Linke, D.; Link, D.; Lames, M. Validation of Electronic Performance and Tracking Systems EPTS under Field Conditions. PLoS ONE 2018, 13, e0199519. [Google Scholar] [CrossRef]
  21. Pinheiro, G.d.S.; Jin, X.; Costa, V.T.D.; Lames, M. Body Pose Estimation Integrated with Notational Analysis: A New Approach to Analyze Penalty Kicks Strategy in Elite Football. Front. Sports Act. Living 2022, 4, 818556. [Google Scholar] [CrossRef] [PubMed]
  22. Dunn, M.; Hart, J.; James, D. Wearing Electronic Performance and Tracking System Devices in Association Football: Potential Injury Scenarios and Associated Impact Energies. Proceedings 2018, 2, 232. [Google Scholar] [CrossRef]
  23. Shan, G.; Zhang, X. From 2D Leg Kinematics to 3D Full-Body Biomechanics-the Past, Present and Future of Scientific Analysis of Maximal Instep Kick in Soccer. Sports Med. Arthrosc. Rehabil. Ther. Technol. 2011, 3, 23. [Google Scholar] [CrossRef] [PubMed]
  24. Blair, S.; Robertson, S.; Duthie, G.; Ball, K. Biomechanics of Accurate and Inaccurate Goal-Kicking in Australian Football: Group-Based Analysis. PLoS ONE 2020, 15, e0241969. [Google Scholar] [CrossRef] [PubMed]
  25. Chambers, R.; Gabbett, T.J.; Cole, M.H.; Beard, A. The Use of Wearable Microsensors to Quantify Sport-Specific Movements. Sports Med. 2015, 45, 1065–1081. [Google Scholar] [CrossRef]
  26. FIFA, (Fédération Internationale de Football Association). PLAYERMAKER. Available online: https://www.fifa.com/origin1904-p.cxm.fifa.com/technical/football-technology/resource-hub (accessed on 31 October 2023).
  27. FIFA, (Fédération Internationale de Football Association). FIFA Innovation Programme Project Summary. FIFA Innovation Challenge: Innovative EPTS. Project Title: Lower-Limb Foot-Mounted IMU in Football. Company/Product: PlayerMaker. Available online: https://digitalhub.fifa.com/m/7d4998e61ed66f63/original/executive-summary-for-website-template-12-pm.pdf (accessed on 9 July 2024).
  28. Oliva-Lozano, J.M.; Muyor, J.M. Understanding the FIFA Quality Performance Reports for Electronic Performance and Tracking Systems: From Science to Practice. Sci. Med. Footb. 2022, 6, 398–403. [Google Scholar] [CrossRef]
  29. Moura, F.A. The Video Assistant Referee (VAR) in Soccer: Where Is the Biomechanics? J. Phys. Educ. 2023, 34, e3435. [Google Scholar] [CrossRef]
  30. Adesida, Y.; Papi, E.; McGregor, A.H. Exploring the Role of Wearable Technology in Sport Kinematics and Kinetics: A Systematic Review. Sensors 2019, 19, 1597. [Google Scholar] [CrossRef]
  31. Aroganam, G.; Manivannan, N.; Harrison, D. Review on Wearable Technology Sensors Used in Consumer Sport Applications. Sensors 2019, 19, 1983. [Google Scholar] [CrossRef]
  32. Camomilla, V.; Bergamini, E.; Fantozzi, S.; Vannozzi, G. Trends Supporting the In-Field Use of Wearable Inertial Sensors for Sport Performance Evaluation: A Systematic Review. Sensors 2018, 18, 873. [Google Scholar] [CrossRef]
  33. De Fazio, R.; Mastronardi, V.M.; De Vittorio, M.; Visconti, P. Wearable Sensors and Smart Devices to Monitor Rehabilitation Parameters and Sports Performance: An Overview. Sensors 2023, 23, 1856. [Google Scholar] [CrossRef] [PubMed]
  34. Fong, D.T.-P.; Chan, Y.-Y. The Use of Wearable Inertial Motion Sensors in Human Lower Limb Biomechanics Studies: A Systematic Review. Sensors 2010, 10, 11556–11565. [Google Scholar] [CrossRef] [PubMed]
  35. Poitras, I.; Dupuis, F.; Bielmann, M.; Campeau-Lecours, A.; Mercier, C.; Bouyer, L.J.; Roy, J.-S. Validity and Reliability of Wearable Sensors for Joint Angle Estimation: A Systematic Review. Sensors 2019, 19, 1555. [Google Scholar] [CrossRef] [PubMed]
  36. Rana, M.; Mittal, V. Wearable Sensors for Real-Time Kinematics Analysis in Sports: A Review. IEEE Sens. J. 2021, 21, 1187–1207. [Google Scholar] [CrossRef]
  37. Rum, L.; Sten, O.; Vendrame, E.; Belluscio, V.; Camomilla, V.; Vannozzi, G.; Truppa, L.; Notarantonio, M.; Sciarra, T.; Lazich, A.; et al. Wearable Sensors in Sports for Persons with Disability: A Systematic Review. Sensors 2021, 21, 1858. [Google Scholar] [CrossRef]
  38. Santos-Gago, J.M.; Ramos-Merino, M.; Vallarades-Rodriguez, S.; Álvarez-Sabucedo, L.M.; Fernández-Iglesias, M.J.; García-Soidán, J.L. Innovative Use of Wrist-Worn Wearable Devices in the Sports Domain: A Systematic Review. Electronics 2019, 8, 1257. [Google Scholar] [CrossRef]
  39. Seckin, A.C.; Ates, B.; Seckin, M. Review on Wearable Technology in Sports: Concepts, Challenges and Opportunities. Appl. Sci. 2023, 13, 10399. [Google Scholar] [CrossRef]
  40. Augustus, S.; Amca, A.M.; Hudson, P.E.; Smith, N. Improved Accuracy of Biomechanical Motion Data Obtained during Impacts Using a Time-Frequency Low-Pass Filter. J. Biomech. 2020, 101, 109639. [Google Scholar] [CrossRef]
  41. Nunome, H.; Lake, M.; Georgakis, A.; Stergioulas, L.K. Impact Phase Kinematics of Instep Kicking in Soccer. J. Sports Sci. 2006, 24, 11–22. [Google Scholar] [CrossRef]
  42. Palucci Vieira, L.H.; Carling, C.; da Silva, J.P.; Santinelli, F.B.; Polastri, P.F.; Santiago, P.R.P.; Barbieri, F.A. Modelling the Relationships between EEG Signals, Movement Kinematics and Outcome in Soccer Kicking. Cogn. Neurodyn. 2022, 16, 1303–1321. [Google Scholar] [CrossRef]
  43. Lees, A.; Rahnama, N. Variability and Typical Error in the Kinematics and Kinetics of the Maximal Instep Kick in Soccer. Sports Biomech. 2013, 12, 283–292. [Google Scholar] [CrossRef] [PubMed]
  44. Vieira, L.H.P.; Clemente, F.M.; Marquez, F.A.C.; Olivares, W.M.R.; Villafuerte, K.R.V.; Carpes, F.P. Accuracy Standards of Wearable Technologies for Assessment of Soccer Kicking: Protocol for a Systematic Literature Review. JMIR Res. Protoc. 2024, 13, e57433. [Google Scholar] [CrossRef] [PubMed]
  45. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef] [PubMed]
  46. Methley, A.M.; Campbell, S.; Chew-Graham, C.; McNally, R.; Cheraghi-Sohi, S. PICO, PICOS and SPIDER: A Comparison Study of Specificity and Sensitivity in Three Search Tools for Qualitative Systematic Reviews. BMC Health Serv. Res. 2014, 14, 579. [Google Scholar] [CrossRef]
  47. Vergnes, J.-N.; Marchal-Sixou, C.; Nabet, C.; Maret, D.; Hamel, O. Ethics in Systematic Reviews. J. Med. Ethics 2010, 36, 771–774. [Google Scholar] [CrossRef]
  48. Winter, E.M.; Maughan, R.J. Requirements for Ethics Approvals. J. Sports Sci. 2009, 27, 985. [Google Scholar] [CrossRef]
  49. Silva, R.; Rico-González, M.; Lima, R.; Akyildiz, Z.; Pino-Ortega, J.; Clemente, F.M. Validity and Reliability of Mobile Applications for Assessing Strength, Power, Velocity, and Change-of-Direction: A Systematic Review. Sensors 2021, 21, 2623. [Google Scholar] [CrossRef]
  50. Multhuaptff, W.; Moreno-Villanueva, A.; Soler-López, A.; Fernández-Peña, E.; Rico-González, M.; Clemente, F.M.; Bravo-Cucci, S.; Pino-Ortega, J. Concurrent-Validity and Reliability of Photocells in Sport: A Systematic Review. J. Hum. Kinet. 2024, 92, 53. [Google Scholar] [CrossRef]
  51. Mokkink, L.B.; Terwee, C.B.; Knol, D.L.; Stratford, P.W.; Alonso, J.; Patrick, D.L.; Bouter, L.M.; de Vet, H.C.W. Protocol of the COSMIN Study: COnsensus-Based Standards for the Selection of Health Measurement INstruments. BMC Med. Res. Methodol. 2006, 6, 2. [Google Scholar] [CrossRef]
  52. Mokkink, L.B.; Terwee, C.B.; Knol, D.L.; Stratford, P.W.; Alonso, J.; Patrick, D.L.; Bouter, L.M.; de Vet, H.C. The COSMIN Checklist for Evaluating the Methodological Quality of Studies on Measurement Properties: A Clarification of Its Content. BMC Med. Res. Methodol. 2010, 10, 22. [Google Scholar] [CrossRef]
  53. Terwee, C.B.; Mokkink, L.B.; Knol, D.L.; Ostelo, R.W.J.G.; Bouter, L.M.; de Vet, H.C.W. Rating the Methodological Quality in Systematic Reviews of Studies on Measurement Properties: A Scoring System for the COSMIN Checklist. Qual. Life Res. 2012, 21, 651–657. [Google Scholar] [CrossRef] [PubMed]
  54. Whiting, P.F. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann. Intern. Med. 2011, 155, 529. [Google Scholar] [CrossRef] [PubMed]
  55. Kim, S.Y.; Park, J.E.; Lee, Y.J.; Seo, H.-J.; Sheen, S.-S.; Hahn, S.; Jang, B.-H.; Son, H.-J. Testing a Tool for Assessing the Risk of Bias for Nonrandomized Studies Showed Moderate Reliability and Promising Validity. J. Clin. Epidemiol. 2013, 66, 408–414. [Google Scholar] [CrossRef] [PubMed]
  56. Bastiaansen Bram, J.C.; Vegter Riemer, J.K.; Wilmes, E.; de Ruiter Cornelis, J.; Goedhart, E.A.; Lemmink Koen, A.P.M.; Brink, M.S. Biomechanical Load Quantification of National and Regional Soccer Players with an Inertial Sensor Setup during a Jump, Kick, and Sprint Task: Assessment of Discriminative Validity. Sports Eng. 2024, 27, 17. [Google Scholar] [CrossRef]
  57. Blair, S.; Duthie, G.; Robertson, S.; Hopkins, W.; Ball, K. Concurrent Validation of an Inertial Measurement System to Quantify Kicking Biomechanics in Four Football Codes. J. Biomech. 2018, 73, 24–32. [Google Scholar] [CrossRef]
  58. Burland, J.P.; Outerleys, J.B.; Lattermann, C.; Davis, I.S. Reliability of Wearable Sensors to Assess Impact Metrics during Sport-Specific Tasks. J. Sports Sci. 2021, 39, 406–411. [Google Scholar] [CrossRef]
  59. Cuperman, R.; Jansen, K.M.B.; Ciszewski, M.G. An End-to-End Deep Learning Pipeline for Football Activity Recognition Based on Wearable Acceleration Sensors. Sensors 2022, 22, 1347. [Google Scholar] [CrossRef]
  60. de Vries, S.I.; Engels, M.; Garre, F.G. Identification of Children’s Activity Type with Accelerometer-Based Neural Networks. Med. Sci. Sports Exerc. 2011, 43, 1994–1999. [Google Scholar] [CrossRef]
  61. Duncan, M.J.; Dobell, A.; Noon, M.; Clark, C.C.T.; Roscoe, C.M.P.; Faghy, M.A.; Stodden, D.; Sacko, R.; Eyre, E.L.J. Calibration and Cross-Validation of Accelerometery for Estimating Movement Skills in Children Aged 8–12 Years. Sensors 2020, 20, 2776. [Google Scholar] [CrossRef]
  62. Lewis, G.; Towlson, C.; Roversi, P.; Domogalla, C.; Herrington, L.; Barrett, S. Quantifying Volume and High-Speed Technical Actions of Professional Soccer Players Using Foot-Mounted Inertial Measurement Units. PLoS ONE 2022, 17, e0263518. [Google Scholar] [CrossRef]
  63. Marris, J.; Barrett, S.; Abt, G.; Towlson, C. Quantifying Technical Actions in Professional Soccer Using Foot-Mounted Inertial Measurement Units. Sci. Med. Footb. 2022, 6, 203–214. [Google Scholar] [CrossRef] [PubMed]
  64. Steijlen, A.; Burgers, B.; Wilmes, E.; Bastemeijer, J.; Bastiaansen, B.; French, P.; Bossche, A.; Jansen, K. Smart Sensor Tights: Movement Tracking of the Lower Limbs in Football. Wear. Technol. 2021, 2, e17. [Google Scholar] [CrossRef] [PubMed]
  65. Stoeve, M.; Schuldhaus, D.; Gamp, A.; Zwick, C.; Eskofier, B.M. From the Laboratory to the Field: IMU-Based Shot and Pass Detection in Football Training and Game Scenarios Using Deep Learning. Sensors 2021, 21, 3071. [Google Scholar] [CrossRef]
  66. Wilmes, E.; de Ruiter, C.J.; Bastiaansen, B.J.C.; van Zon, J.F.J.A.; Vegter, R.J.K.; Brink, M.S.; Goedhart, E.A.; Lemmink, K.A.P.M.; Savelsbergh, G.J.P. Inertial Sensor-Based Motion Tracking in Football with Movement Intensity Quantification. Sensors 2020, 20, 2527. [Google Scholar] [CrossRef] [PubMed]
  67. Yu, C.; Huang, T.-Y.; Ma, H.-P. Motion Analysis of Football Kick Based on an IMU Sensor. Sensors 2022, 22, 6244. [Google Scholar] [CrossRef]
  68. Palucci Vieira, L.H.; Vargas-Villafuerte, K.R.; Clemente, F.M.; Marquez, F.A.; Rea Olivares, W.M.; Carpes, F.P. Work in Progress: Reviewing Measurement Properties of Wearables in Computing Ball Kicking Features. In Proceedings of the 22nd LACCEI International Multi-Conference for Engineering, Education and Technology (LACCEI 2024), San Jose, Costa Rica, 17–19 July 2024; Larrondo Petrie, M.M., Texier, J., Eds.; Fundacion LACCEI: Bogota, Colombia, 2024; p. 1188. Available online: https://laccei.org/LACCEI2024-CostaRica/index-by-track.html#track20 (accessed on 6 December 2024).
  69. Currell, K.; Jeukendrup, A.E. Validity, Reliability and Sensitivity of Measures of Sporting Performance. Sports Med. 2008, 38, 297–316. [Google Scholar] [CrossRef]
  70. Hunter, A.H.; Angilletta, M.J.; Pavlic, T.; Lichtwark, G.; Wilson, R.S. Modeling the Two-Dimensional Accuracy of Soccer Kicks. J. Biomech. 2018, 72, 159–166. [Google Scholar] [CrossRef]
  71. Sado, N.; Yazawa, M.; Tominaga, T.; Akutsu, K. Inter-Individual Variability in Elliptical and Diagonal Error Distributions Potentially Relevant to Optimal Motor Planning in Football Instep Kicking. Hum. Mov. Sci. 2024, 97, 103272. [Google Scholar] [CrossRef]
  72. Andersen, T.B.; Dörge, H.C. The Influence of Speed of Approach and Accuracy Constraint on the Maximal Speed of the Ball in Soccer Kicking. Scand. J. Med. Sci. Sports 2011, 21, 79–84. [Google Scholar] [CrossRef]
  73. Amiri-Khorasani, M.; Osman, N.A.A.; Yusof, A. Kinematics Analysis: Number of Trials Necessary to Achieve Performance Stability during Soccer Instep Kicking. J. Hum. Kinet. 2010, 23, 15–20. [Google Scholar] [CrossRef]
  74. Berjan Bacvarevic, B.; Pazin, N.; Bozic, P.R.; Mirkov, D.; Kukolj, M.; Jaric, S. Evaluation of a Composite Test of Kicking Performance. J. Strength Cond. Res. 2012, 26, 1945–1952. [Google Scholar] [CrossRef] [PubMed]
  75. Buszard, T.; Reid, M.; Krause, L.; Kovalchik, S.; Farrow, D. Quantifying Contextual Interference and Its Effect on Skill Transfer in Skilled Youth Tennis Players. Front. Psychol. 2017, 8, 1931. [Google Scholar] [CrossRef] [PubMed]
  76. Benson, L.C.; Clermont, C.A.; Ferber, R. New Considerations for Collecting Biomechanical Data Using Wearable Sensors: The Effect of Different Running Environments. Front. Bioeng. Biotechnol. 2020, 8, 86. [Google Scholar] [CrossRef] [PubMed]
  77. Villarejo-García, D.H.; Moreno-Villanueva, A.; Soler-López, A.; Reche-Soto, P.; Pino-Ortega, J. Use, Validity and Reliability of Inertial Movement Units in Volleyball: Systematic Review of the Scientific Literature. Sensors 2023, 23, 3960. [Google Scholar] [CrossRef] [PubMed]
  78. Borg, D.N.; Barnett, A.G.; Caldwell, A.R.; White, N.M.; Stewart, I.B. The Bias for Statistical Significance in Sport and Exercise Medicine. J. Sci. Med. Sport 2023, 26, 164–168. [Google Scholar] [CrossRef]
  79. Knudson, D. Confidence Crisis of Results in Biomechanics Research. Sports Biomech. 2017, 16, 425–433. [Google Scholar] [CrossRef]
  80. Küderle, A.; Roth, N.; Zlatanovic, J.; Zrenner, M.; Eskofier, B.; Kluge, F. The Placement of Foot-Mounted IMU Sensors Does Affect the Accuracy of Spatial Parameters during Regular Walking. PLoS ONE 2022, 17, e0269567. [Google Scholar] [CrossRef]
  81. Palucci Vieira, L.H.; Santiago, P.R.P.; Aquino, R. Footwear Technology, Running Outputs, and Technical Performance in Soccer Match-Play. J. Appl. Physiol. 2024, 137, 831–832. [Google Scholar] [CrossRef]
  82. Palucci Vieira, L.H.; Carling, C.; Kalva-Filho, C.A.; Santinelli, F.B.; Velluto, L.A.G.; da Silva, J.P.; Clemente, F.M.; Kellis, E.; Barbieri, F.A. Recovery of Kicking Kinematics and Performance Following Repeated High-Intensity Running Bouts in the Heat: Can a Rapid Local Cooling Intervention Help Young Soccer Players? J. Sports Sci. 2023, 41, 430–440. [Google Scholar] [CrossRef]
  83. Maly, T.; Sugimoto, D.; Izovska, J.; Zahalka, F.; Mala, L. Effect of Muscular Strength, Asymmetries and Fatigue on Kicking Performance in Soccer Players. Int. J. Sports Med. 2018, 39, 297–303. [Google Scholar] [CrossRef]
  84. Truppa, L.; Guaitolini, M.; Garofalo, P.; Castagna, C.; Mannini, A. Assessment of Biomechanical Response to Fatigue through Wearable Sensors in Semi-Professional Football Referees. Sensors 2020, 21, 66. [Google Scholar] [CrossRef] [PubMed]
  85. Rađa, A.; Erceg, M.; Grgantov, Z. Inter and Intra Positional Differences in Ball Kicking Between U-16 Croatian Soccer Players. Montenegrin J. Sports Sci. Med. 2016, 5, 11–15. [Google Scholar]
  86. Hernandez-Martinez, J.; Perez-Carcamo, J.; Canales-Canales, S.; Coñapi-Union, B.; Cid-Calfucura, I.; Herrera-Valenzuela, T.; Branco, B.H.M.; Valdés-Badilla, P. Body Composition and Physical Performance by Playing Position in Amateur Female Soccer Players. Appl. Sci. 2024, 14, 5665. [Google Scholar] [CrossRef]
  87. Khorasani, M.A.; Osman, N.A.A.; Yusof, A. Biomechanical Responds of Instep Kick between Different Positions in Professional Soccer Players. J. Hum. Kinet. 2009, 22, 21–27. [Google Scholar] [CrossRef]
  88. Ermidis, G.; Randers, M.B.; Krustrup, P.; Mohr, M. Technical Demands across Playing Positions of the Asian Cup in Male Football. Int. J. Perform. Anal. Sport 2019, 19, 530–542. [Google Scholar] [CrossRef]
  89. Dellal, A.; Chamari, K.; Wong, D.P.; Ahmaidi, S.; Keller, D.; Barros, R.; Bisciotti, G.N.; Carling, C. Comparison of Physical and Technical Performance in European Soccer Match-Play: FA Premier League and La Liga. Eur. J. Sport Sci. 2011, 11, 51–59. [Google Scholar] [CrossRef]
  90. Kovář, M.; Cuberek, R. Use of Inertial Measurement Units in Handball: A Review. Stud. Sport. 2024, 18, 83–100. [Google Scholar] [CrossRef]
  91. Chakma, A.; Md Faridee, A.Z.; Roy, N.; Sajjad Hossain, H.M. Shoot like Ronaldo: Predict Soccer Penalty Outcome with Wearables. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops, Austin, TX, USA, 23–27 March 2020; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
  92. Dorschky, E.; Schuldhaus, D.; Koerger, H.; Eskofier, B.M. A Framework for Early Event Detection for Wearable Systems. In Proceedings of the 2015 ACM International Symposium on Wearable Computers, Osaka, Japan, 7–11 September 2015; Association for Computing Machinery, Inc.: New York, NY, USA, 2015; pp. 109–112. [Google Scholar]
  93. Emam, A.M.; Ali, O.T.; Atia, A. Football Activities Classification. In Proceedings of the 2023 Intelligent Methods, Systems, and Applications (IMSA), Giza, Egypt, 15–16 July 2023; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2023; pp. 520–525. [Google Scholar]
  94. Kim, W.; Kim, M. Soccer Kick Detection Using a Wearable Sensor. In Proceedings of the 2016 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea, 19–21 October 2016; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2016; pp. 1207–1209. [Google Scholar]
  95. Kondo, Y.; Ishii, S.; Aoyagi, H.; Hossain, T.; Yokokubo, A.; Lopez, G. FootbSense: Soccer Moves Identification Using a Single IMU. In Sensor- and Video-Based Activity and Behavior Computing, Proceedings of the 3rd International Conference on Activity and Behavior Computing (ABC 2021), Bangkok, Thailand, 22–23 October 2021; Ahad, M.A., Inoue, S., Roggen, D., Fujinami, K., Eds.; Smart Innovation, Systems and Technologies; Springer: Singapore, 2022; Volume 291, pp. 115–131. [Google Scholar]
  96. Mascher, K.; Laller, S.; Wieser, M. Development of Smart Shin Guards for Soccer Performance Analysis Based on Mems Accelerometers, Machine Learning, and GNSS. In Proceedings of the ICL-GNSS 2021 WiP Proceedings: Proceedings of the International Conference on Localization and GNSS (ICL-GNSS 2021), Tampere, Finland, 1–3 June 2021; Ometov, A., Nurmi, J., Lohan, E.-S., Torres-Sospedra, J., Kuusniemi, H., Eds.; CEUR Workshop Proceedings (CEUR-WS.org). Volume 2880. [Google Scholar]
  97. Zhou, B.; Wirth, M.; Martindale, C.; Koerger, H.; Zwick, C.; Cruz, H.; Eskofier, B.; Lukowicz, P. Smart Soccer Shoe: Monitoring Foot-Ball Interaction with Shoe Integrated Textile Pressure Sensor Matrix. In Proceedings of the 2016 ACM International Symposium on Wearable Computers, Heidelberg, Germany, 12–16 September 2016; Association for Computing Machinery: New York, NY, USA, 2016; pp. 64–71. [Google Scholar]
  98. Zhou, P.; Yu, A.; Wu, M. Motion Control of Mobile Robot for Moving Object Capture and Shooting. In Proceedings of the 2006 4th IEEE International Conference on Industrial Informatics, Singapore, 16–18 August 2006; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2007; pp. 1369–1374. [Google Scholar]
  99. Vieira, L.H.P.; Cunha, S.A.; Moraes, R.; Barbieri, F.A.; Aquino, R.; Oliveira, L.d.P.; Navarro, M.; Bedo, B.L.S.; Santiago, P.R.P. Kicking Performance in Young U9 to U20 Soccer Players: Assessment of Velocity and Accuracy Simultaneously. Res. Q. Exerc. Sport 2018, 89, 210–220. [Google Scholar] [CrossRef]
  100. Hunter, A.H.; Smith, N.M.A.; Camata, T.V.; Crowther, M.S.; Mather, A.; Souza, N.M.; Ramos-Silva, L.F.; Pazetto, N.F.; Moura, F.A.; Wilson, R.S. Age- and Size-Corrected Kicking Speed and Accuracy in Elite Junior Soccer Players. Sci. Med. Footb. 2022, 6, 29–39. [Google Scholar] [CrossRef]
  101. Holfelder, B.; Schott, N. Object Control Skill Performance Across the Lifespan: A Cross Sectional Study. Res. Q. Exerc. Sport 2022, 93, 825–834. [Google Scholar] [CrossRef]
  102. Luhtanen, P. Kinematics and Kinetics of Maximal Instep Kicking in Junior Soccer Players. In Proceedings of the first World Congress of Science and Football, Liverpool, UK, 13–17 April 1987; Reilly, T., Lees, A., Davids, K., Murphy, W.J., Eds.; Science and Football (Routledge Revivals). Routledge: London, UK, 2013; pp. 441–448, ISBN 978-0-203-72003-5. [Google Scholar]
  103. Ali, A.; Williams, C.; Hulse, M.; Strudwick, A.; Reddin, J.; Howarth, L.; Eldred, J.; Hirst, M.; McGregor, S. Reliability and Validity of Two Tests of Soccer Skill. J. Sports Sci. 2007, 25, 1461–1470. [Google Scholar] [CrossRef] [PubMed]
  104. Rodríguez-Lorenzo, L.; Fernandez-del-Olmo, M.; Martín-Acero, R. A Critical Review of the Technique Parameters and Sample Features of Maximal Kicking Velocity in Soccer. Strength Cond. J. 2015, 37, 26. [Google Scholar] [CrossRef]
  105. Rađa, A.; Kuvačić, G.; Giorgio, A.D.; Sellami, M.; Ardigò, L.P.; Bragazzi, N.L.; Padulo, J. The Ball Kicking Speed: A New, Efficient Performance Indicator in Youth Soccer. PLoS ONE 2019, 14, e0217101. [Google Scholar] [CrossRef] [PubMed]
  106. Cereatti, A.; Gurchiek, R.; Mündermann, A.; Fantozzi, S.; Horak, F.; Delp, S.; Aminian, K. ISB Recommendations on the Definition, Estimation, and Reporting of Joint Kinematics in Human Motion Analysis Applications Using Wearable Inertial Measurement Technology. J. Biomech. 2024, 173, 112225. [Google Scholar] [CrossRef]
Sensors 24 07912 g001
Figure 1. Description of the number of included studies according to the publication year.
Figure 1. Description of the number of included studies according to the publication year.
Sensors 24 07912 g001
Sensors 24 07912 g002
Figure 2. PRISMA flow diagram.
Figure 2. PRISMA flow diagram.
Sensors 24 07912 g002
Sensors 24 07912 g003
Figure 3. Judgments made about each risk of bias item for each included study [56,57,58,59,60,61,62,63,64,65,66,67]. (+) Low risk. (–) High risk. (?) Unclear risk.
Figure 3. Judgments made about each risk of bias item for each included study [56,57,58,59,60,61,62,63,64,65,66,67]. (+) Low risk. (–) High risk. (?) Unclear risk.
Sensors 24 07912 g003
Sensors 24 07912 g004
Figure 4. Risk of bias for each item as percentages across all the included studies.
Figure 4. Risk of bias for each item as percentages across all the included studies.
Sensors 24 07912 g004
Sensors 24 07912 g005
Figure 5. Interaction between the measurement properties tested and kinematic data collected by the wearables. In the figures, open circles represent the number of studies included and vel: velocity; acc: acceleration; ang: angular; and disp: displacement.
Figure 5. Interaction between the measurement properties tested and kinematic data collected by the wearables. In the figures, open circles represent the number of studies included and vel: velocity; acc: acceleration; ang: angular; and disp: displacement.
Sensors 24 07912 g005
Sensors 24 07912 g006
Figure 6. Evidence gap map regarding the measurement properties tested and the wearable devices/systems used. Open circles represent the number of studies found.
Figure 6. Evidence gap map regarding the measurement properties tested and the wearable devices/systems used. Open circles represent the number of studies found.
Sensors 24 07912 g006
Table 1. Search terms used for the current systematic review through a PICO framework.
Table 1. Search terms used for the current systematic review through a PICO framework.
PopulationInterventionComparisonOutcome
soccerwearable *validitykick *
football *inertial measurement unitreliabilityshoot *
association footballIMUmeasurement errorpass *
11-a-sideacceleromet *accuracyskill
microtechnologyprecisiontechnical
micro-electrical mechanical system
MEMS
global positioning system
global navigation satellite system
local positioning system
GPS
GNSS
LPS
* Wildcard term.
Table 2. Some characteristics extracted from the methods used in the studies included in the current review.
Table 2. Some characteristics extracted from the methods used in the studies included in the current review.
ReferenceMetrics CollectedSexAgeLevelNPositional
Roles		TestedWearable
Device/System		FrequencyExperimental ProtocolForm of Data AnalysisValidity ResultsReliability ResultsAccuracy Results
ValidityReliabilityAccuracy
Bastiaansen et al. [56]Hip load; knee load (derived from joint angular accelerations); and knee extension velocityMenAdultElite; sub-elite28All (pooled)✓ IMUs (MPU-9150, InvenSense, San Jose, CA, USA)500 HzFive 5 m maximal instep kicks aiming at a full-sized goal without accuracy demandsData compared across playing levelsMANOVA----
Blair et al. [57]Foot speed; pelvis velocity; shank, thigh, and knee (angular velocity); shank and pelvis (sagittal angle) at impact; and max support-knee extension, min kick-leg knee angle, and max hip extensionMenAdultSub-elite10--✓ IMUs (MVN Link, Xsens Technologies B.V., Enschede, The Netherlands)240 HzFive 12 m instep kicks, five 12 m inside kicks, five 20 m instep kicks, and five maximal instep kicks Data compared against a 12-camera motion analysis system (T-40 series, Vicon Nexus v2, Oxford, UK) GLMM; variances; error; Studentized
residual vs. predicted plots 		----
Burland et al. [58]Tibial acceleration value; cumulative impact load; total steps; and cumulative bone stimulus (number of steps multiplied by the tibial acceleration)Men; womenAdultSub-elite10-- ✓ VICON IMeasureU Blue Trident dual-g sensors (IMeasureU, Auckland, New
Zealand)		1600 HzThree trials passing the ball forward at a self-selected paceData analyzed across three test sessions at least 7–10 days apart--ICC--
Cuperman et al. [59]Triaxial accelerations and angular velocities of pelvis, thigh, and shank MenAdultSub-elite11-- ✓IMUs (Ivensense
MPU-9150) 		500 HzFootball-related activities, including
passes, shots, jumps, and sprints, among others, to simulate an actual football match 		Data compared against a manually annotated category (activity)----F1 score Prediction Accuracy
de Vries et al. [60]Accelerometer data (counts) of the ankleMen; womenYouthSub-elite58-- ✓ActiGraph accelerometers (GT1M/GT3X; ActiGraph,
Pensacola, FL, USA) 		--Monitored 20 min activities including sitting, standing, walking, running, rope skipping,
kicking the ball, and cycling		Data compared against research assistant recordings----%Correctly classified activity Contingency tables
Duncan et al. [61]Triaxial accelerometry data for the ankleMen; womenYouthSub-elite20--✓ GENEActiv
accelerometers (Activinsights, Cambridge, UK) 		80 HzMonitored 5 min for each activity with 5 min resting: lying supine, standing, running, and instep passing a football (size 3 ball, over a distance of 5 m at a cadence of 10 and 20 passes/min) and dribblingData correlated with metabolic equivalents; compared among activity typesrho-Spearman; ROC curve AUC
Sensitivity Specificity		----
Lewis et al. [62]Velocity of the foot—or ball release velocityMenAdultElite4All (pooled)✓✓ IMUs (PlayerMaker™, Tel Aviv, Israel; including two components from MPU-9150, InvenSense, CA, USA)--Twelve kicks in a static ball (six with each foot) at low, moderate, or high subjective intensities with foot region self-selectedData correlated with joint angular velocity from a high-speed camera system (Quintic Consultancy Ltd., Sutton Coldfield, UK); data analyzed within session for each subjective intensityr-PearsonCV--
Marris et al. [63]Ball touch and release occurrences derived from accelerometer and gyroscope tracesMenAdultSub-elite12--✓✓ IMUs (PlayerMaker™, Tel Aviv, Israel; including two components from MPU-9150, InvenSense, CA, USA) 1000 HzA total of 8640 ball touches and 5760 releases in technical soccer tasks where a given player served the ball to another, considering two pre-determined distances (13.2 and 18.7 m)Data contrasted with manual coding/video (SportsCode Elite, v. 11.2.23, SportsTec, Warriewood, Australia); data analyzed across three codings made by the performance analyst concerning a same specific subset of tasksProportion of agreementCV--
Steijlen et al. [64]Hip and knee joint angles and angular velocities MenAdult--1--✓ IMUs ICM-20649 (InvenSense, San Jose, CA, USA)250 HzThree trials kicking a ball
preceded by a few steps, at three different intensities (50, 80, and 100% of maximum effort)		Data compared against and correlated with eight cameras (Vicon V5 cameras, Vicon Motion Systems Ltd., Oxford, UK)RMSD CMC----
Stoeve et al. [65]Triple-axis linear acceleration and angular velocityMen; womenYouth; adultSub-elite836-- ✓IMUs (brand not specified)200 HzA total of 8424 shots and 24,254
passes taken from 10 sessions in a laboratory (controlled exercises) and 10 during training or games (field sessions not following a fixed protocol)		Data compared against labeling made by trained experts using video camera recordings----F1 score Sensitivity
Wilmes et al. [66]Three-dimensional acceleration, angular velocity, and magnetic field strength MenAdultSub-elite11--✓ IMUs (MPU 9150, Invensense, San Jose, CA, USA)500 HzFour short passes (low intensity), four long passes (medium intensity), and four maximum instep kicks (maximal intensity) all interspersed with about 10 s restingData compared against and correlated with eight optoelectronic motion cameras (Vicon V5 cameras, Vicon
Motion Systems Ltd., Oxford, UK)		ANOVA RMSD CMC----
Yu et al. [67]Three-dimensional displacements, acceleration, and angular velocity; foot velocity and backwing height------10--✓ IMU sensor (ICM-20649, TDK InvenSense MEMS Motion Tracking™ Device) 100 HzInstep kicking testData compared against “Tracker” using two high-speed cameras positioned in front and side view (brand not specified)RMSD %Error Bland–Altman plots ----
N: number of participants; min: minimum; max: maximum; ICC: intraclass correlation coefficient; GLMM: general linear mixed model; % percentage; RMSD: root mean square difference; CMC: coefficient of multiple correlation; -- represents not applicable, information not reported, or unclear.
Table 3. Results regarding measurement properties extracted from the studies included in the current review.
Table 3. Results regarding measurement properties extracted from the studies included in the current review.
ReferenceWearable Device/SystemValidityReliabilityAccuracySummary
ConcurrentDiscriminativeCrossedBetween-SessionWithin-SessionIntra-UnitEvent Detection
Bastiaansen et al. [56]MPU-9150, InvenSense Knee extension velocity and load
p = 0.02
ES = 0.94–0.95

Hip load
p = 0.97
ES = 0.02 		Authors claim the discriminative validity of the wearable for the knee load but not for the hip load
Blair et al. [57]MVN Link, Xsens TechnologiesAll metrics collected
p = --
ES = trivial		 Authors claim the overall concurrent validity of the wearable
Burland et al. [58]IMeasureU Blue Trident dual-g, VICON Cumulative impact load and step count
ICC = 0.58–0.87
p ≤ 0.056

Cumulative bone stimulus
ICC = 0.95–0.96
p < 0.001		 Authors indicate lower reliability of the wearable to measure ball kicking as compared to other soccer actions
Cuperman et al. [59]Ivensense
MPU-9150		 CNN + bLSTM model
F1 score = 96.67%
Accuracy = 98.3%

QDA, kNN, and decision tree models
Accuracy = 40–90%		Authors claim better accuracy of the wearable using deep learning processing models as compared to traditional machine learning
de Vries et al. [60]GT1M/GT3X, ActiGraph One-axis ANN models
Correctly classified activity = 71.4–77.9%

Three-axis ANN models
Correctly classified activity = 82.1–82.4%		Authors claim better accuracy of the wearable using classification models considering triaxial data as compared to uniaxial
Duncan et al. [61]GENEActiv, Activinsights Acc count
rho = 0.67–0.75
p = 0.0001

AUC = 0.62–0.72
Sensitivity = 59.1–75.3
Specificity = 66.3–68.1		 Authors indicate low cross-validity of the wearable output
Lewis et al. [62]PlayerMaker™/MPU-9150, InvenSenseBall release velocity
r2 = 0.96
p = --		 Ball release velocity
CV = 3.93–14.35%		 Authors claim the concurrent validity and reliability of the wearable
Marris et al. [63]PlayerMaker™/MPU-9150, InvenSenseBall touches and releases
PA = 95.1–97.6%
SE = 0.0–0.1%		 Ball touches and releases
CV = 1.8–2.3%
SE = 0.0–0.2%		 Authors claim the concurrent validity and reliability of the wearable
Steijlen et al. [64]ICM-20649, InvenSenseHip and knee angle RMSD = 4–18°
CMC = 0.63–0.99

Angular velocity
RMSD = 61–103°/s
CMC = 0.93–0.99
p = --		 Authors claim the concurrent validity of the wearable
Stoeve et al. [65]brand not specified CNN model
F1 score = 0.887–0.928
Sensitivity = 45–84%

SVM model
F1 score = 0.648–0.815
Sensitivity = 21–39%

LSTM and convLSTM models
F1 score = 0.777–0.910
Sensitivity = --		Authors claim the accuracy of the wearable using the CNN processing model but not the SVM model; deep learning models outperformed machine learning
Wilmes et al. [66]MPU 9150, InvensenseHip and knee angle RMSD = 5–8°
CMC = 0.96–0.97

Angular velocity
RMSD = 78–177°/s
CMC = 0.81–0.89
p = --		 Authors claim the concurrent validity of the wearable
Yu et al. [67]ICM-20649, TDK InvenSenseReconstructed position
RMSD = 0.07 m

Foot velocity
RMSD = 7.47 m/s
Error = 4%

Backswing height
RMSD = 0.74 m
Error = 3%		 Authors claim the concurrent validity of the wearable
p = p-value; ES = effect size; ICC: intraclass correlation coefficient; RMSD: root mean square difference; CMC: coefficient of multiple correlation; -- represents information not reported or unclear.
Table 4. Ratings of methodological quality for the studies that addressed validity aspects.
Table 4. Ratings of methodological quality for the studies that addressed validity aspects.
ReferenceItem 1Item 2Item 3Item 4Item 5Item 6Item 7Rating
Bastiaansen et al. [56]GoodFairPoorFairExcellentExcellentNAModerate
Blair et al. [57]GoodFairPoorExcellentExcellentPoorNAModerate
Duncan et al. [61]GoodFairPoorFairExcellentExcellentExcellentModerate
Lewis et al. [62]ExcellentFairPoorExcellentExcellentExcellentNAModerate
Marris et al. [63]GoodFairPoorExcellentExcellentPoorNAModerate
Steijlen et al. [64]GoodFairPoorExcellentExcellentExcellentNAModerate
Wilmes et al. [66]GoodFairPoorExcellentExcellentExcellentNAModerate
Yu et al. [67]GoodFairPoorGoodExcellentPoorNAModerate
Items from the COSMIN checklist (form H); NA: not applicable.
Table 5. Ratings of methodological quality for the studies that addressed reliability aspects.
Table 5. Ratings of methodological quality for the studies that addressed reliability aspects.
ReferenceItem 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8Item 9Item 10Item 11Rating
Burland et al. [58]GoodFairPoorExcellentExcellentExcellentGoodExcellentGoodFairExcellentModerate
Lewis et al. [62]ExcellentFairPoorExcellentGoodFairGoodFairGoodFairPoorModerate
Marris et al. [63]GoodFairPoorExcellentGoodFairGoodFairGoodFairPoorModerate
Items from the COSMIN checklist (form B). Items 12–14 did not apply to studies included in the present review.
Table 6. Ratings of methodological quality for the studies that addressed accuracy aspects.
Table 6. Ratings of methodological quality for the studies that addressed accuracy aspects.
ReferenceDomain 1Domain 2Domain 3Domain 4Rating
Item AItem BItem AItem BItem AItem BItem A
Cuperman et al. [59]HighUnclearLowUnclearUnclearUnclearUnclearLow
de Vries et al. [60]HighUnclearLowLowUnclearLowUnclearModerate
Stoeve et al. [65]HighLowLowLowUnclearLowUnclearModerate
Items from the QUADAS-2 tool.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Palucci Vieira, L.H.; Clemente, F.M.; Silva, R.M.; Vargas-Villafuerte, K.R.; Carpes, F.P. Measurement Properties of Wearable Kinematic-Based Data Collection Systems to Evaluate Ball Kicking in Soccer: A Systematic Review with Evidence Gap Map. Sensors 2024, 24, 7912. https://doi.org/10.3390/s24247912

AMA Style

Palucci Vieira LH, Clemente FM, Silva RM, Vargas-Villafuerte KR, Carpes FP. Measurement Properties of Wearable Kinematic-Based Data Collection Systems to Evaluate Ball Kicking in Soccer: A Systematic Review with Evidence Gap Map. Sensors. 2024; 24(24):7912. https://doi.org/10.3390/s24247912

Chicago/Turabian Style

Palucci Vieira, Luiz H., Filipe M. Clemente, Rui M. Silva, Kelly R. Vargas-Villafuerte, and Felipe P. Carpes. 2024. "Measurement Properties of Wearable Kinematic-Based Data Collection Systems to Evaluate Ball Kicking in Soccer: A Systematic Review with Evidence Gap Map" Sensors 24, no. 24: 7912. https://doi.org/10.3390/s24247912

APA Style

Palucci Vieira, L. H., Clemente, F. M., Silva, R. M., Vargas-Villafuerte, K. R., & Carpes, F. P. (2024). Measurement Properties of Wearable Kinematic-Based Data Collection Systems to Evaluate Ball Kicking in Soccer: A Systematic Review with Evidence Gap Map. Sensors, 24(24), 7912. https://doi.org/10.3390/s24247912

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop