Next Article in Journal
Changes in the Stress Tolerance of Dwarf Rabbits in Animal-Assisted Interventions
Next Article in Special Issue
Effect of Medial-Lateral, Inferior-Superior, and Rotational Positions of Palms on Muscle Recruitment during the Push-Up Exercise
Previous Article in Journal
Retrospective Study of the Association between Peri-Implantitis and Keratinized Mucosa
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment of Pain and External Load in Amputee Football Using Digital Pain Drawing and GNSS Tracking—A Pilot Study

1
Institute of Physical Culture Sciences, Faculty of Health and Physical Education, University of Szczecin, 70-453 Szczecin, Poland
2
Department of Paralympic Sport, University School of Physical Education in Wrocław, 51-612 Wrocław, Poland
3
Health Sciences Department, Jan Długosz University of Humanities and Natural Sciences in Częstochowa, 42-200 Częstochowa, Poland
4
Instituto Politécnico de Viana do Castelo, Escola Superior Desporto e Lazer, Rua Escola Industrial e Comercial de Nun’Álvares, 4900-347 Viana do Castelo, Portugal
5
Research Center in Sports Performance, Recreation, Innovation and Technology (SPRINT), 4960-320 Melgaço, Portugal
6
Instituto de Telecomunicações, Delegação da Covilhã, 1049-001 Lisbon, Portugal
7
Department of Physical Education, Gdańsk University of Physical Education and Sport, 80-336 Gdańsk, Poland
8
Department of Adapted Physical Activity, University of Physical Education in Poznań, 61-871 Poznań, Poland
9
SMI, Department of Health Science and Technology, Aalborg University, 9220 Aalborg, Denmark
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(14), 6978; https://doi.org/10.3390/app12146978
Submission received: 23 June 2022 / Revised: 6 July 2022 / Accepted: 7 July 2022 / Published: 10 July 2022
(This article belongs to the Special Issue Biomechanical and Physiological Measurement in Sports)

Abstract

:

Featured Application

Setting further directions for scientific studies of Amputee Football for designing injury prevention programs, developing methods of targeted physiotherapeutic treatment, minimizing injury burden, designing crutches, designing training process and monitoring psychophysical state of the players, and designing applications for pain monitoring.

Abstract

Amputee Football (AF) players move using lofstrand crutches (LC) and a single leg during training and matches, which may expose them to excessive loads. Due to a lack of scientific articles describing these issues, this pilot study aims to gain insight into the assessment of pain experiences, as well as external loads during training and matches in AF. An observational study design was followed. Twelve male AF players (2 goalkeepers—GK and 10 field players—FP), 29.9 ± 8.7 years, stature 178.3 ± 6.5 cm, body mass 77.2 ± 8.9 kg were involved in the study. After stature and body mass measures, participants filled out the questionnaire for pain. The players were familiarized with digital pain drawing (DPD). Afterwards, they reported pain typically felt in relation to regular AF training and playing matches—recall pain (RP). During a two-day training camp (TC) with two training sessions each day and during a two-day international tournament (IT) with two matches each day, the players were monitored using DPD, rate of perceived exertion (RPE) and overall pain level (OPL) scales. In addition, during an international tournament (IT), match players were monitored using a GNSS tracking system for external load assessment. All of the participants reported multiple locations of pain after AF training or a match. The area of recall pain (RP) was the highest: +0.5% when compared to the end of TC and +43% when compared to end of IT. The pain area registered at the end of IT was significantly lower (p = 0.028) compared to RP and lower without statistical significance when compared to the end of TC. Average RPE was 3.31 ± 1.38 and average OPL was 2.86 ± 1.81 in 0–10 scale. Typical RPE was higher than that registered at the end of IT. Also typical OPL was higher compared to that registered after the end of TC, which was higher than after the end of IT. The average distance covered by a FP during a match ranged from 2483.14 ± 583.64 m to 2911.08 ± 828.90 m. AF field players suffered pain as a consequence of training and matches. The loads coming from playing and training, combined with pain, may lead to injuries. Further research directions should include assessments of the relationship of pain characteristics, injuries and GNSS tracking parameters.

1. Introduction

Amputee football (also known as amputee soccer, ampfootball, ampfutbol—AF) is a variation of football for people with limb impairments. The rules are similar to football with some modifications—play time is 2 × 25 min with a 10-min halftime. Goalkeepers (GKs) have upper limb amputation or an undeveloped upper limb, field players (FPs) have lower limb amputation or undeveloped lower limbs and move using forearm crutches (Lofstrand Crutches—LC). In both cases, artificial limbs and prosthetics cannot be used during a match. Player substitutions are allowed with returns. The pitch dimensions of 60 × 40 m, goal dimensions of 5 × 2 m and penalty area of 10 × 8 m are all smaller compared to football [1,2]. Due to the fact that this sport does not have a long history, there are still few scientific papers on it. To the authors’ best knowledge, this paper is the first to describe the pain experienced by AF players in relation to the external load during training and matches using digital pain drawing and GNSS tracking.
Physical and physiological demands of the AF match are showing that AF is an intensive sport in which aerobic energy production dominates energy provision similar to football. Both AF and football are characterized by sustained movement incorporating frequent bursts of high-intensity activity interspersed with recovery periods. Registered after a match, LA is 5–6 mmol/L, which indicates that players are taxing the anaerobic energy systems regularly. As reported in the work of Simim et al. (2018), the average HR is 153 ± 15 bpm and HRmax 179 ± 14 bpm [3]. Nowak AM (2020) reports remarkably similar values: average HR 162 ± 17.8 bpm and HR max 182 ± 12.8 bpm. However, these works differ in parameters of total covered distance during a match: 5650 ± 1070 m in the work of Simim et al. [3] and 1800 ± 390 m in the work of Nowak (2020) [4]. Maechana et al. reports the value of 2984.2 ± 56.1 m [5].

1.1. Excessive Load in Amputee Football

Common to many elite athletes, pain and injury hinder performance and often relate to practice and game conditions [6,7]. The specificity of AF in which the players heavily rely on their upper body strength and endurance for speed and play [8], moving with the upper limbs and crutches, may pose a risk of overload, injury and pain to the upper limbs and upper body. Moreover, fatigue conditions can be found especially in the upper limbs—players demonstrate −17% worse performance in after-match push-up tests and −8% worse in medicine ball throw comparing to pre-match. Decreased performance of the single lower limb is also noticeable—players had 5% lower results in countermovement jumping comparing post-match to pre-match [9].
According to Tatar et al. (2018), playing AF with LC put on upper limbs a load exceeding 100% of the body weight (BW)—111% during walking, 119% during running and 175% BW during kicking the ball. The upper limbs are not naturally developed for locomotion purposes. Loads put on the single lower limb during running on crutches showed values of 188–196% BW [10]. These mechanisms of overload frequently repeated during trainings and matches may increase the incidence of both upper and lower limb injuries and pain [11,12].

1.2. External Load—Monitoring Distance during Game Play

GNSS—Global Navigation Satellite Systems tracking (named in common as GPS tracking) of individual athletes during football game play can provide information about the physical demands and external load [13,14]. In particular, data describing total distance (TD—total distance covered during a match), sprint distance (SD—distance covered with a speed of a minimum 4.75 m/s for at least 1 s. A sprint effort ends when speed has decreased below 75% of the sprint threshold) and high intensity running (HIR—distance covered with speed over 5 m/s) during game play in AF players. Although GPS tracking is a good tool for the assessment of match loads, simultaneously neither GPS nor HR monitoring is adequate for monitoring the training effort. Rico-González et al. (2022) indicate the following parameters as the most important in assessing the load in team sports: accelerations and decelerations, impacts, high intensity efforts, aerobic and anaerobic components—speed zones, total distance and relative distance per minute [15].
An understanding of the relationship of pain and loads on single lower extremity and upper extremities, including the hands, for mobilizing the crutches swiftly and accurately, is lacking [10]. An understanding of the internal and external loads would help to improve the training process, team management, design prevention and rehabilitation programs. However, to achieve the above setting, further study directions are needed.

1.3. Pain Monitoring

The monitoring of the different psychophysical parameters of players has gained more significance in professional sport and should especially have more emphasis in sport for the disabled. The same external load (for example, running the same distance at the same time) can provoke different internal loads in every individual (e.g., pending factors such as trainability, physical status, context) and result in different individual pains. With respect to pain, there are a number of questionnaires used for assessment, but mainly they do not include emphasis on the location, area, quality and changes in time regarding these parameters. For measuring perceptive load scales as a rate of perceived exertion (RPE), use of a modified Borg’s scale is common. The overall pain level (OPL) scale also uses subjective indication on a numerical 0–10 scale. One of the novel methods of pain assessment is the monitoring of pain by digital pain drawing. Digital pain drawings are a modern and reliable method for quantitative and qualitative pain assessment and is used successfully to assess pain in football players [16,17,18]. These digital pain drawings allow participants to communicate location, area, quality and intensity of pain in a timely manner and in relation to training and matches, bringing visualization of pain location, intensity and quality to a body scheme [17,18].
Previous research suggests that the arms and a single leg undergo excessive loads [10], which may cause pain and lead to injuries.

1.4. Aim

The aim of this study was to assess pain in relation to external load in Amputee Football players during training and matches.

2. Materials and Methods

2.1. Participants

Convenience sampling was applied. Twelve adult male AF players aged 29.9 ± 8.7 years, with a stature of 178.3 ± 6.5 cm and a body mass of 77.2 ± 8.9 kg, were involved in the study. All participants are from the Polish National Amputee Football Team, which includes 2 goalkeepers (GK) and 10 field players (FP). Eligibility criteria were presence and active participation in the training sessions and matches during TC and IT, no illness, no injuries and no medication. Among field players, there were 6 persons with lower limb amputations (2 above knee, 4 under knee) and 4 persons with underdeveloped lower limb—congenital agenesia. Five of the field players have healthy right lower limbs and the other five have healthy left lower limbs. Among goalkeepers, there were 2 players with upper limb amputation—one had right upper limb healthy and the other—left upper limb. Only four of the participants use crutches in everyday life, while other FP use crutches only for playing and training. AF goalkeepers do not use crutches in everyday life, obviously. Parameters mentioned adove are summarized in Table 1. The average years of playing Amputee Football was 7.4 ± 2.1 years. Subjects participated in the study of their own free will and were informed that they may terminate their participation at any time. Participants read and signed an informed consent form approved by the Senate Research Ethics Committee (project identification code: 26/2016 approval date: 13 October 2016).

2.2. Study Overview

This exploratory study observed one cohort of 12 AF players during a training camp and an international tournament. The training camp and international tournament were 7 days apart. The first was a two-day training camp of the Polish National Team (training camp—TC) and the second was an International Ampfootball Tournament (international tournament—IT), both held in Warsaw, Poland. During TC on the first day, participants underwent the initial measures (body mass, stature) and filled in a questionary about pain and injuries. Every participant was familiarized with the pain drawing software Navigate Pain (Aglance Solutions, Aalborg, Denmark) and how to mark all pain areas on the body diagram using a computer. Every match of the IT was monitored using the GNSS system, delivering a number of parameters (Figure 1).

2.3. Pain Assessment

After initial measures and familiarization with the Navigate Pain app, the participants drew pain maps showing a recall of typical pain (RP)—a recall of what they experience typically after a match following a one-week-long training microcycle. The study of Galve Villa, Palsson and Boudreau (2021) showed high accuracy and repeatability by comparing the current pain and its recall using a digital pain drawing [18]. During the 2-day TC, there were 2 training sessions each day of 90 min each. The players were drawing digital pain maps of their current state immediately after the end of the second training session. During 2 days of IT, there were 2 matches each day of 50 min each. The players were drawing digital pain maps of their current state immediately after the end of each of four tournament matches. The use of Navigate Pain App showed validity and reliability for assessment of pain parameters in patients [16,17,18]. Immediately after ending training or a match, the players individually noted with software a rating of perceived exertion (RPE—CR10) and overall pain level (OPL) using the visual numerical scale. The scales’ values were from 0 to 10, where 0 means no exertion or pain, 1 is the lowest and 10 the highest value of exertion or pain, respectively. Using RPE shows validity and reliability for the quantifying session and match load [19,20,21]. The main outcome extracted from digital pain drawings was the total pain area registered in pixels. The main outcomes extracted from RPE and OPL were average values after each day of TC and each match of IT.

2.4. GNSS Tracking Methods and Conditions

In order to determine the actual workload of key performance indicators (KPIs) using GNSS technology, an analysis of 4 matches of the international tournament was carried out. The position of goalkeeper has been removed due to his special role and rules of the game, which makes the GK stay in the penalty box of strictly reduced dimensions. The study only takes into account the actual playing time of individual players on the pitch, including all timeouts requested by the coach, additional time and the timeout between halves. The parameters characterizing the external loads were: total distance (TD) [m], distance in bands: 1.0–3.0 m/s; 3.0–5.0 m/s; 5.0–7.0 m/s; 7.0–9.0 m/s in meters, session time [min], activity time [min], peak acceleration [m/s2], peak deceleration [m/s2], maximum speed [m/s], sprint total distance (STD) [m], mean sprint distance [m], mean sprint duration [s], GNSS load and inertial load. Descriptions of all parameters can be found in Appendix A.
In addition to the sampling frequency of the device (10Hz), the accuracy of positioning measurement and its derivatives is influenced by other factors. The most important of these include the number of satellites connected and their horizontal dilution of precision (HDOP) [22,23]. HDOP values >1 are defined as ideal, while values >2 are defined as excellent [24]. In this study, the average number of satellites was 16.2 ± 3.19, and HDOP was 1.44 ± 0.32 during the first match and 16.75 ± 1.85, respectively, for HDOP 1.17 ± 0.13 in the second match; on the second day, 18.66 ± 1.88 for HDOP 1.22 ± 0.10 in the first match and 20 ± 4.65 for HDOP 0.95 ± 0.27 in the second game.
The research used a GNSS sensor from Integrated Bionics Inc., 2020 (Houston, TX, USA) called TITAN2 (technical parameters are provided in Appendix B). The data was initially prepared using the manufacturer’s web platform (export to Excel) and then subjected to statistical analysis. All device and software settings were original and were not modified for this research.

2.5. Questionnaire

The questionnaire that all field players (n = 10, goalkeepers were excluded) filled out after initial measures consisted of 19 questions. There were eight questions related to pain sensation in particular major areas of the body (hand and fingers, wrist, elbow, shoulder, ankle and foot, including toes, knee, hip and the groin area). There were 2 questions about pain after matches or training and during a match or training. The next 4 questions were about phantom pain and phantom feeling. There was one question about pain of the stump. The above 5 questions were addressed only to players with amputation (n = 6). There was 1 question about pain in the underdeveloped limb and one question asking if the players feel pain which they identify or connect with moving on crutches. In all the above questions the Likert scale was used with 5 possible answers: never, rarely, often, very often, always. There were another 2 questions asking how pain influences the players’ performance and motivation during training and matches. Possible answers to these two questions were: not at all; yes, to a small extent; yes, to a large extent; hard to define; increase performance. The questionnaire was prepared in Polish and was translated to English for the purpose of dissemination, as shown in Appendix C.

2.6. Statistical Analysis

Statistical analysis was performed using the Statistical Package for Social Sciences (SPSS, IBM). Descriptive statistics is presented in the form of mean and standard deviation. Normality of the data was tested using Shapiro–Wilk test. To investigate the area of pain reported by AF players, the total number of pixels for drawn areas of pain was extracted. The normality of data distribution was tested using the Shapiro–Wilk test for normality. The test showed that not all pain area data has normal distribution. The same holds true for RPE and OPL data. The data for total distance from GNSS has normal distribution. To check differences in total distance between every match of the IT, the paired-sample T-Test was used.
The Wilcoxon Signed-rank test was performed to investigate differences in pain area, RPE and OPL between results of RP, 1–2TC and 1–4IT. Data are presented as mean ± standard deviation (SD), unless otherwise stated. Statistical significance for all analyses was accepted at p < 0.05.

3. Results

3.1. Questionnaire—Pain

Every participant reported pain during training and matches, as well as after this type of activity, as shown in Figure 2 and Figure 3. Further, six out of ten players reported that pain limited their performance, and four out of ten players stated that pain does not limit their performance at all. Simultaneously, 9 out of 10 reported that their current pain does not influence their motivation to play and train.

3.2. RPE and OPL

Field players’ recall of typical RPE was 48.4% higher compared to that after the end of IT (4th match) with p = 0.014. There were no significant differences comparing following days of TC and following matches of IT, neither between recall RPE and the end of TC (2nd day TC) nor end of TC (2nd day TC) and end of IT (4th match) (Figure 4).
Declared by filed players, recall OPL was 35.6% lower compared to that after end of TC, p = 0.017. OPL after end of TC was 42.2% higher than after end of IT, p = 0.027. There were no significant differences between OPL after following matches nor comparing recall OPL to this after the end of IT (4th match) (Figure 4).
The data describing GNSS tracking is presented in Table 2. GNSS monitoring of KPIs shows the average total distance covered during a match by a player ranged from 2483.14 ± 583.64 m in match III to 2911.08 ± 828.90 m in match I. There were no significant differences between distances in any match (p value from 0.053 to 0.894). The maximal distance covered by a single player during a match ranged from 3349.30 m in match II to 4310.20 m in match IV. The average active time (which is the time when the player’s velocity was higher than 0 m/s) of 21.2 min represents 30.3% of the average session time of 70.2 min. The average top speed ranged from 5.65 ± 0.44 to 6.32 ± 0.36 m/s. The maximal registered speed reached by a player was 7.73 m/s, which equals 27.83 km/h.

3.3. Pain Area

The average total pain area (sum of the front and back areas in pixels, divided by the number of players) was the highest when players drew digital pain maps describing a recall of typical pain with no significant differences compared to the end of TC, p = 0.959. The total pain area was 42.7% lower at the end of IT compared to the pain area of a recall of typical situation, p = 0.028, but there were no significant differences in comparing the end of IT to the end of TC, p = 0.074. The differences between the following matches of IT and between the following days of the TC were not significant (Figure 5). Digital pain drawings individualized according to the disability of the representative player from the front and back views are presented below in Figure 6 and Figure 7.

4. Discussion

In this exploratory pilot study, all Amputee Football players reported pain in multiple locations in association with training and matches. Most of the AF field players stated that pain limits their performance but not their motivation. The reported pain area increased rapidly due to training and was substantially greater after training than during matches of the international tournament. However, pain area increased progressively following each tournament match across each day, which may be crucial in longer tournaments such as the European or World Championship. These pain patterns align with higher reports of RPE and pain (OPL) which were higher during training camp (360 min total training time) than during the tournament (280 min total playing time). One explanation for this difference is the lower time played and the lower activity time in the tournament compared to the time and activity time of the training sessions. The relationship of pain and injury parameters such as type (acute or overuse), incidence, severity and burden are to be explored and analyzed.
Total distance covered during the following matches of IT did not differ significantly. If AF players were to maintain the average pace of locomotion in a 90-min football match, the distance covered would be over 5000 m, while the distance covered by non-disabled professional footballers during the match was 10,672.79 m ± 347.74 m [25]. The lower distance covered in AF compared to Football is reasoned by a lower time, smaller pitch and other types of locomotion used by the players in AF. However, the above do not prevent AF players reach high velocity (27.83 km/h in this study) and high parameters of acceleration and deceleration, though these comparisons are not fully reasonable in cases of overall load because internal and external load during running and moving on crutches with the same speed probably differ. Similarly, with acceleration, deceleration, changes of direction, jumping, landing and kicking are all caused by different biomechanics of these pairs of similar movements in AF and football.
GKs differ significantly compared to field players in the case of total pain area, OPL and RPE, which may result in significantly lower values of total distance and average HR which are proven by different studies [5]. This is caused mainly by a very small penalty box (10 × 8 m), which results in very low distance covered compared to AF field players, and a small number of defensive situations during a match. Due to the specific role of the AF, GKs and the reduced area of play in our study, GKs were excluded from the GNSS match analysis.
So far, to the authors’ best knowledge, there are few scientific articles describing GPS or GNSS tracking parameters in AF. The results of our study in the means of GNSS load monitoring are comparable with the study of Maehana et al. (2017), but not with the study of Nowak (2020). Our study and Maehana’s study were conducted on National Teams Representations (Polish and Japanese, respectively). The distance covered by high intensity running (HIR-High Intensity Running defined as running speed equal 13 km/h (3.61 m/s) or higher in the study of Maehana et al. was 205.3 ± 100.5 m. Moreover, the mean heart rate (HR) during the match was 176.8 ± 7.9 bpm (96.3% of HRmax). Average RPE was 15 (in 26 point scale) in both of the first half and the second half [5]. Maehana et al., defines load in AF as high—the distance covered over the 50 min of the match was 2984.2 ± 56.1 m [5]. In our study it was 2483.14 ± 583.64 m to 2911.08 ± 828.90 m. These are noticeably higher values comparing to the results of Nowak (2020), who made a study on Polish Ekstraleague teams and reports values of 661.81 ± 389.40 m for the first half and 625.29 ± 305.31 m for the second, which equals 1287.1 m on average during a match [4]. The differences between the results of our study and Nowak’s study can be caused by different match times (50 vs. 40 min respectively) and including GK position into the analysis, which was made by Nowak, and which significantly decreases the average results in the study of Nowak (2020) (in our study GKs’ were excluded from the GNSS analysis). It should be also noted that the requirements of international tournaments may place higher demands on players in relation to league games, which were analyzed by Nowak.
Unfortunately, the lack of standardization of individual speed zones (speed bands) for the AF analysis makes it impossible to compare them between studies. In the work of Simim et al. (2018) and in the study of Nowak (2020), as well as in this study, other speed zones have been defined. According to the authors, it is related to the use of other tools from other manufacturers of GPS or GNSS systems with predefined zones. A similar situation occurs in the interpretation of the distance covered in HIR in the work of Maehana et al., presented as (HIR: ≥13 km/h) [5] in Nowak’s (2020) study; HIR was determined to be above 15.00 km/h [4], and in this study sprint is an effort of locomotion with speed ≥17.1 km/h for at least 1 s.
Nowak’s study analyzed HR response and showed that GKs have significantly lower HRmax and average HR compared to field players and cover significantly less distance than field players [4]. The study of Fujishita et al. showed that experienced players have higher speed than inexperienced [26], and the study of Wieczorek et al. showed that there is no relationship between hand grip strength and sprint effectiveness in AF players [27]. The study of Nowak (2021) showed that there is no statistically significant relationship between upper limb power and max speed [28]. All of the above studies assessed performance parameters, and unlike this present study, aimed to relate them to pain assessment which is considered important in long-term team management, injury risk and many other aspects, such as the psychological.
Kasińska and Tasiemski, in their systematic review of scientific articles about AF published in 2016, point to the need for research about the types and causes of injuries [1], which could lead to decreasing the injury incidence, severity and injury burden. To the authors’ best knowledge, the present study is the first to analyze pain in AF. Pain is often a signal of overload and can in some cases foretell injury. Injury prevention is important in sport, but it seems to be even more essential in sport for the disabled such as AF. Monteiro (2014) and Auricchio (2017) claim that AF can be a way to improve the quality of life of people with amputations [29,30], but study of Tatar et al. (2018) shows overload of the upper limbs and single lower limb in AF [10], and our study shows that this is connected with pain in location of upper and lower limbs. The present study shows that pain monitoring in AF can provide a lot of valuable information for coaches, physiotherapists, doctors, athletes and scientists, letting them improve the players’ performance and minimize the negative consequences in everyday living. Precise GNSS analysis of KPIs can bring crucial data needed in programing the training process in AF. Considering the above research combining GNSS and pain monitoring should be continued, and standardization of the used methods is crucial to make the results of different studies comparable.

4.1. Limitations of the Study

A limitation of this study is the small experimental group and short period of time monitored, as well as lack of specific training load monitoring despite the RPE and OPL. Still, due to the substantial number of collected digital pain maps alone, this study allows us to define further research directions, i.e., to achieve the goal set in this study.

4.2. Further Directions

Proposed further directions for studies:
-
collect more data from training sessions and matches of different players and teams to make the results more reliable, making it possible to assess the influence of the position on the pitch on the parameters of the external load and pain
-
search for the relationship of pain qualitative and quantitative parameters with GPS tracking parameters (impact zone or collisions recorded by accelerometer and collated with video analysis)
-
search for the relationship of qualitative and quantitative pain parameters with injury incidence and injury burden
-
make the first attempt to design a prevention program and test its effectiveness in reducing or eliminating the level of pain
-
make the first attempt to evaluate the influence of physiotherapeutic methods and equipment on decreasing the pain parameters
-
search for the relationship of pain qualitative and quantitative parameters with physical performance in tests and real game situations.

4.3. Practical Application

Use of the GNSS system is needed in monitoring the key performance indicators in team sports of the disabled, similar to able-bodied players.
KPIs of match load are demands specific to a sport discipline, which should be reflected in training load and individualization.
Using the KPIs and pain parameters monitoring may be helpful to determine the time a player should stay on the pitch, presenting the best performance and avoiding consequences in the form of over-normal pain sensations.
Using the data based on the KPIs and pain monitoring, optimal time on the pitch for each player may increase overall performance of the team.
The analysis of subjective parameters such as RPE, OPL and digital pain drawing can provide important knowledge for coaching staff which results in team management being more substantiated and based on evidence.

5. Conclusions

Amputee Football training and playing induce high external loads in field players and are connected with pain. Further studies should precisely assess the relation between pain qualitative and quantitative parameters and key performance indicators. The relationship of pain parameters and injuries needs to be thoroughly analyzed in AF. The standardization of the methods of GNSS monitoring parameters is needed.

Author Contributions

Conceptualization, J.M.; methodology, J.M. and M.N.; software, S.A.B.; validation, J.M. and M.N.; formal analysis, J.M. and P.M.; investigation, J.M., M.N. and Z.K.; resources, J.M., Z.K. and P.M.; data curation, J.M. and M.N.; writing—original draft preparation, J.M.; writing—review and editing, S.A.B., F.M.C., A.K. and J.M.; visualization, J.M.; supervision, A.K., F.M.C. and S.A.B.; project administration, A.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Participants read and signed an informed consent form approved by the Senate Research Ethics Committee (project identification code: 26/2016 approval date: 13 October 2016).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to good practice in sport because the research was made with players from National Team the opponents may use the data against.

Acknowledgments

The authors are very grateful to Ampfutbol Polska, the whole coaching staff and the players of Polish Representation for all support and friendship—“Doobry wieczór Pooolskooo!!!”.

Conflicts of Interest

SAB is a co-developer of the software application Navigate Pain, which was used to collect pain reports. All other authors declare no conflict of interest.

Appendix A. Definitions of Used GNSS Parameters

A sprint effort begins when an athlete surpasses 4.75 m/s for at least 1 s. A sprint effort ends when speed has decreased below 75% of the sprint threshold.
For each sprint, the distance covered is calculated. Sprint total distance is the sum of all distances covered.
For each sprint, the distance covered is calculated. Sprint mean distance is the average of all distances covered.
For each sprint, the duration is calculated. Sprint mean duration is the average of all sprints’ durations.
Inertial Load—A scoring value accounting for the intensity and duration of effort based on accelerometer readings. The score is weighted with an exponentially increasing coefficient. Units—arbitrary unit.
GNSS Load—A scoring value accounting for intensity and duration of efforts based on GNSS tracking parameters. The score is weighted with an exponentially increasing coefficient. Units—arbitrary unit.

Appendix B. GNSS System Technical Details

Technical parameters of the used GPS system Titan2 were as follows: GPS ENGINE (U-Blox M8 Concurrent GNSS LCC Module, TCXO, ROM, SAW, LNA); triple GNSS antenna (GPS + GLONASS + GALILEO -SGGP.18A Series Taoglas), high resolution sampling (10HZ), inertial measurement (1000 HZ accelerometer), battery for 7hr, the use of SBAS—EGNOS (Satellite Based Augmentation Stations) (European Geostationary Navigation Overlay Service) error correction.

Appendix C. Amputee Football Player Pain Questionnaire

Amputee Football Player Pain Questionnaire—translation (original in Polish)
Information about the survey:
The survey is anonymous. Participation in the survey is voluntary, the results will be used only to prepare a research paper, no data will be disclosed to unauthorized persons, you can opt out at any stage of the survey. Please give honest and truthful answers. To best understand the question, read it carefully and slowly, including information in parentheses about how to complete it.
Part I—players characteristics:
  • Age (years): …………
  • Gender: ………………
  • Field position (most often): ……………………………………….
  • Years playing Amputee Football: ……………………………….
  • Indicate the situation regarding your person (choose one of the following, if you choose “other answer” fill in the gap):
    amputation as a result of or as a consequence of an injury
    amputation as a result of another disease
    underdeveloped limb (congenital agenesia)
    other answer: …………………………………………..
Part II: pain in amputee football
  • As an amputee football player, do you feel pain during training or a match (choose one of the following)?
    Never
    Rarely
    Often
    Very often
    Always
  • As an amputee football player, do you feel any pain after finishing training or a match (same or next day) (choose one of the following)?
    Never
    Rarely
    Often
    Very often
    Always
  • Does pain limit your performance during training or matches?
    No, not at all
    Yes, in a small extent
    Yes, in a large extent
    Hard to define
  • How pain influence your motivation to training and matches? (choose one of the following):
    Does not limit motivation
    Limit motivation in small extent
    Limit motivation in large extent
    Hard to define
  • Do you feel pain in area of hips? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of groins? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of knees? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of ankles and feet? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of shoulders? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of elbows? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of wrists? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel pain in area of hands and fingers? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
  • Do you feel phantom feelings? (painless sensations) (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Do you feel phantom pain (painful sensations) of amputated limb?
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Did you experience phantom pain in the initial period (up to 1 year) after amputation? (choose one of the following)
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Did you experience a phantom sensation in the initial period (up to 1 year) after amputation? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Do you suffer from stump pain? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Do you suffer from pain of the underdeveloped limb?
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable
  • Do you experience pain from walking on crutches? (choose one of the following):
    No, never
    Yes, rarely
    Yes, often
    Yes, always or nearly always
    not applicable

References

  1. Kasińska, Z.; Tasiemski, T. Ampfutbol w praktyce i w badaniach naukowych. Postep. Rehabil. 2016, 30, 55–66. [Google Scholar] [CrossRef]
  2. World Amputee Football International 7v7 Rules. 2021. Available online: https://www.worldamputeefootball.org/rules_i.htm (accessed on 22 June 2022).
  3. Simim, M.A.M.; da Mota, G.R.; Marocolo, M.; da Silva, B.V.C.; de Mello, M.T.; Bradley, P.S. The demands of amputee soccer impair muscular endurance and power indices but not match physical performance. Adapt. Phys. Act. Q. 2018, 35, 76–92. [Google Scholar] [CrossRef]
  4. Nowak, A.M. Match performance in Polish amputee soccer extra ligue—A pilot study. Adv. Rehabil. 2020, 34, 16–25. [Google Scholar] [CrossRef]
  5. Maehana, H.; Miyamoto, A.; Koshiyama, K.; Yanagiya, T.T.; Yoshimura, M. Profle of match performance and heart rate response in Japanese amputee soccer players. J. Sports Med. Phys. Fitness 2018, 58, 816–824. [Google Scholar] [CrossRef]
  6. Hootman, J.M.; Dick, R.; Agel, J. Epidemiology of collegiate injuries for 15 sports: Summary and recommendations for injury prevention initiatives. J. Athl. Train. 2007, 42, 311–319. [Google Scholar]
  7. Romero, E.A.S.; Lim, T.; Pérez, J.L.A.; Castaldo, M.; Lozano, P.M.; Villafañe, J.H. Identifying clinical and MRI characteristics associated with quality of life in patients with anterior cruciate ligament injury: Prognostic factors for long-term. Int. J. Environ. Res. Public Health 2021, 18, 2845. [Google Scholar] [CrossRef]
  8. Nowak, A.M.; Molik, B.; Kosmol, A.; Szczepaniak, M.; Marszałek, J. Application of the arm-cranking 30-second Wingate Anaerobic Test (the WAnT) to assess power in amputee football players—A pilot study. Acta Bioeng. Biomech. 2021, 23, 13–23. [Google Scholar] [CrossRef] [PubMed]
  9. Simim, M.A.; Bradley, P.S.; da Silva, B.V.C.; Mendes, E.L.; de Mello, M.T.; Marocolo, M.J.; da Mota, G.R. The quantification of game-induced muscle fatigue in amputee soccer players. J. Sports Med. Phys. Fit. 2017, 57, 766–772. [Google Scholar] [CrossRef]
  10. Tatar, Y.; Gercek, N.; Ramazanoglu, N.; Gulmez, I.; Unzun, S.; Sanli, G.; Karagozoglu, C.; Cotuk, H.B. Load distribution on the foot and lofstrand crutches of amputee football players. Gait Posture 2018, 64, 169–173. [Google Scholar] [CrossRef]
  11. Thomopoulos, S.; Parks, W.C.; Rifkin, D.B.; Derwin, K.A. Mechanisms of tendon injury and repair. J. Orthop. Res. 2015, 33, 832–839. [Google Scholar] [CrossRef] [Green Version]
  12. Patel, H.; Lala, S.; Helfner, B.; Wong, T.T. Tennis overuse injuries in the upper extremity. Skelet. Radiol. 2021, 50, 629–644. [Google Scholar] [CrossRef] [PubMed]
  13. Ehrmann, F.E.; Duncan, C.S.; Sindhusake, D.; Franzsen, W.N.; Greene, D.A. GPS and Injury Prevention in Professional Soccer. Journal of Strength and Conditioning Research. J. Strength Cond. Res. 2016, 30, 360–367. [Google Scholar] [CrossRef]
  14. Nobari, H.; Banoocy, N.K.; Oliveira, R.; Pérez-Gómez, J. Win, Draw, or Lose? Global Positioning System-Based Variables’ Effect on the Match Outcome: A Full-Season Study on an Iranian Professional Soccer Team. Sensors 2021, 21, 5695. [Google Scholar] [CrossRef]
  15. Rico-González, M.; Puche-Ortuño, D.; Clemente, F.M.; Aquino, R. The Most Demanding Exercise in Different Training Tasks in Professional Female Futsal: A Mid-Season Study through Principal Component Analysis. Healthcare 2022, 10, 838. [Google Scholar] [CrossRef]
  16. Boudreau, S.A.; Badsberg, S.; Christensen, S.W.; Egsgaard, L.L. Digital Pain Drawings: Assessing Touch-Screen Technology and 3D Body Schemas. Clin. J. Pain 2016, 32, 139–145. [Google Scholar] [CrossRef] [PubMed]
  17. Muracki, J.; Kumorek, M.; Kisilewicz, A.; Pożarowszczyk, B.; Larsen, D.B.; Kawczyński, A.; Boudreau, S.A. Practical use of the navigate pain application for the assessment of the area, location, and frequency of the pain location in young soccer goalkeepers. J. Hum. Kinet. 2019, 69, 125–135. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Villa, M.G.; Palsson, T.S.; Boudreau, S.A. Spatiotemporal patterns of pain distribution and recall accuracy: A dose-response study. Scand. J. Pain 2022, 22, 154–166. [Google Scholar] [CrossRef]
  19. Borg, G. Perceived exertion as an indicator of somatic stress. Scand. J. Rehabil. Med. 1970, 2, 92–98. [Google Scholar]
  20. Foster, C.; Florhaug, J.A.; Franklin, J.; Gottschall, L.; Hrovatin, L.A.; Parker, S.; Doleshal, P.; Dodge, C. A New Approach to Monitoring Exercise Training. J. Strength Cond. Res. 2001, 15, 109–115. [Google Scholar] [CrossRef]
  21. Simpson, M.J.; Jenkins, D.G.; Scanlan, A.T.; Kelly, V.G. Relationships between external- And internal-workload variables in an elite female netball team and between playing positions. Int. J. Sports Physiol. Perform. 2020, 15, 841–846. [Google Scholar] [CrossRef]
  22. Terziotti, P.; Sim, M.; Polglaze, T. A comparison of displacement and energetic variables between three team sport GPS devices. Int. J. Perform. Anal. Sport 2018, 18, 823–834. [Google Scholar] [CrossRef]
  23. Polglaze, T.; Tan, J.H.Y.; Peeling, P. Placement of team sport GPS devices for reliability assessment. Proc. Inst. Mech. Eng. Part P J. Sport. Eng. Technol. 2022, 236, 143–147. [Google Scholar] [CrossRef]
  24. Malone, J.J.; Lovell, R.; Varley, M.C.; Coutts, A.J. Unpacking the black box: Applications and considerations for using gps devices in sport. Int. J. Sports Physiol. Perform. 2017, 12, 18–26. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Castagna, C.; Varley, M.; Póvoas, S.C.A.; D’Ottavio, S. Evaluation of the match external load in soccer: Methods comparison. Int. J. Sports Physiol. Perform. 2017, 12, 490–495. [Google Scholar] [CrossRef] [PubMed]
  26. Fujishita, H.; Urabe, Y.; Maeda, N.; Komiya, M.; Sakai, S.; Hirata, K.; Sakamitsu, T.; Kimura, H. Biomechanics of single-leg running using lofstrand crutches in amputee soccer. J. Phys. Ther. Sci. 2018, 30, 1483–1487. [Google Scholar] [CrossRef] [Green Version]
  27. Wieczorek, M.; Wilinski, W.; Struzik, A.; Rokita, A. Hand Grip Strength vs. Sprint Effectiveness in Amputee Soccer Players. J. Hum. Kinet. 2015, 48, 133–139. [Google Scholar] [CrossRef] [Green Version]
  28. Nowak, A.M.; Kwapis, A.; Kosmol, A. Assessment of the upper limbs maximum power and the locomotion speed in amputee football players. Motriz. Rev. Educ. Fis. 2021, 27, 1–6. [Google Scholar] [CrossRef]
  29. Monteiro, R.; Pfeifer, L.; Santos, A.; Sousa, N. Soccer Practice and Functional and Social Performance of Men With Lower Limb Amputations. J. Hum. Kinet. 2014, 43, 33. [Google Scholar] [CrossRef] [Green Version]
  30. Auricchio, J.R.; Bernardes, N.; Moreno, M.A. Study of the quality of life in amputee soccer players. Man. Ther. Posturol. Rehabil. J. 2020, 15, 1–5. [Google Scholar] [CrossRef] [Green Version]
Figure 1. The study design. OPL = Overall Pain Level, RPE = Rate of Perceived Exertion.
Figure 1. The study design. OPL = Overall Pain Level, RPE = Rate of Perceived Exertion.
Applsci 12 06978 g001
Figure 2. The frequency of pain across different body areas as a result of training and matches in Amputee Football field players (n = 10).
Figure 2. The frequency of pain across different body areas as a result of training and matches in Amputee Football field players (n = 10).
Applsci 12 06978 g002
Figure 3. The frequency of phantom pain, stump pain and pain identified as coming from moving on crutches to disability in Amputee Football field players. Different n numbers: n = 10 for all field players, n = 4 for players with congenital agenesia, n = 6 for players with lower limb amputation.
Figure 3. The frequency of phantom pain, stump pain and pain identified as coming from moving on crutches to disability in Amputee Football field players. Different n numbers: n = 10 for all field players, n = 4 for players with congenital agenesia, n = 6 for players with lower limb amputation.
Applsci 12 06978 g003
Figure 4. The rate of perceived exertion of field players (RPE—FP) and goalkeepers (RPE—GK) and overall pain level of field players (OPL—FP) and goalkeepers (OPL—GK) during subsequent days of training camp (1TC-1st day and 2TC-2nd day of training camp) and subsequent matches on international tournament (1–4 IT for 1st,2nd,3rd and 4th match, respectively) in Amputee Football compared with RPE and OPL assessed as a recall of typical feelings after match (R).
Figure 4. The rate of perceived exertion of field players (RPE—FP) and goalkeepers (RPE—GK) and overall pain level of field players (OPL—FP) and goalkeepers (OPL—GK) during subsequent days of training camp (1TC-1st day and 2TC-2nd day of training camp) and subsequent matches on international tournament (1–4 IT for 1st,2nd,3rd and 4th match, respectively) in Amputee Football compared with RPE and OPL assessed as a recall of typical feelings after match (R).
Applsci 12 06978 g004
Figure 5. The average total pain area (sum of the areas from the front and back views of the body scheme) of an amputee football field player during training camp and an international tournament. R.P.—a recall of typical pain after a match. Maximal possible value is 396,630 (pixels).
Figure 5. The average total pain area (sum of the areas from the front and back views of the body scheme) of an amputee football field player during training camp and an international tournament. R.P.—a recall of typical pain after a match. Maximal possible value is 396,630 (pixels).
Applsci 12 06978 g005
Figure 6. Digital pain drawings of the front body area of the representative player in different time moments (the same as shown in the GNSS data summary table). RP-recall pain, 1–2TC—following days of training camp, 1–4IT—following days of international tournament. Numbers represent the area of pain in pixels and the percentage of the whole front body area, individualized for the representative player.
Figure 6. Digital pain drawings of the front body area of the representative player in different time moments (the same as shown in the GNSS data summary table). RP-recall pain, 1–2TC—following days of training camp, 1–4IT—following days of international tournament. Numbers represent the area of pain in pixels and the percentage of the whole front body area, individualized for the representative player.
Applsci 12 06978 g006
Figure 7. Digital pain drawings of the back body area of the representative player in different time moments (the same as shown in the GNSS data summary table). RP-recall pain, 1–2TC—following days of training camp, 1–4IT—following days of international tournament. Numbers represent the area of pain in pixels and the percentage of the whole back body area, individualized for the representative player.
Figure 7. Digital pain drawings of the back body area of the representative player in different time moments (the same as shown in the GNSS data summary table). RP-recall pain, 1–2TC—following days of training camp, 1–4IT—following days of international tournament. Numbers represent the area of pain in pixels and the percentage of the whole back body area, individualized for the representative player.
Applsci 12 06978 g007
Table 1. Group characteristics.
Table 1. Group characteristics.
ParameterAll Players
(n = 12)
Field Players
(n = 10)
Goalkeepers
(n = 2)
Stature (m)178.3 ± 6.5177.6 ± 6.9182.0 ± 2.8
Body mass (kg)77.2 ± 8.974.7 ± 7.489.8 ± 3.2
Age (years)29.9 ± 8.730.7 ± 8.826.0 ± 9.9
Training years (years)7.4 ± 2.17.7 ± 2.26.0 ± 0.0
Disability typeAmputation: 862
Congenital agenesia: 440
Amputation height Transfemoral: 2Transhumeral: 2
Transtibial: 4Other: 0
Healthy limb Right lower limb: 5 Right upper limb: 1
Left lower limb: 5Left upper limb: 1
Using LC everyday (%) 40%-
Table 2. The key performance indicators based on the match analysis using GNSS tracking technology (only field players) * indicates which results achieved by representative player—RP were the highest among the entire team; A.U.—arbitrary units.
Table 2. The key performance indicators based on the match analysis using GNSS tracking technology (only field players) * indicates which results achieved by representative player—RP were the highest among the entire team; A.U.—arbitrary units.
ParameterMatch IMatch IIMatch IIIMatch IV
MeasureAVG ± SDMaxRPAVG ± SDMaxRPAVG ± SDMaxRPAVG ± SDMaxRP
Session Time (min)70.6 ± 0,0070.6070.6069.82 ± 0.5670.1070.1070.4 ± 0.0070.4070.4069.8 ± 0.0069.8069.80
Active Time (min)23.28 ± 5.4432.2032.20 *19.76 ± 3.9125.5021.0018.4 ± 4.5126.6017.3023.5 ± 4.6033.7026.90
GNSS Load (A.U.)29.58 ± 13.1352.2052.20 *16.84 ± 3.0821.1021.10 *17.32 ± 8.3934.3017.4026.11 ± 12.1951.7051.70 *
Inertial Load (A.U.)259.38 ± 146.31512.70512.70 *177.92 ± 66.33275.60275.60 *210.07 ± 71.98327.90216.60250.56 ± 97.47421.80421.80 *
Top Speed (m/s)6.32 ± 0.366.776.77 *5.71 ± 0.175.975.685.65 ± 0.446.415.586.09 ± 0.657.737.73 *
Peak Acceleration (m/s2)3.89 ± 0.484.514.51 *3.46 ± 0.394.074.07 *3.82 ± 0.254.224.033.96 ± 0.785.115.11 *
Peak Deceleration (m/s2)4.38 ± 0.435.034.603.54 ± 0.404.013.894.10 ± 0.424.883.964.31 ± 0.705.494.94
Sprint Total Distance (STD) (m)58.77 ± 33.2199.6994.3016.64 ± 7.8430.1912.8930.68 ± 7.5441.260.0044.30 ± 40.75131.54131.54 *
Sprint Mean Distance (m)19.44 ± 4.4724.9223.5812.86 ± 1.3815.1012.8915.34 ± 3.7720.630.0018.84 ± 4.6424.9618.79
Sprint Mean Duration (s)4.19 ± 0.825.304.752.99 ± 0.263.353.103.48 ± 0.654.400.004.11 ± 0.915.354.07
Total Distance (TD) (m)2911.08 ± 828.904127.564127.56 *2637.77 ± 440.563349.302801.282483.14 ± 583.643526.872465.262858.13 ± 736.794310.202990.93
Band1 (1.0–3.0 m/s) (m)1922.93 ± 530.862601.992601.99 *1735.56 ± 358.742293.621732.221596.79 ± 363.862267.171539.041834.62 ± 560.122995.081786.88
Band2 (>3.0–5.0 m/s) (m)503.10 ± 235.83903.10903.10 *310.70 ± 78.15438.17438.17 *335.75 ± 172.04665.73383.54438.05 ± 172.40715.65573.42
Band3 (>5.0–7.0 m/s) (m)47.86 ± 27.9789.4989.49 *10.32 ± 5.0618.918.6213.17 ± 7.3821.024.1238.00 ± 41.64145.67145.67 *
Band4 (>7.0–9.0 m/s) (m)0.00 ± 0.000.000.000.00 ± 0.000.000.000.00 ± 0.000.000.0015.59 ± 0.0015.5915.59 *
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Muracki, J.; Kawczyński, A.; Nowak, M.; Clemente, F.M.; Makar, P.; Kasińska, Z.; Boudreau, S.A. Assessment of Pain and External Load in Amputee Football Using Digital Pain Drawing and GNSS Tracking—A Pilot Study. Appl. Sci. 2022, 12, 6978. https://doi.org/10.3390/app12146978

AMA Style

Muracki J, Kawczyński A, Nowak M, Clemente FM, Makar P, Kasińska Z, Boudreau SA. Assessment of Pain and External Load in Amputee Football Using Digital Pain Drawing and GNSS Tracking—A Pilot Study. Applied Sciences. 2022; 12(14):6978. https://doi.org/10.3390/app12146978

Chicago/Turabian Style

Muracki, Jarosław, Adam Kawczyński, Michał Nowak, Filipe Manuel Clemente, Piotr Makar, Zofia Kasińska, and Shellie Ann Boudreau. 2022. "Assessment of Pain and External Load in Amputee Football Using Digital Pain Drawing and GNSS Tracking—A Pilot Study" Applied Sciences 12, no. 14: 6978. https://doi.org/10.3390/app12146978

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop