Next Article in Journal
Provincial Correctional Service Workers: The Prevalence of Mental Disorders
Next Article in Special Issue
Contextualising the 2019 E-Cigarette Health Scare: Insights from Twitter
Previous Article in Journal
Content Analysis of Electronic Nicotine Delivery System Publications in Core Clinical Journals from 2012 to 2018
Previous Article in Special Issue
A Cost-Minimization Analysis of a Medical Record-based, Store and Forward and Provider-to-provider Telemedicine Compared to Usual Care in Catalonia: More Agile and Efficient, Especially for Users
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Validation of a Short Questionnaire to Assess Healthcare Professionals’ Perceptions of Asynchronous Telemedicine Services: The Catalan Version of the Health Optimum Telemedicine Acceptance Questionnaire

1
Health Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la Salut, 08272 Sant Fruitós de Bages, Spain
2
Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l’Atenció Primària de Salut Jordi Gol i Gurina, 08272 Sant Fruitós de Bages, Spain
3
Department of Economics and Business, Universitat de Vic-Universitat Central de Catalunya, 08500 Vic, Spain
4
Unitat d’anàlisi i Qualitat, Xarxa Sanitària i Social de Santa Tecla, 43003 Tarragona, Spain
5
Centre d’Atenció Primària Sant Joan de Vilatorrada, Gerència Territorial de la Catalunya Central, Institut Català de la Salut, 08250 Sant Joan de Vilatorrada, Spain
6
TIC Salut Social–Generalitat de Catalunya, 08005 Barcelona, Spain
7
CRES&CEXS–Universitat Pompeu Fabra, 08002 Barcelona, Spain
8
Hospital Sant Joan de Déu, Digital Care Research Group, Universitat de Vic-Universitat Central de Catalunya, 08500 Vic, Spain
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(7), 2202; https://doi.org/10.3390/ijerph17072202
Submission received: 14 February 2020 / Revised: 22 March 2020 / Accepted: 24 March 2020 / Published: 25 March 2020
(This article belongs to the Special Issue Social Media and Public Health: Opportunities and Challenges)

Abstract

:
Telemedicine is both effective and able to provide efficient care at a lower cost. It also enjoys a high degree of acceptance among users. The Technology Acceptance Model proposed is based on the two main concepts of ease of use and perceived usefulness and is comprised of three dimensions: the individual context, the technological context and the implementation or organizational context. At present, no short, validated questionnaire exists in Catalonia to evaluate the acceptance of telemedicine services amongst healthcare professionals using a technology acceptance model. This article aims to statistically validate the Catalan version of the EU project Health Optimum telemedicine acceptance questionnaire. The study included the following phases: adaptation and translation of the questionnaire into Catalan and psychometric validation with construct (exploratory factor analysis), consistency (Cronbach’s alpha) and stability (test–retest) analysis. After deleting incomplete responses, calculations were made using 33 participants. The internal consistency measured with the Cronbach’s alpha coefficient was good with an alpha coefficient of 0.84 (95%, CI: 0.79–0.84). The intraclass correlation coefficient was 0.93 (95% CI: 0.852–0.964). The Kaiser–Meyer–Olkin test of sampling showed to be adequate (KMO = 0.818) and the Bartlett test of sphericity was significant (Chi-square 424.188; gl = 28; p < 0.001). The questionnaire had two dimensions which accounted for 61.2% of the total variance: quality and technical difficulties relating to telemedicine. The findings of this study suggest that the validated questionnaire has robust statistical features that make it a good predictive model of healthcare professional’s satisfaction with telemedicine programs.

1. Introduction

Although at present there is no clear consensus as to the economic impact of telemedicine [1,2], recent evidence has shown it can be effective, provide efficient care at a lower cost [3,4,5,6,7] and enjoy a high degree of acceptance among users. Moreover, telemedicine reduces journeys by road and therefore decreases the environmental impact of atmospheric pollutants emitted by vehicles [8]. Published studies have already provided some insights as to the acceptance drivers. Eddy et al. reported high patient satisfaction with teledermatology and amongst physicians, although this satisfaction was higher in primary care doctors than in dermatologists [9]. McKoy et al. used questionnaires to assess the acceptance of a teledermatology service and reported that 82% of users saw it as a valid alternative to face-to-face consultations [10]. In another qualitative study using semi-structured interviews with 32 healthcare professionals, MacNeill et al. showed mixed points of view: while it was broadly welcomed by nursing staff, some primary care physicians were worried that telemedicine could increase their workload and it could potentially undermine their professional autonomy [11]. A comprehensive systematic review recently published by Mounessa et al. reported that patients and healthcare providers were in general highly satisfied with the two types of telemedicine: store-and-forward and real time telemedicine [11]. Whilst all the studies provided valuable inputs to help understand the complex heterogeneous effect of the participants’ acceptance of new healthcare models, it is of the utmost importance that any published evidence uses validated questionnaires to perform reliable and comprehensive evaluations [12].
The Technology Acceptance Model (TAM) proposed by Davis (1989) [11] is based on the two main concepts of ease of use and perceived usefulness, and comprises three dimensions: the individual context, the technological context and the implementation or organizational context. Based on his benchmark, studies have adapted their methodology to create validated questionnaires. For example, Orruño et al. (2011) evaluated teledermatology adoption by healthcare professionals using a modified 33-item version of this model grouped into eight theoretical dimensions. The Cronbach’s alpha for each theoretical variable and internal consistency of the questionnaire reported good results which suggest the proposed questionnaire is a valid tool for assessing acceptance in this setting [13].
In the REgioNs of Europe WorkINg toGether for HEALTH (RENEWING HEALTH) project, Kidholm et al. reviewed the scientific literature to find questionnaires used in European telemedicine projects to assess the stakeholders’ perceptions. One such questionnaire was used in the EU project Health Optimum (Delivery OPTIMisation through telemedicine) [14,15]. This questionnaire for healthcare professionals includes eight general questions irrespective of their medical specialty and focuses on the physicians’ perception of the quality of the telemedicine service, their convenience, technical and other difficulties and potential effects on the health of the patients using the service (see Appendix A). The questionnaire was not validated and although it does not strictly use the TAM, it adopts some of its dimensions. An easy-to-answer questionnaire increases the response rate but no other short validated questionnaire using the TAM to assess healthcare professionals’ perceptions of asynchronous telemedicine services was found.
Whilst the Catalan Ministry of Health is putting a lot of effort into developing telemedicine, the degree of acceptance of these tools has yet to be investigated using validated questionnaires.
It is not possible to deploy telehealth services without first carrying out a thorough assessment of practitioner’s perceptions of their usefulness, a factor which is critical in fostering their deployment in public and private healthcare systems. Brief questionnaires are preferable as a means to improve response rates as studies using long questionnaires based on the TAM have reported low response rates [16,17]. For these reasons, the aim of our study is the statistical validation of the Catalan version of the EU project Health Optimum telemedicine acceptance questionnaire.

2. Methods

2.1. Likert Scale

The original questionnaire used an incomplete Likert scale with an even number of answers to the questions, while it is recommended that questionnaires use an odd number of answers [18]. Furthermore, there were more positive options than negative in the first two questions. We decided to add an extra possible response to the first four questions in order to have an odd number of options and therefore obtain a complete Likert scale.

2.2. Translation into Catalan and Data Collection

To achieve the higher content validity, a translated and back-translated methodology was used [19,20]. The original English version of the questionnaire (Appendix A) was translated into Catalan independently by two authors who are native Catalan speakers and who are both proficient in English. An agreed Catalan version of the questionnaire was obtained following several drafts (Appendix B).
In order to validate the new Catalan version of the Health Optimum questionnaire, the Google Forms tool was used to send it to primary healthcare professionals in the Catalan central region who had used telemedicine services in the past. WhatsApp health professional groups were used to disseminate the questionnaire. Members were asked to answer the questionnaire and to resend it to other potential respondents. The twitter account of the principal investigator (@jvalaball, >10K followers) was also used for further dissemination. Additional information regarding the respondents’ basic characteristics was added to the questionnaire: age, sex and professional role and the kind of telemedicine services available in the Catalan central region which they had used (teledermatology, teleulcers or teleaudiometries). Non health professionals were asked not to answer the questionnaire.
The questionnaire called “Questionnaire to assess healthcare professionals’ perceptions of asynchronous telemedicine services” (Qüestionari per avaluar la percepció dels professionals sanitaris amb els serveis de telemedicina asíncrona) had 8 questions with a complete Likert scale of 5 answers the first 4 questions and 3 answers the last 4 questions. It was first sent at the beginning of April 2018 and it was closed 3 d later. It was resent 2 weeks later, asking participants to answer it again in order to check for consistency. The questionnaire was closed definitively 5 d later. Completing the questionnaire was considered as an indication of consent to participate in the study. The study protocol was approved by the University Institute for Primary Care Research (IDIAP) Jordi Gol Health Care Ethics Committee (Code P16/046).

2.3. Scale Level Descriptive Analysis

Following Argimon et al.’s methodology [21], we have assessed the variability in responses to the questionnaire calculating the average and the standard deviation (SD) and calculated the frequencies to check for floor and ceiling effects [22]. These effects are important as they can influence the validity, reliability and responsiveness of a questionnaire and they are used to check the percentage of participants with very low and very high scores. We have taken this effect to exist when 15% or more of the responses are found in the higher or lower values [23].
We checked the discriminating capacity of the items using the discriminative rate, which compares the responses in the two extreme groups (individuals who have obtained a total score below the 33rd percentile and individuals who have scored above the 66th percentile). Discriminative rates above 0 indicate discriminating capacity of the items [22].

2.4. Internal Consistency

The internal consistency of the Catalan version of the Health Optimum questionnaire was tested using Cronbach’s alpha. This statistical test checks for the degree of common information that share the items of a scale of measurement. It is an average of the variances between the variables that are part of a scale. Values of α above 0.7 indicate an appropriate internal consistency [22].

2.5. Temporal Stability—Reliability

To check for the intra-observer stability of our questionnaire, a test–retest methodology was used. Its reliability was calculated with intraclass correlations (ICC) between the scores at Time 1 and Time 2 at the individual item level. Intraclass correlation coefficients over 0.75 are considered good and over 0.90 excellent [22].

2.6. Factor Analysis

Factor analysis was used to simplify the information given by a correlation matrix to make it more easily interpretable [23]. To check for the appropriate conditions to perform a factor analysis we have used 2 methods: the Kaiser–Meyer–Olkin test and the Bartlett sphericity test. The first ranges from 0 to 1, with a value lower than 0.5 indicating that it is inappropriate to do the analysis. A significant result of the Bartlett sphericity test (p < 0.05) indicates that it is pertinent to make a factor analysis. To check for the number of different dimensions of the questionnaire, 3 different criteria were used: (1) Kaiser rule which selects the number of factors with a value greater than 1; (2) the percentage of explained variance which is determined by the accumulated percentage of variation extracted in each factor (varimax rotation with Kaiser’s normalization was used to simplify the number of dimensions); (3) a scree plot which graphically represents the number of factors or dimensions extracted, we retained the factors or components to the left of the inflection point on the graph [23].
The statistical programs STATA version 15/SE and SPSS v23 (SPSS Inc., Chicago, IL, USA) were used for these statistical analyses. Results were considered significant with p < 0.05.

3. Results

3.1. Test–Retest

Although 212 responses were received, only 37 individuals responded to the questionnaire twice as required. After checking the response times, we found four respondents that although responded twice to the questionnaire, didn’t wait the necessary minimum two weeks period between the responses. After excluding these other four respondents, calculations were made using 33 participants: 24/33 were women (72%) and their average age was 50.9 (SD: 8.87) years. Among them, 24/33 (72%) were family physicians, 2/33 were nurses (6%), 1/33 was a dermatologist (3%) and 4/33 (12%) had other specialties.
The internal consistency measured with the Cronbach’s alpha coefficient showed none of the items significantly altered the consistency of the instrument. The overall alpha coefficient was 0.84 (95%, CI: 0.79–0.84). Table 1 shows the alpha coefficient for each of the eight items in the questionnaire. With respect to temporal stability, the intraclass correlation coefficient value stands at 0.93 (95% CI: 0.852–0.964), thus showing excellent reliability of the test.

3.2. Descriptive Analysis of the Items

All discriminative rates were above zero and, for the first four questions, they were above one. The highest discrimination item was Item 3, “Clinical quality” (discrimination index = 1.40), and the lowest discrimination item was Item 8, “Future use” (0.51). Table 1 describes the average score for each group and the discrimination index of each of the eight items in the questionnaire.
The lowest ceiling effect score was for Item 3 “Clinical quality” and the highest score was for Item 5, “Health effects”. Furthermore, two out of five items were above the ceiling-effect criterion of 15%.

3.3. Exploratory Factor Analysis

The Kaiser–Meyer–Olkin test of sampling was adequate (KMO = 0.818) and the Bartlett test of sphericity was significant (Chi-square 424.188; gl = 28; p < 0.001), indicating that the items were appropriate for a factor analysis. Two factors emerged with an eigenvalue greater than one (Table 2). The factor with questions about the quality of telemedicine technology (Items 1–5) was named Quality. The other factor contained items relating to technical difficulties in telemedicine (Items 6–8) and was named Difficulties. The two constructs together accounted for 61.2% of total variance, all factor loadings being higher than 0.40. Figure 1 shows the scree plot representing the number of dimensions extracted.
These results show that the Catalan version of the Health Optimum questionnaire to assess practitioner’s perceptions of telemedicine tools is statistically robust.

4. Discussion

The validation of the telemedicine questionnaire for healthcare professionals showed good reliability and an acceptable level of validity. Two questions suffered from the ceiling effect which limit the validity, but all the questions had discriminative rates above zero, higher for the first four questions, showing that all the questions had a discriminative capacity. We could increase the response options to the items to help optimize their discriminative power, but this would reduce the comparability of the scores with the original questionnaire.
The internal consistency was good, with Cronbach’s alpha coefficients for the overall questionnaire and Factor 1 above 0.80. The Cronbach’s alpha coefficient was slightly below the acceptable cut-off for Factor 2 (0.67). The low number of items included in the second factor may contribute to this finding. These results are slightly worse than the one reported by Argimon et al. using a similar methodology [24]. The ceiling effect was found in two questions, though this was not measured in the English original version, meaning that we are unable to conclude that it is an intrinsic characteristic of the questionnaire or a weakness in the Catalan translation.
Our results show that the Catalan version of the Health Optimum questionnaire is a robust tool for assessing healthcare professional’s satisfaction with a telemedicine program. All the questions presented a positive discrimination rate, especially Item 3, “Clinical quality”. This means that the questionnaire is clearly able to distinguish the clinical quality of a telemedicine program.
The results show that it was adequate to perform the factor analysis and showed that the questionnaire had two dimensions, which accounts for 61.2% of the total variance: one concerning the quality of the telemedicine technology and another about technical difficulties relating to telemedicine.
The questionnaire will be useful to evaluate healthcare professionals’ perceptions of the growing number of different asynchronous telemedicine programs available in Catalonia as well as allowing comparisons between them to inform which characteristics make them more accepted by professionals [25].

Limitations

This study has several limitations. The pilot questionnaire was distributed to healthcare professionals via WhatsApp healthcare professional groups asking them to only answer if they had used a telemedicine program in the past. As the questionnaire was anonymous we can not verify whether they had in fact used these programs and we are unable to be sure that the questionnaire was not distributed to other individuals who are unrelated to healthcare professions. The number of respondents (33) was small compared to other studies [24], nevertheless this number was suficient to validate the questionnaire.

5. Conclusions

The Catalan version of the Health Optimum telemedicine acceptance questionnaire has been validated with this study showing robust statistical features that make it a quality tool to assess healthcare professional’s satisfaction with telemedicine programs. As the validation was not performed looking at a specific telemedicine program, the validated questionnaire is potentially valid for any telemedicine program.

Author Contributions

Conceptualization, J.V.-A., G.F.M., J.L.G.D. and G.S.V.; Data curation, J.V.-A.; Formal analysis, J.V.-A., G.F.M. and F.L.S.; Investigation, J.V.-A.; Methodology, G.F.M. and J.L.G.D.; Project administration, J.V.-A. and F.G.C.; Resources, X.M.G.; Supervision, J.V.-A. and J.L.G.D.; Validation, F.L.S.; Visualization, A.R.-C.; Writing—original draft, J.L.G.D., G.S.V. and F.G.C.; Writing—review and editing, J.V.-A., G.F.M., X.M.G., A.R.-C. and F.L.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

This study was conducted with the support of the Secretary of Universities and Research of the Department of Business and Knowledge at the Generalitat de Catalunya.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Original EU Project Health Optimum Telemedicine Acceptance Questionnaire

  • 1. How do you rate the overall quality of the telemedicine consultation?
    • Excellent
    • Good
    • Fair
    • Poor
  • 2. How would you rate the technical quality of the telemedicine consultation?
    • Excellent
    • Good
    • Fair
    • Poor
  • 3. How do you rate the quality of care delivered by the telemedicine service when compared to the quality of traditional care?
    • Better
    • About the same
    • Not as good
    • Not sure
  • 4. Were you comfortable during the telemedicine consultation?
    • Yes, very comfortable
    • Yes, somewhat comfortable
    • No, somewhat uncomfortable
    • No, very uncomfortable
  • 5. Do you feel that the telemedicine consultation service may influence the health status of your patients?
    • Improved health
    • No change
    • Negative effects on health
  • 6. Did you experience technical difficulties that might affect the quality of care delivered by the telemedicine service?
    • Not at all
    • Sometimes
    • Often
  • 7. Did you experience organisational or other difficulties that might affect the quality of care delivered by the telemedicine service?
    • Not at all
    • Sometimes
    • Often
  • 8. Would you continue to use the telemedicine service?
    • Yes, in the same way as the service has be deployed
    • Yes, but with improvements
    • No

Appendix B. Catalan Version of the Questionnai

  • 1. Com qualificaries la qualitat global de les consultes de telemedicina?
    • Excel·lent
    • Bona
    • Normal
    • Regular
    • Dolenta
  • 2. Com qualificaries la qualitat tècnica de les consultes de telemedicina?
    • Excel·lent
    • Bona
    • Normal
    • Regular
    • Dolenta
  • 3. Com qualificaries la qualitat assistencial proporcionada pels serveis de telemedicina comparant-la amb la qualitat de l’atenció habitual?
    • Molt millor
    • Millor
    • Igual
    • Pitjor
    • Molt pitjor
  • 4. Et sents còmode durant les consultes de telemedicina?
    • Sí, molt còmode
    • Sí, bastant còmode
    • Ni còmode, ni incòmode
    • No, bastant incòmode
    • No, molt incòmode
  • 5. Creus que els serveis de telemedicina poden influir en l’estat de salut dels teus pacients?
    • Sí, poden millorar la seva salut
    • No, no canvien la seva salut
    • Sí, poden tenir efectes negatius en la seva salut
  • 6. Has experimentat dificultats tècniques que poden haver afectat la qualitat de l’atenció proporcionada pels serveis de telemedicina?
    • No, mai
    • Alguns cops
    • Sovint
  • 7. Has experimentat dificultats de tipus organitzatiu o d’altres dificultats que poden haver afectat la qualitat de l’atenció proporcionada pels serveis de telemedicina?
    • No
    • Algunes vegades
    • Sovint
  • 8. Continuaràs utilitzant els serveis de telemedicina?
    • Sí, igual que ara
    • Sí, però amb millores
    • No

References

  1. NICE. National Institute for Health and Care Excellence Evidence Standards Framework for Digital; NICE: London, UK, 2019. [Google Scholar]
  2. World Health Organization. WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening; WHO: Genev, Switzerland, 2019. [Google Scholar]
  3. Delgoshaei, B.; Mobinizadeh, M.; Mojdekar, R.; Afzal, E.; Arabloo, J.; Mohamadi, E. Telemedicine: A systematic review of economic evaluations. Med. J. Islamic Repub. Iran 2017, 31, 754–761. [Google Scholar] [CrossRef]
  4. Langabeer, J.R.; Champagne-Langabeer, T.; Alqusairi, D.; Kim, J.; Jackson, A.; Persse, D.; Gonzalez, M. Cost-benefit analysis of telehealth in pre-hospital care. J. Telemed. Telecare 2017, 23, 747–751. [Google Scholar] [CrossRef] [PubMed]
  5. Vidal-Alaball, J.; Garcia Domingo, J.L.; Garcia Cuyàs, F.; Mendioroz Peña, J.; Flores Mateo, G.; Deniel Rosanas, J.; Sauch Valmaña, G. A cost savings analysis of asynchronous teledermatology compared to face-to-face dermatology in Catalonia. BMC Health Serv. Res. 2018, 18, 650. [Google Scholar] [CrossRef] [PubMed]
  6. Van Der Heijden, J.P.; De Keizer, N.F.; Bos, J.D.; Spuls, P.I.; Witkamp, L. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br. J. Dermatol. 2011, 165, 1058–1065. [Google Scholar] [CrossRef] [PubMed]
  7. Bashshur, R.L.; Shannon, G.W.; Tejasvi, T.; Kvedar, J.; Gates, M. The Empirical Foundations of Teledermatology: A Review of the Research Evidence. Telemed. Health 2015, 21, 10–27. [Google Scholar] [CrossRef] [PubMed]
  8. Vidal-Alaball, J.; Franch-Parella, J.; Lopez Segui, F.; Garcia Cuyàs, F.; Mendioroz Peña, J. Impact of a Telemedicine Program on the Reduction in the Emission of Atmospheric Pollutants and Journeys by Road. Int. J. Environ. Res. Public Health 2019, 16, 4366. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Eedy, D.J.; Wootton, R. Teledermatology: A review. Br. J. Dermatol. 2001, 4, 696–707. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. McKoy, K.C.; DiGregorio, S.; Stira, L. Asynchronous Teledermatology in an Urban Primary Care Practice. Telemed. J. Health 2004, 10, S-70–S-80. [Google Scholar]
  11. MacNeill, V.; Sanders, C.; Fitzpatrick, R.; Hendy, J.; Barlow, J.; Knapp, M.; Rogers, A.; Bardsley, M.; Newman, S.P. Experiences of front-line health professionals in the delivery of telehealth: A qualitative study. Br. J. Gen. Pract. 2014, 64, 401–407. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Thijssing, L.; Tensen, E.; Jaspers, M. Patient’s Perspective on Quality of Teleconsultation Services. Stud. Health Technol. Inf. 2016, 228, 132–136. [Google Scholar]
  13. Orruño Aguado, E.; Gagnon, M.P.; Asua, J.; Abdeljelil, A. Ben Evaluation of teledermatology adoption by health-care professionals using a modified Technology Acceptance Model. J. Telemed. Telecare 2011 2011, 17, 303–307. [Google Scholar] [CrossRef] [PubMed]
  14. Kidholm, K.; Nielsen, A.D.; Prior, R. REgioNs of Europe WorkINg toGether for HEALTH; European Comission: Brussells, Belgium, 2011. [Google Scholar]
  15. Momentum Health Optimum. Available online: http://www.telemedicine-momentum.eu/health-optimum-dk/ (accessed on 15 January 2020).
  16. Asua, J.; Orruño Aguado, E.; Reviriego, E.; Gagnon, M.P. Healthcare professional acceptance of telemonitoring for chronic care patients in primary care. BMC Med. Inform. Decis. Mak. 2012, 12, 139. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Gagnon, M.P.; Orruño Aguado, E.; Asua, J.; Abdeljelil, A.B.; Emparanza Knör, J. Using a Modified Technology Acceptance Model to Evaluate Healthcare Professionals’ Adoption of a New Telemonitoring System. Telemed. Health 2012, 18, 54–59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Jamieson, S. Likert Scale, in Encyclopædia Britannica. Available online: https://www.britannica.com/topic/Likert-Scale (accessed on 20 March 2018).
  19. Beaton, D.E.; Bombardier, C.; Guillemin, F.; Ferraz, M.B. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000, 25, 3186–3191. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Guillemin, F.; Bombardier, C.; Beaton, D. Cross-cultural adaptation of health-related quality of life measures: Literature review and proposed guidelines. J. Clin. Epidemiol. 1993, 46, 1417–1432. [Google Scholar] [CrossRef]
  21. Argimon-Pallàs, J.M.; Flores Mateo, G.; Jiménez-Villa, J.; Pujol-Ribera, E.; Foz, G.; Bund-Vidiella, M.; Juncosa, S.; Fuentes-Bellido, C.M.; Pérez-Rodríguez, B.; Margalef-Pallarès, F.; et al. Study protocol of psychometric properties of the Spanish translation of a competence test in evidence based practice: The Fresno test. BMC Health Serv. Res. 2009, 9, 37. [Google Scholar] [CrossRef] [Green Version]
  22. Ware, J.E.; Gandek, B. Methods for testing data quality, scaling assumptions, and reliability: The IQOLA Project approach. J. Clin. Epidemiol. 1998, 51, 945–952. [Google Scholar] [CrossRef]
  23. Rattray, J.; Jones, M.C. Essential elements of questionnaire design and development. J. Clin. Nurs. 2007, 16, 234–243. [Google Scholar] [CrossRef] [PubMed]
  24. Argimon-Pallàs, J.M.; Flores Mateo, G.; Jiménez-Villa, J.; Pujol-Ribera, E. Psychometric properties of a test in evidence based practice: The Spanish version of the Fresno test. BMC Med. Educ. 2010, 10, 45. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. TiC Salut i Social. Mapa de Tendencias|TIC Salut Social. Available online: https://ticsalutsocial.cat/es/area/observatorio/ (accessed on 28 February 2020).
Figure 1. Scree plot.
Figure 1. Scree plot.
Ijerph 17 02202 g001
Table 1. Psychometric validation.
Table 1. Psychometric validation.
Descriptive StatisticsItem Parameter EstimatesReliability
Questionnaire’s ItemsMeanStandard DeviationIDI 1Ceiling Effect (%)Floor Effect (%)Item-CorrelationCronbach Alpha
1. Global quality a3.510.851.248.31.40.790.79
2. Technical quality a3.470.871.135.62.80.750.80
3. Clinical quality a3.010.861.402.80.70.730.81
4. Convenience a3.670.891.3616.01.40.750.80
5. Health effects b2.680.610.5475.7-0.560.83
6. Technical difficulties b2.030.560.59--0.670.81
7. Organizational difficulties b2.100.570.60--0.640.82
8. Future use b2.530.510.51--0.540.84
1 IDI: Item Discrimination Index, a Items scored on a 5-point response scale ranging from 1 “very dissatisfied” to 5 “very satisfied”; b Items scored on a 3-point response scale.
Table 2. Exploratory factor analysis (EFA): data on commonalities of items, item loadings in Factor 1 and Factor 2.
Table 2. Exploratory factor analysis (EFA): data on commonalities of items, item loadings in Factor 1 and Factor 2.
Questionnaire ItemsFactor 1Factor 2Communalities (h2)
1. Global quality a0.7120.4300.692
2. Technical quality a0.5260.5710.603
3. Clinical quality a0.7850.2340.671
4. Convenience a0.8270.2010.722
5. Health effects a0.670-0.450
6. Technical difficulties b0.1540.8360.722
7. Organizational difficulties b-0.8700.763
8. Future use b0.2810.4410.274
Eigenvalues3.761.14
Average variance explained (%)47.0414.21
Cronbach’s α reliability0.8300.672
Extraction method, principal component analysis, rotation method, varimax with Kaiser normalization; rotation converged in three iterations. Factors: a Quality, b Difficulties.

Share and Cite

MDPI and ACS Style

Vidal-Alaball, J.; Flores Mateo, G.; Garcia Domingo, J.L.; Marín Gomez, X.; Sauch Valmaña, G.; Ruiz-Comellas, A.; López Seguí, F.; García Cuyàs, F. Validation of a Short Questionnaire to Assess Healthcare Professionals’ Perceptions of Asynchronous Telemedicine Services: The Catalan Version of the Health Optimum Telemedicine Acceptance Questionnaire. Int. J. Environ. Res. Public Health 2020, 17, 2202. https://doi.org/10.3390/ijerph17072202

AMA Style

Vidal-Alaball J, Flores Mateo G, Garcia Domingo JL, Marín Gomez X, Sauch Valmaña G, Ruiz-Comellas A, López Seguí F, García Cuyàs F. Validation of a Short Questionnaire to Assess Healthcare Professionals’ Perceptions of Asynchronous Telemedicine Services: The Catalan Version of the Health Optimum Telemedicine Acceptance Questionnaire. International Journal of Environmental Research and Public Health. 2020; 17(7):2202. https://doi.org/10.3390/ijerph17072202

Chicago/Turabian Style

Vidal-Alaball, Josep, Gemma Flores Mateo, Josep Lluís Garcia Domingo, Xavier Marín Gomez, Glòria Sauch Valmaña, Anna Ruiz-Comellas, Francesc López Seguí, and Francesc García Cuyàs. 2020. "Validation of a Short Questionnaire to Assess Healthcare Professionals’ Perceptions of Asynchronous Telemedicine Services: The Catalan Version of the Health Optimum Telemedicine Acceptance Questionnaire" International Journal of Environmental Research and Public Health 17, no. 7: 2202. https://doi.org/10.3390/ijerph17072202

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop