Evaluation of E-Learning Experience among Health and Allied Health Professions Students during the COVID-19 Pandemic in Slovenia: An Instrument Development and Validation Study
Abstract
:1. Introduction
2. Materials and Methods
2.1. Scale Development Process
2.1.1. Scale Development and Generation of Items
2.1.2. Assessing the Content and Face Validity
2.1.3. Construct Validity Assessment and Reliability of the Scale
2.2. Measuring E-Learning Experience
Participants
2.3. Data Collection
2.4. Data Analysis
2.5. Ethical Considerations
3. Results
4. Discussion
Limitations
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Rouleau, G.; Gagnon, M.-P.; Côté, J.; Payne-Gagnon, J.; Hudson, E.; Bouix-Picasso, J.; Dubois, C.-A. Effects of E-Learning in a Continuing Education Context on Nursing Care: A Review of Systematic Qualitative, Quantitative and Mixed Studies Reviews (Protocol). BMJ Open 2017, 7, e018441. [Google Scholar] [CrossRef] [PubMed]
- Phillips, R.; McNaught, C.; Kennedy, G. Evaluating E-Learning: Guiding Research and Practice. Connecting with E-Learning; Routledge, Taylor & Francis Group: Abingdon, UK, 2011. [Google Scholar]
- Ličen, S. Nursing Education in the Time of COVID-19: What Has It Taught Us? Slov. Nurs. Rev. 2021, 55, 4–6. [Google Scholar] [CrossRef]
- Fidalgo, P.; Thormann, J.; Kulyk, O.; Lencastre, J.A. Students’ Perceptions on Distance Education: A Multinational Study. Int. J. Educ. Technol. High. Educ. 2020, 17, 18. [Google Scholar] [CrossRef]
- Barbu, A.; Popescu, M.A.M.; Moiceanu, G. Perspective of Teachers and Students towards the Education Process during COVID-19 in Romanian Universities. Int. J. Environ. Res. Public Health 2022, 19, 63409. [Google Scholar] [CrossRef] [PubMed]
- Zalat, M.M.; Hamed, M.S.; Bolbol, S.A. The Experiences, Challenges, and Acceptance of e-Learning as a Tool for Teaching during the COVID-19 Pandemic among University Medical Staff. PLoS ONE 2021, 16, e0248758. [Google Scholar] [CrossRef] [PubMed]
- Stevanović, A.; Božić, R.; Radović, S. Higher Education Students’ Experiences and Opinion about Distance Learning during the COVID-19 Pandemic. J. Comput. Assist. Learn. 2021, 37, 1682–1693. [Google Scholar] [CrossRef] [PubMed]
- Koch, L.F. The Nursing Educator’s Role in e-Learning: A Literature Review. Nurse Educ. Today 2014, 34, 1382–1387. [Google Scholar] [CrossRef] [PubMed]
- Lawn, S.; Zhi, X.; Morello, A. An Integrative Review of E-Learning in the Delivery of Self-Management Support Training for Health Professionals. BMC Med. Educ. 2017, 17, 183. [Google Scholar] [CrossRef] [PubMed]
- Khalil, R.; Mansour, A.E.; Fadda, W.A.; Almisnid, K.; Aldamegh, M.; Al-Nafeesah, A.; Alkhalifah, A.; Al-Wutayd, O. The Sudden Transition to Synchronized Online Learning during the COVID-19 Pandemic in Saudi Arabia: A Qualitative Study Exploring Medical Students’ Perspectives. BMC Med. Educ. 2020, 20, 285. [Google Scholar] [CrossRef] [PubMed]
- Al Zahrani, E.M.; Al Naam, Y.A.; AlRabeeah, S.M.; Aldossary, D.N.; Al-Jamea, L.H.; Woodman, A.; Shawaheen, M.; Altiti, O.; Quiambao, J.V.; Arulanantham, Z.J.; et al. E-Learning Experience of the Medical Profession’s College Students during COVID-19 Pandemic in Saudi Arabia. BMC Med. Educ. 2021, 21, 443. [Google Scholar] [CrossRef] [PubMed]
- Almahasees, Z.; Mohsen, K.; Amin, M.O. Faculty’s and Students’ Perceptions of Online Learning During COVID-19. Front. Educ. 2021, 6, 638470. [Google Scholar] [CrossRef]
- Warshawski, S. Academic Self-Efficacy, Resilience and Social Support among First-Year Israeli Nursing Students Learning in Online Environments during COVID-19 Pandemic. Nurse Educ. Today 2022, 110, 105267. [Google Scholar] [CrossRef] [PubMed]
- Coopasami, M.; Knight, S.; Pete, M. E-Learning Readiness amongst Nursing Students at the Durban University of Technology. Health SA Gesondheid 2017, 22, 300–306. [Google Scholar] [CrossRef]
- Al-Alwani, A. Evaluation Criterion for Quality Assessment of E-Learning Content. E-Learn. Digit. Media 2014, 11, 532–542. [Google Scholar] [CrossRef]
- Shah, M.; Nair, S.; Richardson, J. Measuring and Enhancing the Student Experience; Chandos Publishing Elsevier: Amsterdam, The Netherlands, 2017. [Google Scholar]
- Chan, S.L.; Lin, C.C.; Chau, P.H.; Takemura, N.; Fung, J.T.C. Evaluating Online Learning Engagement of Nursing Students. Nurse Educ. Today 2021, 104, 104985. [Google Scholar] [CrossRef]
- Teo, T. Development and Validation of the E-Learning Acceptance Measure (ElAM). Internet High. Educ. 2010, 13, 148–152. [Google Scholar] [CrossRef]
- Kor, P.P.K.; Liu, J.Y.W.; Kwan, R.Y.C. Exploring Nursing Students’ Learning Experiences and Attitudes toward Older Persons in a Gerontological Nursing Course Using Self-Regulated Online Enquiry-Based Learning during the COVID-19 Pandemic: A Mixed-Methods Study. Nurse Educ. Today 2022, 111, 105301. [Google Scholar] [CrossRef]
- Misirli, O.; Ergulec, F. Emergency Remote Teaching during the COVID-19 Pandemic: Parents Experiences and Perspectives. Educ. Inf. Technol. 2021, 26, 6699–6718. [Google Scholar] [CrossRef]
- Kim, S.-H.; Park, S. Influence of Learning Flow and Distance E-Learning Satisfaction on Learning Outcomes and the Moderated Mediation Effect of Social-Evaluative Anxiety in Nursing College Students during the COVID-19 Pandemic: A Cross-Sectional Study. Nurse Educ. Pract. 2021, 56, 103197. [Google Scholar] [CrossRef]
- Kim, S.-Y.; Kim, S.-J.; Lee, S.-H. Effects of Online Learning on Nursing Students in South Korea during COVID-19. Int. J. Environ. Res. Public Health 2021, 18, 68506. [Google Scholar] [CrossRef]
- Savci, C.; Cil Akinci, A.; Keles, F. The Association of Perceived Sociability and Social Intelligence with Loneliness in Online Learning among Nursing Students. Nurse Educ. Today 2022, 109, 105226. [Google Scholar] [CrossRef] [PubMed]
- Kyriazos, T.A.; Stalikas, A. Applied Psychometrics: The Steps of Scale Development and Standardization Process. Psychology 2018, 9, 2531–2560. [Google Scholar] [CrossRef] [Green Version]
- McDonald, E.W.; Boulton, J.L.; Davis, J.L. E-Learning and Nursing Assessment Skills and Knowledge-An Integrative Review. Nurse Educ. Today 2018, 66, 166–174. [Google Scholar] [CrossRef] [PubMed]
- Lindell, M.K.; Brandt, C.J. Assessing Interrater Agreement on the Job Relevance of a Test: A Comparison of CVI, T, RWG(J)}, and R*WG(J)} Indexes. J. Appl. Psychol. 1999, 84, 640–647. [Google Scholar] [CrossRef]
- Polit, D.F.; Beck, C.T. Essentials of Nursing Research: Appraising Evidence for Nursing Practice, 9th ed.; Wolters Kluwer Health: Philadelphia, PA, USA, 2016. [Google Scholar]
- Basham, R.; Jordan, C.; Hoefer, R. Reliability and Validity in Qualitative Research. In The Handbook of Social Work Research Methods; Thyer, B.A., Ed.; Sage: Thousand Oaks, CA, USA, 2009; pp. 51–65. [Google Scholar]
- Field, A. Discovering Statistics Using IBM SPSS Statistics, 4th ed.; SAGE Publications Ltd.: Los Angeles, CA, USA, 2013. [Google Scholar]
- Mundfrom, D.J.; Shaw, D.G.; Ke, T.L. Minimum Sample Size Recommendations for Conducting Factor Analyses. Int. J. Test. 2005, 5, 159–168. [Google Scholar] [CrossRef]
- World Medical Association. World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. JAMA 2013, 310, 2191–2194. [Google Scholar] [CrossRef] [Green Version]
- Koo, T.K.; Li, M.Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J. Chiropr. Med. 2016, 15, 155–163. [Google Scholar] [CrossRef] [Green Version]
- Lakbala, P. Barriers in Implementing E-Learning in Hormozgan University of Medical Sciences. Glob. J. Health Sci. 2016, 8, 83–92. [Google Scholar] [CrossRef] [Green Version]
- Ličen, S. The Use of Information-Communication Technology among Nursing Students during Their Course of Study. Slov. Nurs. Rev. 2013, 47, 236–246. [Google Scholar]
- Gunawan, J.; Marzilli, C.; Aungsuroch, Y. Establishing Appropriate Sample Size for Developing and Validating a Questionnaire in Nursing Research. Belitung Nurs. J. 2021, 7, 356–360. [Google Scholar] [CrossRef]
Characteristic | n | % |
---|---|---|
Gender | ||
Male | 61 | 17.8 |
Female | 281 | 82.2 |
Year of study | ||
1st year | 160 | 46.8 |
2nd year | 98 | 28.7 |
3rd year | 84 | 24.5 |
Study programme | ||
Nursing | 155 | 45.3 |
Physiotherapy | 27 | 7.9 |
Nutritional counselling—dietetics | 98 | 28.7 |
Kinesiology | 62 | 18.1 |
Variables | Factor Loadings | Mean Score (SD) | I-CVI (R, C) | |
---|---|---|---|---|
Factor 1 | It is easier to learn through the e-learning platform. | 0.780 | 2.55 (1.163) | 0.94 |
I am sure that the eClassroom is a very effective learning tool. | 0.771 | 3.36 (1.143) | 0.94 | |
The e-learning platform provides an opportunity to learn more than with the traditional form of education. | 0.766 | 2.58 (1.144) | 1.0 | |
The eClassroom meets all my learning needs. | 0.756 | 2.82 (1.238) | 1.0 | |
eClassroom learning improves my learning success in the subject. | 0.738 | 2.76 (1.069) | 1.0 | |
E-learning provides a long-term opportunity to acquire new knowledge (new techniques, etc.) in an easy way. | 0.730 | 3.51 (1.089) | 1.0 | |
The eClassroom allows me to manage my learning time much more efficiently and complete the assigned tasks much more easily. | 0.718 | 3.30 (1.236) | 1.0 | |
The eClassroom helps me to approach learning in a more systematic way. | 0.699 | 3.21 (1.081) | 1.0 | |
The eClassroom has enabled me to familiarise myself with the subject matter. | 0.693 | 3.09 (1.135) | 1.0 | |
E-learning offers more flexible learning (more freedom than traditional learning). | 0.689 | 3.41 (1.133) | 1.0 | |
Overall, I find the eClassroom a successful learning environment. | 0.677 | 3.52 (1.063) | 0.94 | |
I enjoy participating in an organised course of studies in the eClassroom. | 0.677 | 3.49 (1.159) | 1.0 | |
The eClassroom makes it easier for me to communicate with the teacher and other students. | 0.673 | 2.80 (1.165) | 1.0 | |
The eClassroom as a learning environment improves my learning. | 0.567 | 3.19 (1.074) | 1.0 | |
Factor 2 | The teacher skilfully presents the learning contents through the eClassroom. | 0.798 | 3.85 (0.873) | 1.0 |
The teacher addresses students’ problems and tries to find a suitable solution through communication in the eClassroom. | 0.780 | 3.78 (0.973) | 1.0 | |
The teacher responds quickly to students’ questions through emails provided by the eClassroom. | 0.766 | 3.69 (0.992) | 1.0 | |
The teacher has created a pleasant online learning environment in the eClassroom. | 0.762 | 3.80 (0.888) | 1.0 | |
The teacher is adept at communicating via the eClassroom. | 0.750 | 3.75 (0.906) | 1.0 | |
The teacher frequently updates lecture handouts and promptly fixes any errors that occur in the learning content available through the eClassroom. | 0.745 | 3.80 (0.852) | 1.0 | |
Communication with the instructor via the eClassroom is important and valuable. | 0.615 | 3.80 (0.868) | 1.0 | |
I am aware of the opportunity to send messages and communicate with the teacher and other students via the eClassroom. | 0.592 | 3.86 (0.952) | 1.0 | |
The teacher encourages us to communicate with other students through the interactive tools provided by the eClassroom. | 0.583 | 3.22 (1.067) | 1.0 | |
Communicating with the teacher through the eClassroom is not complicated. | 0.532 | 3.68 (0.977) | 1.0 | |
The teacher clearly presents the assessment criteria to the students through the eClassroom. | 0.498 | 3.43 (1.003) | 1.0 | |
Factor 3 | I can easily navigate through the eClassroom system. | 0.826 | 3.59 (0.866) | 1.0 |
Navigating the eClassroom is very easy. | 0.773 | 3.53 (0.944) | 1.0 | |
I can find the information I need in the eClassroom. | 0.771 | 3.68 (0.836) | 1.0 | |
The software instructions and navigation in the eClassroom are very clear. | 0.624 | 3.68 (0.846) | 1.0 | |
The possibility of technical support in the eClassroom is available. | 0.614 | 3.27 (0.915) | 1.0 | |
The eClassroom supports interactivity between students and the system through chats, forums, discussions etc. | 0.505 | 3.32 (1.005) | 1.0 | |
The graphical user interface of the eClassroom is suitable for e-learning systems (icons, windows, other tools, etc.). | 0.462 | 3.53 (0.888) | 1.0 | |
Factor 4 | The e-learning platform cannot be used without proper guidance. | 0.782 | 3.11 (1.087) | 1.0 |
An explanation of use and operation is required before using the e-learning platform. | 0.753 | 3.59 (1.047) | 1.0 | |
Use of the e-learning platform is time-consuming. | 0.418 | 2.55 (1.065) | 1.0 | |
E-learning is difficult from a technical and organisational point of view. | 0.444 | 3.04 (1.178) | 1.0 | |
The more experienced one is with using a computer, the more inclined one is to use e-learning. | 0.426 | 3.20 (1.048) | 1.0 |
Factors/Subscales | 1 | 2 | 3 | 4 |
---|---|---|---|---|
Effectiveness of learning | - | 0.68 ** | 0.71 ** | −0.39 ** |
Teacher’s role | −0.68 ** | - | 0.80 ** | −0.23 ** |
Delivery and support | 0.71 ** | 0.80 ** | - | 0.32 ** |
Perceived barriers | −0.39 ** | −0.23 ** | 0.32 ** | - |
Factors/Subscales | n | M | SD | Cronbach α Coefficient | 95% CI | p Value |
---|---|---|---|---|---|---|
Effectiveness of learning | 14 | 3.21 | 0.891 | 0.952 | 3.02–3.21 | <0.001 |
Teacher’s role | 11 | 3.82 | 0.707 | 0.920 | 3.62–3.77 | <0.001 |
Delivery and support | 7 | 3.57 | 0.693 | 0.887 | 3.44–3.59 | <0.001 |
Perceived barriers | 5 | 3.20 | 0.692 | 0.636 | 3.02–3.17 | <0.001 |
Entire 3E-Scale | 37 | 3.42 | 0.565 | 0.935 | 3.30–3.42 | <0.001 |
Variable | Factor 1 | Factor 2 | Factor 3 | Factor 4 | 3E-Scale |
---|---|---|---|---|---|
M(SD) | |||||
Test Value/p | |||||
Gender | |||||
Male | 2.79 (0.931) | 3.64 (0.807) | 3.57 (0.797) | 3.00 (0.539) | 3.27 (0.712) |
Female | 3.29 (0.881) | 3.82 (0.685) | 3.71 (0.670) | 3.40 (0.521) | 3.51 (0.632) |
(U)7034.00/ 0.028 | (U)8098.00/ 0.499 | (U)7772.00/ 0.269 | (U)7677.00/ 0.199 | (U)7258.00/ 0.061 | |
Study programme | |||||
Nursing | 3.36 (0.876) | 3.73 (0.649) | 3.71 (0.665) | 3.40 (0.516) | 3.44 (0.624) |
Physiotherapy | 3.11 (0.787) | 3.82 (0.545) | 3.86 (0.424) | 3.20 (0.481) | 3.43 (0.443) |
Nutritional counselling—dietetics | 3.29 (0.892) | 3.82 (0.745) | 3.57 (0.704) | 3.40 (0.548) | 3.51 (0.681) |
Kinesiology | 2.79 (0.907) | 3.82 (0.832) | 3.57 (0.665) | 3.00 (0.492) | 3.35 (0.705) |
(χ2)14.245/ 0.003 | (χ2)3.032/ 0.387 | (χ2)5.631/ 0.131 | (χ2)11.335/ 0.001 | (χ2)6.221/ 0.101 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Prosen, M.; Karnjuš, I.; Ličen, S. Evaluation of E-Learning Experience among Health and Allied Health Professions Students during the COVID-19 Pandemic in Slovenia: An Instrument Development and Validation Study. Int. J. Environ. Res. Public Health 2022, 19, 4777. https://doi.org/10.3390/ijerph19084777
Prosen M, Karnjuš I, Ličen S. Evaluation of E-Learning Experience among Health and Allied Health Professions Students during the COVID-19 Pandemic in Slovenia: An Instrument Development and Validation Study. International Journal of Environmental Research and Public Health. 2022; 19(8):4777. https://doi.org/10.3390/ijerph19084777
Chicago/Turabian StyleProsen, Mirko, Igor Karnjuš, and Sabina Ličen. 2022. "Evaluation of E-Learning Experience among Health and Allied Health Professions Students during the COVID-19 Pandemic in Slovenia: An Instrument Development and Validation Study" International Journal of Environmental Research and Public Health 19, no. 8: 4777. https://doi.org/10.3390/ijerph19084777