Next Article in Journal
Measuring CO2 Emissions in E-Commerce Deliveries: From Empirical Studies to a New Calculation Approach
Previous Article in Journal
Energy Management System for the Optimal Operation of PV Generators in Distribution Systems Using the Antlion Optimizer: A Colombian Urban and Rural Case Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 14
Issue 23
10.3390/su142316074
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Nursing Students’ Perceived Satisfaction with Flipped Learning Experiences: A Mixed-Methods Study

by
María Consuelo Sáiz-Manzanares
1,*,
Celia Carrillo
2,
María del Camino Escolar Llamazares
2,
Sandra Rodríguez Arribas
3 and
Diego Serrano Gómez
1
1
Group DATAHES, Departamento de Ciencias de la Salud, Facultad de Ciencias de la Salud, Universidad of Burgos, Research P° Comendadores s/n, 09001 Burgos, Spain
2
Departamento de Biotecnología y Ciencias de los Alimentos, Facultad de Ciencias, Universidad de Burgos, Pza. Misael Bañuelos s/n, 09001 Burgos, Spain
3
Departamento de Ingeniería Informática, Escuela Politécnica Superior, Campus Río Vena (Edificio A): Avda, Universidad de Burgos, Cantabria s/n, 09006 Burgos, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(23), 16074; https://doi.org/10.3390/su142316074
Submission received: 21 October 2022 / Revised: 26 November 2022 / Accepted: 29 November 2022 / Published: 1 December 2022
Browse Figures
Versions Notes

Abstract

:
Nowadays, technological teaching tools, such as virtual labs, have become essential, especially in nursing degrees. These resources help implement practical learning based on self-regulation and it is important to know how satisfied students are with them. This means it is important to study students’ perceived satisfaction with virtual labs in flipped learning experiences, which was the general objective of the present study. The aims of the study were: (1) to determine whether there were significant differences in nursing students’ perceived levels of satisfaction according to the type of subject or gender; (2) to ascertain what strengths and weaknesses nursing students perceived about using virtual labs; and (3) to determine the kinds of feelings (positive, negative or neutral) nursing students had using virtual labs. A mixed research methodology was applied, with a sample of 222 undergraduate nursing students at Burgos University (Spain). There were significant differences in the satisfaction perceived by students depending on the subject. Students reported medium-high satisfaction with virtual labs, although they did note that digital skills are needed to use them and suggested incorporating intelligent assistants. Virtual labs seem to be effective, although further studies are needed.

1. Introduction

Active methodologies such as flipped learning (FL) experiences facilitate students’ approaches to the object of learning. This type of active methodology is based on learners’ discovery of their own learning, enabling development of critical thinking and autonomous resolution of tasks or problems. These competences are especially important in health science degrees, such as bachelor’s degrees in nursing [1]. Many studies have shown FL to be effective in efficient, consolidated learning [2]. In addition, FL encourages self-regulated learning (SRL) and collaborative work [3,4,5,6]. This conceptual model corresponds to Zimmerman’s [7] model. It identifies three phases: the planning phase (the student or group analyse the task, set goals, and plan); the execution phase (the student or group implement the plan they designed); and the self-reflection phase (the student or group assess how they did and try to explain why they achieved the results they did).
Therefore, using FL increases students’ motivation towards learning, as it is based on practice [7,8]. However, its effectiveness in application may depend on the design of the FL experiences [9]. The use of these experiences has become even more valuable since the beginning of the SARS-CoV-2 pandemic [8,10], which meant that some teaching which was originally offered face-to-face had to be given in blended or online modes. In this context, FL experiences in blended learning environments have proved to be very effective and motivating for students [11,12]. Moreover, using these experiences is related to an increase in student learning outcomes [13,14]. However, recent studies have shown that FL experiences are most effective when they are applied in classes with small numbers of students and are limited in time. These studies encourage further investigation into the causes of those results [15].
There is an idea, which is increasingly reflected in the nursing scientific community, that the design of curricula and syllabuses should have a high load of practical experiences. These should be designed using resources that are highly motivating for students, such as FL activities, virtual labs, gamification, virtual reality (VR) and augmented reality, and simulation [2,16,17,18,19].

2. Literature Review

A review was performed of the state of the art in terms of using flipped learning in teaching health science subjects. In addition, the review looked at the use of virtual labs, understood as a specific example of flipped learning. Subsequently, it explored the machine learning (ML) techniques most commonly used in this field to study students’ perceived satisfaction about these teaching experiences. Finally, review examined the use of mixed methods (quantitative and qualitative) for analysing satisfaction.

2.1. Flipped Learning Experiences in Health Sciences Students

FL is becoming more and more widely used in university teaching environments. One reason for that is study plans incorporating acquisition of the practical competencies needed for 21st century society. Fundamentally, this is because modern society is continually developing and requires ever-improving technological resources in all areas of knowledge. The study by Dong et al. [1] confirmed that students who worked in flipped classrooms together with collaborative work had better academic results than students who did not. FL also strengthened the development of critical thinking skills, although the study did not find any changes in student satisfaction with teaching. Along those lines, the study by Lai et al. [20] concluded that students’ perceived self-efficacy was related to autonomy and behavioural engagement. According to some students, FL experiences entailed more effort and more time working. This was also one conclusion from Sosa Díaz et al. [21], in which some of the university student participants indicated that they preferred traditional teaching methods (individual, non-collaborative work without active participation from the student). The reason they gave was that they were not used to the newer methodology because it had not been used in their pre-university education.
In a study with food science and technology students, Mshayisa and Basitere [22] found that using FL in practical laboratory classes helped students to acquire good skills as long as they used the FL materials (videos and online quizzes) beforehand. When the students did this, their academic performance increased by 60%. In another study, Chan et al. [23] concluded that medical students’ acquisition of clinical skills could be more effective through FL visualization. Learning outcomes were better in students who watched a video demonstration before doing a practical class. The students also indicated that the methodology had been useful to them in acquiring clinical skills. A study by Youhasan et al. [24] concluded that students gave a satisfactory evaluation of the FL methodology, although they indicated that the challenge was in the provision of technological resources. Similarly, in a study with physiotherapy students, Grover and Phadke [25] found that the FL methodology helped to improve academic results.
In nursing students, Chen et al. [26] found that success with FL seemed to depend on students’ experience with this type of methodology and teachers’ use of self-regulated practices. With nephrology students, Yang et al. [27] found that students who participated in FL experiences had better academic results, especially in aspects related to solving clinical cases. Sailsman [28] found that using FL was a challenge for clinical nurses. Successful implementation was through training teachers in how to use it and student engagement. That engagement was based on increasing student motivation. Similarly, successful implementation of FL experiences in nursing seems to be related to how difficult students believe the topic to be. Subjects with a larger conceptual component, such as anatomy and physiology, are where students indicated greater acceptance of FL methodologies [29].
In summary, FL experiences have been shown to be an effective tool in nursing and medicine courses during the COVID-19 pandemic, both in the learning process and in motivation and students’ moods [30,31]. Nonetheless, designing a work plan is fundamental to achieving good results [32]. In addition, there needs to be more research to understand the best contexts for implementing FL experiences [4,33].

2.2. Virtual Labs

The benefits of using virtual labs in undergraduate nursing education include reinforcing scientific knowledge and promoting problem solving and critical thinking [2,16,17]. They help students better understand the nature of science and the attitudes of professionals. Furthermore, recent studies during the COVID-19 pandemic have shown that they demonstrate a similar effectiveness to in-person laboratories [34]. Virtual labs can vary in complexity depending on their design [35], but they are all based on practical activities that are often guided, i.e., self-regulated, remotely by the teacher [36]. In addition, they include different VR resources simulating real environments [18]. The practice is supported by teaching based on training simulation, a very effective teaching method for learning clinical practice [35]. The results from health science students show high user satisfaction, although labs should be tailored to the learning needs of each individual user [18]. However, studies advocate further research in this area, considering this type of resource to be an objective and an opportunity in current teaching [19].

2.3. Use of Machine Learning Techniques in Analysis of Perceived Satisfaction

Active methodologies, such as FL and virtual labs, have an important effect on student and teacher satisfaction because they facilitate student success and engagement with the teaching-learning process [10,37,38]. However, using these methodologies in virtual environments creates huge amounts of data. Machine learning (ML) techniques are currently used to analyse that data. These techniques allow us to measure and analyse the level of student satisfaction with the entire learning process by studying a large amount of data in various types of records [10].
ML can be defined as a part of artificial intelligence (AI) that is based on the application of algorithms to recognise patterns within large datasets and make predictions about that data and/or clusters [39]. In today’s educational context with daily use of e-Learning platforms—producing large amounts of data—ML techniques simplify the organisation and understanding of the data [9,40,41,42]. Sentiment analysis, also known as opinion mining, is one ML technique used in the study of student satisfaction. It facilitates identification of students’ feelings (positive, negative or neutral) from analysis of sentences or phrases using what has been called text mining [43,44]. However, to be effective, this technique requires the use and training of algorithms to achieve accurate results about the polarity of the comments being analysed.
In recent years, data mining techniques—specifically natural language processing (NLP) techniques—have been used for sentiment analysis of user opinion data collected from various different channels (surveys, social networks, etc.) [45]. To be interpreted, open responses have to be processed with ML techniques, such as sentiment analysis [46]. More specifically, in the context of research into learning on virtual platforms, analysing students’ feelings allows us to test the perceived effectiveness in relation to the use of different digital resources [47]. The ultimate aim of sentiment analysis is to determine the emotions behind the words. There are various tools that can be applied for sentiment analysis such as algorithms in Python [48] and in R [49]. Qualitative analysis software can also be used, such as Atlas.ti v. 22 [50], MAXQDA [51], and NVivo [52]. The most recent versions of these programs have incorporated sentiment analysis using artificial intelligence on written texts. The advantage of these programs is that they can be used without having to know programming languages or algorithms.

2.4. Using Mixed-Methods Techniques in Analysis of Perceived Satisfaction

Using active methodologies (FL experiences, virtual labs, etc.) is related to an increase in students’ perceived satisfaction. A large proportion of studies have used mixed methods to check student satisfaction [3,12,13,18,53]. Mixed methods use quantitative and qualitative analysis by combining the strengths of both and compensating for the gaps in each [24]. This is the strength of this research method; its weakness is that it is difficult to work with large samples for both types of analysis, as qualitative analyses require a thoroughness and follow-up that are difficult with large samples. This makes it difficult to generalise results, although it provides a great deal of information on the effectiveness of the resources applied [21].
In summary, innovative teaching practices based on active methodologies have a promising future. However, developing materials, monitoring progress, and data analysis require digital and ML skills, all of which require extensive work on the part of teachers [28].
The use of FL methodologies—included in the design of virtual laboratories aimed at learning practical content in health science subjects, especially in medicine and nursing—is part of the Agenda 2030 objectives [54], specifically objective four which is about Quality Education. In addition, the European Union (EU) is driving the digitalization of education. That includes the use of technological resources (virtual platforms, virtual and augmented reality, etc.), active methodologies (FL, project-based learning, etc.), and skills training for teachers and students. The EU also encourages sustainability of educational resources within the Digital Education Action Plan (2021-2027) [55].
Based on the aforementioned state of the art, the objectives of this study were: (1) to determine whether there were significant differences in nursing students’ levels of perceived satisfaction according to type of subject or gender; (2) to ascertain the strengths and weaknesses nursing students perceived about using virtual labs as support for practical activities and the differences according to the type of subject; (3) to determine the type of feelings (positive, negative and neutral) nursing students had about using virtual labs.
In order to test these objectives, a mixed methods study (quantitative and qualitative study) was performed. The research questions (RQ) were:
Quantitative study
RQ1. Will nursing students’ perceived satisfaction be different with respect to subject?
RQ2. Will nursing students’ perceived satisfaction be different with respect to gender?
Qualitative study
RQ3. What will be the perceived strengths and weaknesses of using virtual labs to support learning?
RQ4. Will there be differences in students’ perceptions of strengths and weaknesses depending on the subjects taken?
RQ5: How will the students feel about the use of virtual labs in the different subjects studied?

3. Materials and Methods

3.1. Participants

The starting point was a population of 268 students enrolled in the Nursing Degree at the University of Burgos. The sample size at a 99% confidence level, with a precision of 3% and a proportion of 5% of the sample size would be 152 students. Adjusting this for losses of 15% gives 179 students. These figures were determined using Fisterra’s formula (https://www.fisterra.com/formacion/metodologia-investigacion/determinacion-tamano-muestral/ last accessed on 14 November 2022). In the present study, non-probabilistic convenience sampling was used and the sample for the study was composed of 222 second- and third-year undergraduate nursing students over two semesters in the 2021–2022 academic year. This meant that the sample size gave a 99% confidence level. The percentage of male students was 7.66% and the percentage of female students was 92.34%. Further specification of the sample is provided in Table 1. This type of sampling was chosen for ethical reasons due to working with people in training, in this case students, in order to ensure equality of opportunities in accessing teaching information.

3.2. Instruments

3.2.1. Virtual Labs

A virtual lab was developed in each of four subjects (Nutrition and Diet Therapy, Research Methodology, Quality Systems and Psychology and Mental Health). The structure of the virtual lab was similar in the four subjects, comprising an activity related to each of the subjects. The labs are open access and can be accessed in the University of Burgos Repository using the links presented in Table 2.
The virtual labs covered practical subject content and were each created following an SRL paradigm. More specifically, the teachers’ recorded voice explained the topic in each virtual lab, highlighting the most important concepts by self-questioning (for example: “What do we need to do?”, “How do we do that?”, “How are we doing?”, and “How did we do?”).

3.2.2. Scale of Perceived Satisfaction with the Virtual Lab (SPSVL) [56]

An ad hoc scale was developed that included 11 closed Likert-type questions measured on a scale of 1 to 5, along with four open-ended questions (see Appendix A). Reliability following Cronbach [57] for the overall SPSVL scale was α = 0.86, and if the item was removed, the range was α = 0.84–0.90. Reliability using McDonald’s [58] Ω = 0.90, and if the item was removed, the interval was Ω = 0.88–0.94.
The instrument was created to include closed and open questions about perceived satisfaction in order to obtain qualitative and quantitative information about students’ satisfaction with their virtual labs. Including open questions allows qualitative analysis of students’ thoughts, and the application of qualitative analytical techniques to those thoughts.

3.2.3. Virtual Learning Environment (VLE)

We used a VLE based on Moodle (Modular Object-Oriented Dynamic Learning Environment) v.3.9 UBUVirtual.

3.3. Procedure

Four virtual labs were developed and applied in four subjects forming part of the nursing degree, two in the first semester and two in the second semester. The labs were about practical work in each subject and were based on the self-regulated instruction of a task. The four virtual labs were created by four teachers who were members of the Teaching Innovation Group “Blended Learning in Health Sciences”, all following the same procedure. First, they chose the most important practical content from each subject. Then, they wrote a script to present the activity which included the objectives and the process. Following that, each teacher recorded a presentation including some discussion of self-regulation. That self-regulation consisted of using questions such as “What do I have to do?” and “What steps do I need to follow to do this task?” Finally, the virtual labs were made available to the students via the virtual learning platform before each practical. They are freely available in the University of Burgos Institutional Repository. Figure 1 shows an outline of the process for creating the virtual labs.
The virtual labs are listed in Table 2.
Figure 2 describes the procedure applied in implementing the virtual labs.
Before starting the work with this methodology, all of the participants in the study were informed about the aim of the study and the data processing, and their written informed consent was obtained. The labs were used over two semesters (each lasting 9 weeks). At the end of the work in the virtual labs, students completed the SPSVL [56] using the UBUVirtual VLE. Students’ responses were voluntary and anonymous. They were encouraged to participate by emphasizing to them that their responses would be useful in improving the virtual labs.

3.4. Research Designs

For the quantitative study, following the classification by Campbell and Stanley [59], a 2 × 2 factorial design was chosen to test hypotheses RQ1 and RQ2. Two independent variables were included, “subject” and “gender”, along with one dependent variable “students’ perceived satisfaction”.
In the qualitative study, a longitudinal descriptive design was applied based on Flick’s [60] classification.

3.5. Data Analysis

To test the study’s quantitative research questions, we performed a two-factor ANCOVA with fixed effects (subject and gender), eta-squared effect value (η2), and Bonferroni test for difference of means. These data were analysed with SPSS v.28 statistical software [61]. To test the qualitative research questions, we performed a correspondence-document analysis, a Sankey analysis, and a sentiment analysis using Atlas.ti v.22 software [53].

3.6. Ethical Considerations

Approval was obtained from the Bioethics Committee of the University of Burgos (No. IO 03/2022). The ethical principles established by the Declaration of Helsinki were followed at all times. Data collection was designed to ensure confidentiality and the anonymity of the participants. Written informed consent was obtained from participants prior to the study.

4. Results

The results of the quantitative and qualitative studies are described below.

4.1. Quantitative Study

To test RQ1 (“Will nursing students’ perceived satisfaction be different with respect to subject?”), a single factor, fixed-effects ANOVA was performed (with subjects as the independent variable) to determine whether there were significant differences in students’ perceived satisfaction—in the SPSVL scale [56]—in the four subjects using virtual labs. Significant differences were found in items 1, 2, 3, 5, 9, 10, and 11. Applying Bonferroni’s test for difference of means between more than two groups, significant differences were found between students in subject 1 compared to the students taking the other subjects, see Table 3.
Nonetheless, it is important to note that the average student satisfaction in all subjects was no lower than 3.88 out of 5 in all cases, indicating high average satisfaction in all subjects.
There were no significant differences in students’ perceived satisfaction with items 4 (referring to the type of feedback given in the lab), 6 (referring to whether the student felt anxiety when performing the lab), 7 (referring to the motivation of the lab for learning) and 8 (referring to perceived ease of use of the labs) (see Table 3).
To test RQ2 (“Will nursing students’ perceived satisfaction be different with respect to gender?”), a single-factor, fixed-effects ANOVA was performed (with the independent variable gender). No significant differences were found between the genders in perceived satisfaction with the virtual labs in any of the SPSVL items [56]. However, it should be noted that there were far fewer men (7.66%) than women (92.34%) taking the course (see Table 4).

4.2. Qualitative Study

To test RQ3 (“What will be the perceived strengths and weaknesses of using virtual labs to support learning?”) and RQ4 (“Will there be differences in students’ perceptions of strengths and weaknesses depending on the subjects taken?”), students’ responses to the four open-ended questions of the SPSVL [56] were analysed. The responses were categorised using Atlas.ti v.22 software [53]. The responses in each category were divided into two groups; positive responses and potential improvements. To differentiate this categorisation within each question, each response was colour coded (see Table 5).
In question 12, 38.6% of responses from students in subject 2 and 49.12% of responses from students in subject 4 said that nothing needed to be added to the virtual labs. Similarly, 100% of the responses from students in subject 1 indicated that it was a simple, useful methodology, although 50% of the responses from students in subjects 2 and 4 indicated that they would like pauses to be added between the videos. Students from all subjects indicated that they would like more explanations in the videos, and students in subjects 2, 3, and 4 noted that these explanations should refer to possible mistakes they might make during practical activities in the virtual lab. Lastly, three responses, from students in subjects 2 and 3, suggested that the virtual labs should take place in classroom time rather than as FL (see Table 6).
None of the students’ responses to question 13—from any of the subjects—indicated that they thought any elements or content should be removed (see Table 7).
The benefits noted by the students in their responses to question 14 included: “helps me understand the concepts”, at similar percentages in all four subjects; “step-by-step guidance on how to perform a task”, particularly from students in subjects 1 and 2; “you can pause the video and watch it again as needed”, particularly from students in subjects 1, 2 and 3; and “the video can be watched at any time”, particularly from students in subjects 1, 2 and 3 (see Table 8).
In their responses to question 15 (see Table 9), students in all subjects indicated that there were no disadvantages to using virtual labs. However, they did report two types of drawbacks: 1) the time needed to watch the videos, particularly from students in subjects 2 and 4; and 2) there was no-one behind the virtual laboratories to give feedback in real time, particularly from students in subjects 1 and 4.
Finally, to test RQ5 (“How will the students feel about the use of virtual labs in the different subjects studied?”), a sentiment analysis was performed using Atlas.ti v.22 [40]. Table 10 shows sentiments identified in the students’ responses from each of the subjects.
Figure 3 shows the sentiment analysis as a Sankey diagram. The predominant sentiment in students’ responses in all subjects was neutral.

5. Discussion

Significant differences in perceived satisfaction with virtual labs were found between students doing different subjects. The subject in which students reported the highest levels of satisfaction had more practically applicable nursing content. This is consistent with the findings from Massey et al. [34]. Nonetheless, the average levels of satisfaction in all of the subjects were medium-high and the students reported that using virtual labs had helped them to better understand the theoretical and practical content of their subjects, which agrees with findings from other studies [2,16,17]. The students also indicated that using virtual labs had helped them to be more engaged with their learning. This backs up the findings from studies by Hew et al. [40], Sáiz-Manzanares et al. [10], and Singh et al. [41]. In addition, the students exhibited interest in using the virtual lab methodology in other subjects, which is consistent with the findings in studies by Dong et al. [1] and Massey et al. [34]. In contrast, we found no differences in perceived satisfaction with respect to the gender variable.
From the qualitative study, firstly we found that using mixed methods in the data analysis allowed us to examine the results via a microanalytical analysis [3,12,13,18,53]. The results of this analysis provided important pointers for improvement in the development of virtual labs. These focus on adding pauses between the steps in the exercises and including more explanations about possible mistakes that students might make. However, in the open-ended responses, all of the students agreed that they would not remove anything. Benefits of virtual labs that the students highlighted include that they help conceptual understanding and that they provide step-by-step guides for practical activities. This is consistent with findings from Zhu et al. [3]. The students also indicated notable aspects for improvement. These were about the time needed to work with the labs and the lack of ability to ask questions in real time. These aspects provide a challenge for future research aimed at enhancing personalisation of virtual labs [18]. It is also worth underscoring the importance of using ML techniques, specifically sentiment analysis [43,44]. The results of this analysis provide important information such as a high percentage of neutral feelings towards using virtual labs. The results motivate further research in this area [19].
Overall, the mean satisfaction perceived by students participating in the FL experiences related to virtual labs was medium-high (3.88 out of 5). However, there were differences between the subjects. These differences centred on theoretical and practical understanding. In addition, no significant differences were found in the feedback noted in the labs, levels of anxiety using the labs, or perceived usability of the labs. We did find differences by subject in the responses to the open questions; students in subjects 2, 3, and 4 wanted more pauses and more references to potential errors students might make, things which students in subject 1 did not raise.
In terms of perceived benefits of labs, the students from the four subjects gave very similar answers. They felt that the virtual labs helped them to better understand concepts. The benefits were related to the step-by-step guides the labs gave for practical tasks. Similarly, none of the students indicated drawbacks in the use of labs, other than indicating that they had to spend time watching the videos before going to the in-person classes. The sentiment analysis confirmed the above. The conclusion is that using virtual labs based on FL methodology is a change in how students learn. That means that the students’ sentiments were not overwhelmingly positive. They did feel the benefits of the methodology for their learning, but they also felt that they needed to make greater efforts in terms of time and conceptual thinking. This might be why we found a higher percentage of neutral, and occasionally negative, feelings than the studies by Lai et al. [20], Sosa Díaz et al. [21], Mshayisa and Basitere [22], Youhasan et al. [24], and Chen et al. [26]. There was also a difference depending on the subject type, between subjects with more clinical components and subjects with less [28,29].

6. Conclusions

In summary, using virtual labs as a teaching support tool for nursing students is a highly promising resource. Below, we examine the theoretical and practical implications of the results, the limitations of the study, potential improvements, and lastly, future lines of research.

6.1. Theoretical Implications of the Study

This study highlights the need to keep working from a framework based on the benefits of using technological tool-based resources such as virtual labs which help students to perform self-regulated learning. The virtual labs created from this paradigm facilitate instruction in planning and modelling of execution and self-reflection in performing these practical activities. The students felt that these elements were a positive help for their performance of these tasks.

6.2. Practical Implications of the Study

At the practical level of teaching health science subjects, virtual labs are a tool that help students to visualize the steps they need to follow to perform practical activities. The students can consult these resources whenever they wish, without a teacher having to be present. This aspect gives the students autonomy and security in their learning process.

6.3. Limitations and Potential Improvements

There are important points for improvement related to the design of the labs, analysis of any computer-related difficulties in using them, and the inclusion of intelligent personal assistants to facilitate real-time feedback for the student.
The study does have limitations. These include the fact that it involved students from a single university, working toward a nursing degree, who were chosen by convenience sampling. That means that although the results might be in line with conclusions from other studies, they need to be taken with caution. Future studies should be with teachers from other universities, and with students doing other health science degrees (medicine, pharmacy, psychology, etc.).

6.4. Future Research

In short, in order to achieve widespread use of virtual labs, students and teachers need more digital training, and this technology needs to be used more in university virtual learning environments. These changes represent a challenge to be addressed by university leaders in collaboration with teaching staff and students in order to build a university based on the implementation of digital advances, a Smart University. All of this means more research in this area.
In addition, future work will examine the characteristics of the course content in different subjects, to determine what types of subjects are most suitable for FL experiences with virtual labs. It will also explore students’ comments while they are using virtual labs in order to understand whether they use self-regulation strategies and if so, which ones. The impact of which year the students are in will also be examined to determine whether it affects the perceived satisfaction with using this methodology. There is still a significant path in this knowledge area to explore and examine more deeply. This is why it is important to continue research on this topic, as the use of FL resources within the implementation of virtual labs is essential in the 21st century, where digitalization has come to stay and expand. Proof of that is the predictions made around the metaverse, where FL experiences applied to virtual labs are suggested as habitual approaches to work in university settings, particularly for health science students.

Author Contributions

Conceptualization, M.C.S.-M., C.C., M.d.C.E.L., S.R.A. and D.S.G.; methodology, M.C.S.-M.; software, M.C.S.-M.; validation, M.C.S.-M., C.C., M.d.C.E.L., S.R.A. and D.S.G.; methodology, M.C.S.-M.; formal analysis, M.C.S.-M., investigation, M.C.S.-M.; resources, M.C.S.-M., C.C., M.d.C.E.L., S.R.A. and D.S.G.; methodology, M.C.S.-M.; data curation, M.C.S.-M., C.C., M.d.C.E.L., S.R.A. and D.S.G.; M.C.S.-M.; writing—original draft preparation, M.C.S.-M.; writing—review and editing, M.C.S.-M., C.C., M.d.C.E.L., S.R.A. and D.S.G.; M.C.S.-M.; visualization, M.C.S.-M.; supervision, M.C.S.-M.; project administration, M.C.S.-M.; funding acquisition, M.C.S.-M., M.d.C.E.L. and S.R.A. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Vice-Rectorate for Teaching Staff from University of Burgos (Spain) (grant numbers TVLab_01, 3/02/21) and the Ministry of Science and Innovation of Spain Projects I+D+i Research Challenges (grant number PID2020-117111RB-I00, 01/01/21).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Universidad de Burgos (protocol code Nº IO 03/2022) for studies involving humans.

Informed Consent Statement

Written informed consent was obtained from all subjects who participated in the study.

Data Availability Statement

The database will be available to interested authors with a signed agreement with the University of Burgos regarding the scientific and responsible use of the data.

Acknowledgments

The authors would like to acknowledge the collaboration of all the nursing students of Universidad de Burgos who participated in this study.

Conflicts of Interest

The authors declare that there are no conflict of interest.

Appendix A

Table
Table A1. Scale of Perceived Satisfaction with the Virtual lab (SPSVL) [56].
Table A1. Scale of Perceived Satisfaction with the Virtual lab (SPSVL) [56].
Item Scale (Open Questions)Rating Scale
12345
1. The virtual lab has helped me to understand the theoretical aspects of the course.12345
2. The virtual lab has helped me to understand the practical aspects of the course.12345
3. The virtual lab has helped me to understand the course content more easily.12345
4. The virtual lab including explanation of wrong answers after each question helps the understanding of the content.12345
5. The virtual lab encourages my participation in the course.12345
6. I feel anxiety or stress while using the virtual lab.12345
7. Virtual labs make the content more entertaining.12345
8. Virtual labs are easy to use.12345
9. I would recommend working with virtual labs to my colleagues.12345
10. I would like to continue working with virtual labs in other subjects.12345
11. My overall satisfaction with the virtual lab is:12345
Closed Questions
12. What elements would you add to or expand in the virtual lab? Why?
13. Which elements of the virtual lab would you remove? Why?
14. Indicate, in your opinion, the benefits of sing virtual labs.
15. Indicate, in your opinion, the drawbacks of virtual labs.

References

  1. Dong, Y.; Yin, H.; Du, S.; Wang, A. The effects of flipped classroom characterized by situational and collaborative learning in a community nursing course: A quasi-experimental design. Nurse Educ. Today 2021, 105, 105037. [Google Scholar] [CrossRef]
  2. Kim, Y.M.; Yoon, Y.S.; Hong, H.C.; Min, A. Effects of a patient safety course using a flipped classroom approach among undergraduate nursing students: A quasi-experimental study. Nurse Educ. Today 2019, 79, 180–187. [Google Scholar] [CrossRef] [PubMed]
  3. Zhu, L.; Lian, Z.; Engström, M. Use of a flipped classroom in ophthalmology courses for nursing, dental and medical students: A quasi-experimental study using a mixed-methods approach. Nurse Educ. Today 2020, 85, 104262. [Google Scholar] [CrossRef]
  4. Chen, C.C. Effects of flipped classroom on learning outcomes and satisfaction: An experiential learning perspective. Sustainability 2021, 13, 9298. [Google Scholar] [CrossRef]
  5. Khojasteh, L.; Hosseini, S.A.; Nasiri, E. The impact of mediated learning on the academic writing performance of medical students in flipped and traditional classrooms: Scaffolding techniques. Res. Pract. Technol. Enhanc. Learn 2021, 16, 17. [Google Scholar] [CrossRef] [PubMed]
  6. Lin, Y.T. Effects of flipped learning approaches on students’ learning performance in software engineering education. Sustainability 2021, 13, 9849. [Google Scholar] [CrossRef]
  7. Zimmerman, B.J. From Cognitive Modeling to Self-Regulation: A Social Cognitive Career Path. Educ. Psychol. 2013, 3, 135–147. [Google Scholar] [CrossRef]
  8. Haftador, A.M.; Shirazi, F.; Mohebbi, Z. Online class or flipped-jigsaw learning? Which one promotes academic motivation during the COVID-19 pandemic? BMC Med. Educ. 2021, 21, 1–8. [Google Scholar] [CrossRef]
  9. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Rodríguez-Díez, J.J.; Rodríguez-Arribas, S.; Díez-Pastor, J.F.; Ji, Y.P. Improve teaching with modalities and collaborative groups in an LMS: An analysis of monitoring using visualisation techniques. J. Comput. High. Educ. 2021, 33, 747–778. [Google Scholar] [CrossRef] [PubMed]
  10. Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Díez-Pastor, J.F.; Rodríguez-Arribas, S.; Marticorena-Sánchez, R.; Ji, Y.P. Monitoring of student learning in learning management systems: An application of educational data mining techniques. Appl. Sci. 2021, 11, 2677. [Google Scholar] [CrossRef]
  11. Leidl, D.M.; Ritchie, L.; Moslemi, N. Blended learning in undergraduate nursing education—A scoping review. Nurse Educ. Today 2020, 86, 104318. [Google Scholar] [CrossRef] [PubMed]
  12. Ropero-Padilla, C.; Rodriguez-Arrastia, M.; Martinez-Ortigosa, A.; Salas-Medina, P.; Folch Ayora, A.; Roman, P.A. Gameful blended-learning experience in nursing: A qualitative focus group study. Nurse Educ. Today 2021, 106, 105109. [Google Scholar] [CrossRef]
  13. Choi, J.Y.; Lee, S.E.; Bae, J.; Kang, S.; Choi, S.; Tate, J.A.; Yang, Y.L. Undergraduate nursing students’ experience of learning respiratory system assessment using flipped classroom: A mixed methods study. Nurse Educ. Today 2021, 98, 104664. [Google Scholar] [CrossRef] [PubMed]
  14. Xu, P.; Chen, Y.; Nie, W.; Wang, Y.; Song, T.; Li, H.; Li, J.; Yi, J.; Zhao, L. The effectiveness of a flipped classroom on the development of Chinese nursing students’ skill competence: A systematic review and meta-analysis. Nurse Educ. Today 2019, 80, 67–77. [Google Scholar] [CrossRef] [PubMed]
  15. Tutal, Ö.; Yazar, T. Flipped classroom improves academic achievement, learning retention and attitude towards course: A meta-analysis. Asia Pacific Educ. Rev. 2021, 22, 655–673. [Google Scholar] [CrossRef]
  16. Downer, T.; Gray, M.; Capper, T. Online learning and teaching approaches used in midwifery programs: A scoping review. Nurse Educ. Today 2021, 103, 104980. [Google Scholar] [CrossRef]
  17. Özbay, Ö.; Çınar, S. Effectiveness of flipped classroom teaching models in nursing education: A systematic review. Nurse Educ. Today 2021, 102, 104922. [Google Scholar] [CrossRef]
  18. Reeves, S.M.; Crippen, K.J.; McCray, E.D. The varied experience of undergraduate students learning chemistry in virtual reality laboratories. Comput. Educ. 2021, 175, 104320. [Google Scholar] [CrossRef]
  19. Cantey, D.S.; Sampson, M.; Vaughn, J.; Blodgett, N.P. Skills, community, and rapport: Prelicensure nursing students in the virtual learning environment. Teach. Learn. Nurs. 2021, 16, 384–388. [Google Scholar] [CrossRef]
  20. Lai, H.-M.; PHsieh, P.-H.; Uden, L.; Yangd, C.-H. A multilevel investigation of factors influencing university students’ behavioral engagement in flipped classrooms. Comput. Educ. 2021, 175, 104318. [Google Scholar] [CrossRef]
  21. Sosa Díaz, M.J.; Guerra Antequera, J.; Cerezo Pizarro, M. Flipped Classroom in the Context of Higher Education: Learning, Satisfaction and Interaction. Educ. Sci. 2021, 11, 416. [Google Scholar] [CrossRef]
  22. Mshayisa, V.V.; Basitere, M. Flipped laboratory classes: Student performance and perceptions in undergraduate food science and technology. J. Food Sci. Educ. 2021, 20, 208–220. [Google Scholar] [CrossRef]
  23. Chan, E.; Botelho, M.G.; Wong, G.T.C. A flipped classroom, same-level peer-assisted learning approach to clinical skill teaching for medical students. PLoS ONE 2021, 16, e0258926. [Google Scholar] [CrossRef] [PubMed]
  24. Youhasan, P.; Chen, Y.; Lyndon, M.; Henning, M.A. Assess the feasibility of flipped classroom pedagogy in undergraduate nursing education in Sri Lanka: A mixed-methods study. PLoS ONE 2021, 16, e0259003. [Google Scholar] [CrossRef] [PubMed]
  25. Grover, P.; Phadke, S. Efficacy of flipped classroom teaching on the cognitive domain in second-year physiotherapy students. Med. J. DY Patil. Vidyapeeth 2021, 14, 642–645. [Google Scholar] [CrossRef]
  26. Chen, L.; Lin, T.; Tang, S. A qualitative exploration of nursing undergraduates’ perceptions towards scaffolding in the flipped classroom of the Fundamental Nursing Practice Course: A qualitative study. BMC Fam. Pract. 2021, 22, 245. [Google Scholar] [CrossRef]
  27. Yang, F.; Lin, W.; Wang, Y. Flipped classroom combined with case-based learning is an effective teaching modality in nephrology clerkship. BMC Med. Educ. 2021, 21, 276. [Google Scholar] [CrossRef]
  28. Sailsman, S. Using the Four Pillars of FLIP to Implement Flipped Learning in an Undergraduate Nursing Research and Evidence-Based Practice Course. Nurs. Educ. Perspect. 2021, 42, E165–E167. [Google Scholar] [CrossRef]
  29. Joseph, M.A.; Roach, E.J.; Natarajan, J.; Karkada, S.; Ramos Cayaban, A.R. Flipped classroom improves Omani nursing students performance and satisfaction in anatomy and physiology. BMC Nurs. 2021, 20, 1. [Google Scholar] [CrossRef]
  30. Aksoy, B.; Pasli Gurdogan, E. Examining effects of the flipped classroom approach on motivation, learning strategies, urinary system knowledge, and urinary catheterization skills of first-year nursing students. Jpn. J. Nurs. Sci. 2022, 19, e12469. [Google Scholar] [CrossRef]
  31. Khodaei, S.; Hasanvand, S.; Gholami, M.; Mokhayeri, Y.; Amini, M. The effect of the online flipped classroom on self-directed learning readiness and metacognitive awareness in nursing students during the COVID-19 pandemic. BMC Nurs. 2022, 21, 22. [Google Scholar] [CrossRef] [PubMed]
  32. Xie, C.; Huang, C.; Yang, X.; Luo, D.; Liu, Z.; Tu, S.; Jie, K.; Xiong, X. Innovations in education of the medical molecular biology curriculum during the COVID-19 pandemic in China. Biochem. Mol. Biol. Educ. 2021, 49, 720–728. [Google Scholar] [CrossRef]
  33. Sáiz-Manzanares, M.C.; Escolar-Llamazares, M.-C.; Arnaiz González, Á. Effectiveness of Blended Learning in Nursing Education. Int. J. Environ. Res. Public Health 2020, 17, 1589. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Massey, A.; Zhang, W.; Amar, A.A. Comparison of non-traditional online and traditional wet-lab experiences in human anatomy and physiology: An innovative approach for pre-licensure nursing education. Nurse Educ. Today 2021, 107, 105149. [Google Scholar] [CrossRef] [PubMed]
  35. Ke, F.; Dai, Z.; Pachman, M.; Yuan, X. Exploring multiuser virtual teaching simulation as an alternative learning environment for student instructors. Instr. Sci. 2021, 49, 831–854. [Google Scholar] [CrossRef]
  36. Fan, Y.; Matcha, W.; Uzir, N.A.; Wang, Q.; Gašević, D. Learning Analytics to Reveal Links Between Learning Design and Self-Regulated Learning. Int. J. Artif. Intell. Educ. 2021, 31, 980–1021. [Google Scholar] [CrossRef]
  37. Hew, K.F.; Hu, X.; Qiao, C.; Tang, Y. What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Comput. Educ. 2020, 145, 103724. [Google Scholar] [CrossRef]
  38. Singh, P.; Narain, B. Student Satisfaction in Educational Organization using Machine Learning. NOVYI MIR 2021, 6, 159–178. [Google Scholar]
  39. Bonaccorso, G. Machine Learning Algorithms; Packt Publishing Ltd.: Birmingham, UK, 2017. [Google Scholar]
  40. Farhat, R.; Mourali, Y.; Jemni, M.; Ezzedine, H. An overview of Machine Learning Technologies and their use in E-learning. In Proceedings of the International Multi-Conference on: “Organization of Knowledge and Advanced Technologies”(OCTA), Tunis, Tunisia, 6–8 February 2020; pp. 1–4. [Google Scholar]
  41. Ho, I.M.K.; Cheong, K.Y.; Weldon, A. Predicting student satisfaction of emergency remote learning in higher education during COVID-19 using machine learning techniques. PLoS ONE 2021, 16, e0249423. [Google Scholar] [CrossRef]
  42. Davies, R.; Allen, G.; Albrecht, C.; Bakir, N.; Ball, N. Using educational data mining to identify and analyze student learning strategies in an online flipped classroom. Educ. Sci. 2021, 11, 668. [Google Scholar] [CrossRef]
  43. Cunningham-Nelson, S.; Baktashmotlagh, M.; Boles, W. Visualizing Student Opinion Through Text Analysis. IEEE Trans. Educ. 2019, 62, 305–311. [Google Scholar] [CrossRef]
  44. Wei, Q.; Shi, X.; Li, Q.; Chen, G. Enhancing Customer Satisfaction Analysis with a Machine Learning Approach: From a Perspective of Matching Customer Comment and Agent Note. In Proceedings of the 53rd Hawaii International Conference on System Sciences, Maui, HI, USA, 7–10 January 2020. [Google Scholar]
  45. Sharma, N.; Mangla, M.; Mohanty, S.N. Supervised Learning Techniques for Sentiment Analysis. In Emerging Technologies in Data Mining and Information Security; Lecture Notes in Networks and Systems; Dutta, P., Chakrabarti, S., Bhattacharya, A., Dutta, S., Shahnaz, C., Eds.; Springer: Singapore, 2023; Volume 490, pp. 423–434. [Google Scholar] [CrossRef]
  46. Gupta, N.; Bhargav, R. Sentiment Analysis in Airlines Industry Using Machine Learning Techniques. In Emerging Technologies in Data Mining and Information Security; Lecture Notes in Networks and Systems; Dutta, P., Chakrabarti, S., Bhattacharya, A., Dutta, S., Shahnaz, C., Eds.; Springer: Singapore, 2023; Volume 490, pp. 189–201. [Google Scholar] [CrossRef]
  47. Rajesh, P.; Akila, D. Sentimental Analysis on E-learning Videos for Learner’s Opinion Using Machine Learning Methodology—Support Vector Machine. In Cyber Technologies and Emerging Sciences; Lecture Notes in Networks and Systems; Maurya, S., Peddoju, S.K., Ahmad, B., Chihi, I., Eds.; Springer: Singapore, 2023; Volume 467, pp. 191–200. [Google Scholar] [CrossRef]
  48. Chamorro-Atalaya, O.; Arce-Santillan, D.; Morales-Romero, G.; Ramos-Salazar, P.; León-Velarde, C.; Auqui-Ramos, E.; Levano-Stella, M. Sentiment analysis through twitter as a mechanism for assessing university satisfaction. Indones. J. Electr. Eng. Comput. Sci. 2022, 28, 430–440. [Google Scholar] [CrossRef]
  49. Villegas-Ch, W.; Molina, S.; Janón, V.D.; Montalvo, E.; Mera-Navarrete, A. Proposal of a Method for the Analysis of Sentiments in Social Networks with the Use of R. Informatics 2022, 9, 63. [Google Scholar] [CrossRef]
  50. ATLAS.ti Corp. Software Package Qualitative Data Analysis; Version 22; Atlas.ti; ATLAS.ti Corp: Berlin, Germany, 2022. [Google Scholar]
  51. VERBI Software. MAXQDA 2022. Available online: https://www.maxqda.com/ (accessed on 28 November 2022).
  52. NVivo Software. Timberlake Consulting 2022, Software. Available online: https://nvivo-spain.com/# (accessed on 28 November 2022).
  53. Shorey, S.; Ang, E.; Ng, E.D.; Yap, J.; Lau, L.S.T.; Chui, C.K. Communication skills training using virtual reality: A descriptive qualitative study. Nurse Educ. Today 2020, 94, 104592. [Google Scholar] [CrossRef]
  54. 2030 Agenda for Sustainable Development and the SDGs. Available online: https://ec.europa.eu/environment/sustainabledevelopment/SDGs/index_en.htm (accessed on 9 August 2020).
  55. European Union. Digital Education Action Plan (2021–2027). Available online: https://education.ec.europa.eu/focus-topics/digital-education/action-plan (accessed on 28 November 2022).
  56. Sáiz-Manzanares, M.C.; (University of Burgos, Burgos, Spain). Scale of Perceived Satisfaction with the Virtual lab (SPSV). 2021; (Unpublished document). [Google Scholar]
  57. Cronbach, L.J. Coefficient alpha and the internal structure of test. Psychometrika 1951, 16, 297–334. [Google Scholar] [CrossRef]
  58. McDonald, R.P. Test Theory: A Unified Treatment; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1999. [Google Scholar]
  59. Campbell, D.; Stanley, J. Experimental Designs in Social Research; [Experimental and Quasi-Experimental Designs for Research]; Amorrortu: Buenos Aires, Argentina, 1991. [Google Scholar]
  60. Flick, U. El Diseño de Investigación Cualitativa; [Designing Qualitative Research]; Morata: Madrid, Spain, 2015. [Google Scholar]
  61. IBM Corp. SPSS Statistical Package for the Social Sciences (SPSS), Version 28; IBM Corp: Armonk, NY, USA, 2022. [Google Scholar]
Sustainability 14 16074 g001 550
Figure 1. Steps for creating the virtual labs.
Figure 1. Steps for creating the virtual labs.
Sustainability 14 16074 g001
Sustainability 14 16074 g002 550
Figure 2. Procedure applied in implementing the virtual labs.
Figure 2. Procedure applied in implementing the virtual labs.
Sustainability 14 16074 g002
Sustainability 14 16074 g003 550
Figure 3. Sankey Diagram of Sentiment Analysis. Note. Subject 1 = Nutrition and Diet Therapy; Subject 2 = Research Methodology; Subject 3 = Quality Systems in Nursing; Subject 4 = Psychology and Health.
Figure 3. Sankey Diagram of Sentiment Analysis. Note. Subject 1 = Nutrition and Diet Therapy; Subject 2 = Research Methodology; Subject 3 = Quality Systems in Nursing; Subject 4 = Psychology and Health.
Sustainability 14 16074 g003
Table
Table 1. Descriptive sample data.
Table 1. Descriptive sample data.
SubjectDescriptive Data
WomenMen
Nn%Age Rangen%Age Range
18–2526–3536–45 18–2526–3536–45
1645890.6391.35%3.45%5.17%69.3866.67%-33.33%
2666395.4595.24%1.59%3.18%34.55100%--
3252184.0095.24%4.76%-416.00100%--
4676292.5488.71%6.45%4.84%57.46%100%--
Note. 1 = Nutrition and Diet Therapy; 2 = Research Methodology; 3 = Quality Systems; 4 = Psychology and Mental Health.
Table
Table 2. Link to the virtual labs used in the subjects.
Table 2. Link to the virtual labs used in the subjects.
SubjectLink to the Virtual Lab
1Nutrition and Diet Therapyhttps://bit.ly/3CJIxW4 (accessed on 28 November 2022)
2Research Methodologyhttps://bit.ly/3CkDRVl (accessed on 28 November 2022)
3Quality Systems in Nursinghttps://bit.ly/3RRtgqy (accessed on 28 November 2022)
4Psychology and Healthhttps://bit.ly/3RTdWJX (accessed on 28 November 2022)
Table
Table 3. One-factor ANOVA with fixed effects (type of subject) and eta-squared effect value.
Table 3. One-factor ANOVA with fixed effects (type of subject) and eta-squared effect value.
ItemSubject 1
n = 64		Subject 2
n = 66		Subject 3
n = 25		Subject 4
n = 67		 Bonferroni Test
M(SD)M(SD)M(SD)M(SD)dfFpη2DM
S1 vs. S2		DM
S1 vs. S3		DM
S1 vs. S4
1. The virtual lab has helped me to understand the theoretical aspects of the course.4.63(0.79)3.85(0.93)3.84(1.07)4.06(0.81)(3,221)5.430.01 *0.070.77 (p = 0.00 *)0.79 (p = 0.001 *)0.57 (p = 0.002 *)
2. The virtual lab has helped me to understand the practical aspects of the course.4.81(0.43)4.20(0.80)4.16(0.99)4.08(0.82)(3,221)7.430.01 *0.090.62 (p = 0.00 *)0.65 (p = 0.001 *)0.74 (p = 0.002 *)
3. The virtual lab has helped me to understand the contents more easily.4.72(0.60)4.17(0.97)3.88(1.15)4.10(0.78)(3,221)4.800.003 *0.060.55 (p = 0.001 *)0.64 (p = 0.04 *)0.61 (p = 0.000 *)
4. The virtual lab including explanations of wrong answers after each question helps the understanding of the content.4.04(0.79)4.18(0.99)4.04(1.06)3.96(1.08)(3,221)0.600.610.08---
5. The virtual lab encourages my participation in the course.4.36(0.81)4.33(1.15)3.72(1.17)3.67(1.11)(3,221)3.460.02 *0.050.56 (p = 0.01 *)0.64 (p = 0.04 *)0.69 (p = 0.001 *)
6. I feel anxiety or stress while using the virtual lab.1.67(1.05)2.17(1.31)2.44(1.42)2.12(1.25)(3,221)0.180.140.03---
7. Virtual labs make the content more entertaining.4.27(0.86)4.02(0.81)4.28(0.98)3.78(1.12)(3,221)1.160.330.02---
8. Virtual labs are easy to use.4.70(0.52)4.21(0.83)3.96(1.14)4.43(0.63)(3,221)2.640.050.04---
9. I would recommend working with virtual labs to my colleagues.4.75(0.47)4.08(1.01)4.04(0.98)3.99(0.99)(3,221)2.900.04 *0.040.67 (p = 0.00 *)0.71 (p = 0.005 *)0.76 (p = 0.000 *)
10. I would like to continue working with virtual labs in other subjects.4.63(0.68)3.74(1.18)4.28(0.89)3.85(1.05)(3,221)2.950.03 *0.040.88 (p = 0.00 *)-0.77 (p = 0.000 *)
11. My overall satisfaction with the virtual lab is:4.77(0.43)4.02(0.75)4.08(1.04)4.03(0.94)(3,221)4.820.00 *0.060.75 (p = 0.00 *)0.69 (p = 0.002 *)0.74 (p = 0.000 *)
Note. * p < 0.05; M = mean; SD = standard deviation; df = degrees of freedom; η2 = eta-squared; DM = Mean Difference; S1 = Subject 1; S2 = Subject 2; S3 = Subject 3; S4 = Subject 4.
Table
Table 4. One-factor fixed effects ANOVA (gender) and eta-squared effect value.
Table 4. One-factor fixed effects ANOVA (gender) and eta-squared effect value.
Item Male
n = 18		Female
n = 204
M(SD)M(SD)dfFpη2
1. The virtual lab has helped me to understand the theoretical aspects of the course.4.06(1.60)4.14(0.91)(1,221)0.300.590.01
2. The virtual lab has helped me to understand the practical aspects of the course.4.17(1.04)4.35(0.78)(1,221)0.660.420.003
3. The virtual lab has helped me to understand the contents more easily.4.28(1.02)4.28(0.87)(1,221)0.080.770.000
4. The virtual lab including explanation of wrong answers after each question helps the understanding of the content.4.12(0.76)4.05(0.96)(1,221)0.180.670.001
5. The virtual lab encourages my participation in the course.3.83(1.29)3.92(1.01)(1,221)0.030.870.000
6. I feel anxiety or stress while using the virtual lab.2.33(1.)412.02(1.24)(1,221)0.310.580.001
7. Virtual labs make the content more entertaining.4.17(0.99)4.03(0.96)(1,221)0.280.600.001
8. Virtual labs are easy to use.4.56(0.62)4.38(0.79)(1,221)1.090.300.005
9. I would recommend working with virtual labs to my colleagues.4.39(0.98)4.23(0.94)(1,221)0.940.330.004
10. I would like to continue working with virtual labs in other subjects.4.33(1.08)4.07(1.05)(1,221)0.560.460.003
11. My overall satisfaction with the virtual lab is:4.33(1.03)4.24(0.83)(1,221)0.270.610.001
Note. M = mean; SD = standard deviation; df = degrees of freedom; η2 = eta-squared.
Table
Table 5. Colour-coded categorisation of responses (positive vs. improvable) to the open-ended questions.
Table 5. Colour-coded categorisation of responses (positive vs. improvable) to the open-ended questions.
QuestionPositive ResponseImprovable Response
12. What elements would you add to or expand in the virtual lab? Why?YellowDark blue
13. Which elements of the virtual lab would you remove? Why?Light green Red
14. What do you think are the benefits of using virtual labs?Dark green-
15. What do you think are the drawbacks of virtual labs?RedTurquoise
Table
Table 6. Categorisation of responses to question 12.
Table 6. Categorisation of responses to question 12.
Categorised ResponsesSubject 1%Subject 2%Subject 3%Subject 4%Total
Sustainability 14 16074 i001 Insert: I wouldn’t add anything it was all correct.002238.6712.282849.1257
Sustainability 14 16074 i001 Insert: It is a useful, simple and practical method to use.1110000000011
Sustainability 14 16074 i002 Insert: Add more pauses between steps.00250002504
Sustainability 14 16074 i002 Insert: More explanatory videos.312624124841625
Sustainability 14 16074 i002 Insert: Have the virtual lab in the classroom.00266.66133.33003
Sustainability 14 16074 i002 Insert: More explanations of possible errors.001168.7516.2542516
Table
Table 7. Categorisation of responses to question 13.
Table 7. Categorisation of responses to question 13.
Categorised ResponsesSubject 1%Subject 2%Subject 3%Subject 4%Total
Sustainability 14 16074 i003 Remove: I would not remove anything.5632.752112.283721.645733.33171
Table
Table 8. Categorisation of responses to question 14.
Table 8. Categorisation of responses to question 14.
Categorised ResponsesSubject 1%Subject 2%Subject 3%Subject 4%Total
Sustainability 14 16074 i003 Advantages: Helps me understand the concepts.3825.685335.812315.543422.97148
Sustainability 14 16074 i003 Advantages: Step-by-step guidance on how to perform a task.2748.212035.71610.7135.3656
Sustainability 14 16074 i003 Advantages: You can pause the video and watch it again as needed.3152.541525.4223.391118.6459
Sustainability 14 16074 i003 Advantages: The video can be watched at any time.3041.11723.2968.222027.473
Table
Table 9. Categorisation of responses to question 15.
Table 9. Categorisation of responses to question 15.
Categorised ResponsesSubject 1%Subject 2%Subject 3%Subject 4%Total
Sustainability 14 16074 i004 Drawbacks: I have not found any drawbacks.6448.124231.58139.771410.53133
Sustainability 14 16074 i005 Drawbacks: It takes time to watch the videos.912.162635.1479.463243.2474
Sustainability 14 16074 i005 Drawbacks: There is no-one to ask if I don’t understand something. 926.47926.47514.711132.3534
Table
Table 10. Sentiment analysis by subjects.
Table 10. Sentiment analysis by subjects.
Sentiments Analysed in the ResponsesSubject 1%Subject 2%Subject 3%Subject 4%Total
Sustainability 14 16074 i006 Feeling: Negative7325.526723.438630.076020.98286
Sustainability 14 16074 i007 Feeling: Neutral18023.8419025.1718324.2420226.75755
Sustainability 14 16074 i008 Feeling: Positive1940.431531.9136.381021.2847
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sáiz-Manzanares, M.C.; Carrillo, C.; Llamazares, M.d.C.E.; Arribas, S.R.; Gómez, D.S. Nursing Students’ Perceived Satisfaction with Flipped Learning Experiences: A Mixed-Methods Study. Sustainability 2022, 14, 16074. https://doi.org/10.3390/su142316074

AMA Style

Sáiz-Manzanares MC, Carrillo C, Llamazares MdCE, Arribas SR, Gómez DS. Nursing Students’ Perceived Satisfaction with Flipped Learning Experiences: A Mixed-Methods Study. Sustainability. 2022; 14(23):16074. https://doi.org/10.3390/su142316074

Chicago/Turabian Style

Sáiz-Manzanares, María Consuelo, Celia Carrillo, María del Camino Escolar Llamazares, Sandra Rodríguez Arribas, and Diego Serrano Gómez. 2022. "Nursing Students’ Perceived Satisfaction with Flipped Learning Experiences: A Mixed-Methods Study" Sustainability 14, no. 23: 16074. https://doi.org/10.3390/su142316074

APA Style

Sáiz-Manzanares, M. C., Carrillo, C., Llamazares, M. d. C. E., Arribas, S. R., & Gómez, D. S. (2022). Nursing Students’ Perceived Satisfaction with Flipped Learning Experiences: A Mixed-Methods Study. Sustainability, 14(23), 16074. https://doi.org/10.3390/su142316074

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop