*2.3. Literature Analysis*

All the 10 selected studies were quantitative studies. The features of each study were organized in a table by four researchers, and the assessments were reviewed among them. The following features of the studies were analyzed: author and publication year, institutional review board (IRB) review, course subject, pedagogical features (strategy, intervention period), participant features (sample size, inclusion criteria), measured variables (instruments, reliability and validity), and outcome variables (Table 1).

*Educ. Sci.* **2021**, *11*, 9


*Educ. Sci.* **2021**, *11*, 9


#### **3. Results**

#### *3.1. General Features of the Selected Studies*

The 10 selected studies were published between 2017 and 2020. The study design used to examine the effects of flipped learning was either pre-test–post-test or quasi-experimental design, and all studies were conducted on nursing students. The mean intervention period was six weeks and six days, and the mean number of participants in the flipped learning group was 91. Six studies presented evidence for determining the sample size for flipped learning (study no. 1, 3, 4, 7, 9, and 10). Eight studies were either approved by an IRB or mentioned obtaining consent from students undergoing flipped learning. Although flipped learning was generally applied to practicum courses, it was also sometimes used in theoretical courses such as fundamentals of nursing, health assessment, health education and methodology, and geriatric nursing.

#### *3.2. Variables for Flipped Learning Outcomes and Results*

The most common variable used to measure the outcomes of flipped learning was satisfaction, namely, satisfaction with learning, the course, or the teaching method (*n* = 5). This was followed by self-directed learning ability (*n* = 4), knowledge (*n* = 3), and critical thinking, problem-solving skills, information literacy, and academic achievement (*n* = 2 each). Other variables included class experience, level of communication anxiety, intent to continue learning, clinical performance, self-efficacy, learning accomplishment, learning attitude, learning motivation, class participation, undergraduate competencies, academic performance, and usefulness. The studies generally reported that flipped learning had a positive impact on the measured variables (Table 1).

#### *3.3. Flipped Learning Teaching Strategies*

To compare the teaching strategies used in studies that examined the effects of flipped learning, the activities were classified into pre-class, in-class, and post-class activities for analysis. Pre-class activities included team-based learning, video-based learning, e-learning courses, individual study, student question formation, watch-summary-question (WSQ) journal, skill practicing, and self-evaluation journal. In-class activities included team-based discussion and cooperative learning, team-based presentation, instructor feedback, handson training, pre-learning readiness assessment, scenario-based application, quiz on the e-learning materials, nursing diagnosis training, discussion on student-formed questions, instructor-led training (discussion, simulation, case study, conference), and 1:1 evaluation and guidance by the instructor. Post-class activities included review of key contents, peer evaluation, post-class evaluation and survey, Q&A, post-class self-study, self-evaluation journal, instructor feedback, application of case study, and team-based answer review and sharing (Table 2).


### **Table 2.** Major features of flipped learning strategies.

#### **4. Discussion**

This study systematically reviewed Korean studies on flipped learning in allied health majors published within the past three years and based on the results, identified the latest trends in flipped learning research. Although we attempted to include studies on flipped learning in various health majors, all 10 selected studies were conducted on nursing students. As per the SIGN checklist for cohort studies, only studies that extracted both a control group and experimental group from the source population were selected.

This led to the exclusion of a number of studies that did not include a comparator (C). These excluded studies were conducted on students majoring in radiology, emergency medicine, occupational therapy (*n* = 2 each), and physical therapy, dental hygiene, and dental technology (*n* = 1 each). This suggests that flipped learning is less frequently applied in other health majors compared to nursing, necessitating an expanded application of flipped learning in more health professions courses. A pre-post analysis with a single group is limited in comparing the effects of flipped learning with that of the traditional classroom approach. Thus, study designs should be chosen carefully to clearly assess the effects of flipped learning.

The courses in which flipped learning was applied were generally practicum courses, including comprehensive practicum II, geriatric nursing, fundamentals of nursing practicum, health assessment, surgical nursing practicum, health education, fundamentals of nursing, health education and methodology, and psychiatric nursing practicum, which was consistent with other studies that analyzed flipped learning methods in health professions education [27–31]. The educational effects of flipped learning approach in practicum courses included improved student attitude, fewer errors in practice [5], and improved student perception, self-directed learning, academic achievement, and satisfaction with the class [27,29,31].

In a meta-analysis of the effects of learner-centered class in nursing, Lee and Yang [9] reported that a learner-centered class is effective in enhancing clinical performance and learning of major-related knowledge; they demonstrated that learner-centered approaches, such as flipped learning, are more effective than the traditional lectures in the study of nursing. Prior to 2017, common measured variables were self-directed learning [27], academic achievement [27], self-efficacy [32,33], critical thinking and communication [32], and learning motivation [33]. On the other hand, in the past three years, a number of variables, such as knowledge and performance, self-directed learning [34,35], critical thinking [34,36,37], academic self-efficacy [36,37], satisfaction with major [36], and communication and problem-solving skills [36]. As flipped learning enables students to repeatedly learn the contents related to their lesson of the day through pre-class learning without restrictions of time and place, students are able to adequately familiarize themselves with the lesson in advance, which increases their understanding of the lesson during the actual class. Many studies on flipped learning confirmed positive changes after applying flipped learning, and they generally used variables that influence students' tendencies or academic competence and performance, that is, variables positively correlated with the flipped learning approach [4,5,7,9]. As shown in previous studies, the results of the literature review section in this study confirmed that flipped learning improved satisfaction with class (*n* = 4), academic achievement (*n* = 3), self-directed learning, problem-solving skills, and information literacy (*n* = 2), and critical thinking. Future studies should discover new significant variables to examine the effects of flipped learning. To expand the utilization of the flipped learning approach, active research and effort are needed to verify its effects.

Flipped learning strategies were analyzed by dividing the class into pre-class, in-class, and post-class activities. Video lectures were primarily used in pre-class activities for practicum courses, while team-based learning or individual study were performed in pre-class activities for theoretical courses. In-class strategies included pre-class readiness assessment, instructor feedback, and hands-on training during practicum courses, while team-based discussions or presentation were performed in theoretical education. Popular post-class strategies included peer evaluation, self-evaluation journal, and instructor feedback, and post-class activities were not used in two cases. Gan et al. [13] analyzed flipped learning cases and proposed a basic operational model for theory and practicum courses, and a similar flipped learning model was used in the ten studies reviewed in this article. Lee and Chang [25] had students write a WSQ journal, which required them to watch, summarize, and question activities as a form of pre-class activity. WSQ journal writing was proposed by Kirch [38], which involves instructor-student interaction in finding the solution during in-class activities and helps students take responsibility for completing the pre-class portion. Lee and Chang [25] reported that the WSQ strategy has a positive influence on self-directed learning and academic achievement. Friedman and Friedman [39] proposed the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model. Lee [40] applied the ADDIE model to flipped learning and proposed pre-learning, in-class learning activities, and evaluation and self-evaluation after class as the major teaching strategies. Gan et al. [13] argued that the post-class part of learning is the stage in which the learned materials are maintained and expanded. Among the studies analyzed

in this review, two studies did not use post-class activities, but appropriate post-class activities, such as reviewing, summarizing, complementing, and writing self-evaluation journals, must be implemented to facilitate continuity of learning. The results of previous studies and this study highlight the need to use flipped learning designs and strategies tailored for each subject to boost their learning effects on students.

#### **5. Conclusions**

The purpose of the study is to identify and synthesize recent literature on the effects and strategies of flipped learning in education for the health professions in South Korea. We used participant–intervention-comparators-outcome (PICO) strategies to identify the published peer-reviewed articles from January 2017 to March 2020 in Korea Med, KCI, NDSL, and KISS. Ten published articles from 83 screened articles met the inclusion criteria. All articles targeted nursing students and mostly focused on practicum classes. The effects of flipped learning were measured by satisfaction, self-motivated learning, information literacy, and critical thinking disposition. We further analyzed pre-class, in-class, and postclass activities. The limitation of this study is that it was not possible to evaluate the effect of different types of flipped learning approaches under the limited number of analyzed studies. Despite these limitation, we found that the flipped learning model improved class performance, overall evaluation, self-motivated learning, self-efficacy, and problemsolving abilities. The study results suggest implementing a tailored flipped learning design based on class characteristics and emphasizing appropriate post-class activities to enhance students' learning abilities.

**Author Contributions:** Conceptualization, H.L. and J.K.; methodology, H.L. and J.K.; formal analysis, J.H.P. and W.S.H.; investigation, J.H.P. and W.S.H.; data curation, J.H.P. and W.S.H.; writing—original draft preparation, J.H.P. and W.S.H.; writing—review and editing, H.L. and J.K.; visualization, J.H.P. and W.S.H.; supervision, H.L. and J.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

