*Article* **I Am a Nursing Student but Hate Nursing: The East Asian Perspectives between Social Expectation and Social Context**

### **Luis Miguel Dos Santos**

Woosong Language Institute, Woosong University, Daejeon 34514, Korea; luisdossantos@woosong.org

Received: 12 March 2020; Accepted: 7 April 2020; Published: 10 April 2020

**Abstract:** From the East Asian social and cultural perspectives and contexts, this study aimed to understand the relationships and behaviors between nursing students' sense of filial piety and their decision-making behind selecting nursing education as their major. Forty-two traditional-age nursing students (i.e., six men and 36 women) at their final year of a bachelor's degree program in nursing were invited. The findings indicated that many nursing students disliked their university major and the potential career pathway as a nursing professional, as none of them selected the major based on their choice and interest. The environmental context and family's recommendations were the major impacts to influence the decision-making process of the participants. The result also indicated that filial piety, parents' recommendations, and elderly people's suggestions were the key factors to influence the selections and decisions of university major and career development pathways. The study provided a blueprint for related staff and professionals to create and design career counselling and services for East Asian youths to enable life investment and development.

**Keywords:** career development; counselling; cultural perspective; decision-making process; East Asian perspective; filial piety; nursing education; nursing shortage; nursing student; turnover

### **1. Introduction**

Nursing education and training are popular university majors for secondary school graduates, university students, and even adult returning students wishing to pursue life-long personal investment and development [1]. Every year, a large number of secondary school graduates and returning students decide to study nursing as their university major and to develop their careers. Although many individuals start their university education directly after completing secondary school, there are also a large number of adult returning students deciding to start their college and university study during mid-adulthood [2]. The selection of one's university major and career development path are decisions that impact the rest of one's life [3]. However, individuals' social, family, personal, educational, and economic patterns influence their decision-making processes, particularly for East Asian people, who have a strong sense of collectivism [4]. The expression of decision-making for career development is more likely employed when describing that process, which can happen at different stages of life [5].

Although nursing education is a very famous and popular university major, an inappropriate matching and mismatching of career development continues to happen in this profession. One study [6] collected information from 648 questionnaires at three hospitals in Taiwan about nurses' job satisfaction and intention to quit their position. The results indicated that a large number of nurses planned to leave their positions due to significant levels of stress and burnout at their workplace. Although they understand their job natures and responsibilities before entering the profession, many nurses decide to switch their career profession due to ideas of balancing work and family [7].

One study [8] employed the Ward Organizational Features Scales (WOFS) to measure the relationship between job satisfaction and work responsibilities of 834 acute ward nurses in the United Kingdom. The relationship between nurses and other medical staff was one of the most critical elements for their career decision. Additionally, many nurses indicated that the workload of their positions always negatively impacted their job satisfaction due to unbalanced responsibilities. It is worth noting that the negative measurement affected the career decisions of these nurses, who worked in a stressful environment [9].

A recent study [10] about the relationship between disappointment and nursing student retention collected data from 17 nursing students during the third year of their nursing program at university level. The report indicated that many nursing students decided to join the nursing profession because of the desire to provide a caring service, a personal background with the healthcare profession, modelling from peers, and potential career advancement. One study [11] indicated that a number of nursing students explained that negative experiences regarding their training, teaching staff, placement experience, and teammates always affected their career decisions. For example, some participants indicated that the excessive responsibilities and expectations from their supervisors could influence their career decisions due to negative placement experiences. Based on the results, it is not hard to predict that many nursing students or fresh graduate nurses may leave the nursing profession within the first few years of their career [11].

Another study indicated that nursing programs always have a high dropout rate due to the unique nature of the profession [12]. Unlike doctors who can provide individualized medical treatments to patients, nurses tend to assist other medical professionals. In other words, nurses cannot offer individual services without the supervision of other medical professionals. Therefore, once nursing students understand the nature of the profession, many decide to drop out [13].

Expectation is another element of dropout [14]. Providing a caring service is meaningful for many individuals who want to join social care and health services. However, the responsibilities of nurses are unique. Unlike social workers and counsellors who provide counselling and mental health services to individuals with problems, nurses need to provide both psychological and physical labor services from cleaning to operation room assistance. Therefore, the responsibilities are more comprehensive than for many professions in the field of social care and healthcare. As a result, after students understand the relationship between the duties and their expectations, they may drop out or leave the profession [15].

However, many individuals decide to enter the nursing profession due to stable job advancement and salary, particularly in developing countries [16]. Nursing, teaching, and social work are three demanding professions in society internationally. Many enter these fields due to financial, family, and personal considerations. Although these professions always welcome learners and second-career professionals, inappropriate matching is not uncommon. However, for various reasons, some mismatched individuals never leave their positions. In such cases, low-level performance, motivation, and morale prevail in the workplace [17].

One study [18] indicted that East Asian students are more likely to select a university major based on the interests and expectations of their parents, while most European students tend to select their major based on their own interests. Jin [19] indicated that East Asians' major selection tends to be influenced by filial piety toward parents. The researcher employed the Career-Related Filial Piety Scale (C-FPS) to investigate the relationship between parents' decisions and students' choices. The results indicated that students usually tend to study at universities their parents like and enter professional fields according to their parents' preferences. More recently, a study [20] conducted in Hong Kong reflected this as well. The study collected information from 522 undergraduate students about their reciprocal and authoritarian behaviors and senses about the relationship between students' filial piety and career decisions. The results indicated that although many students select their career pathways based on their interests, most respect their parents' wishes according to the traditional East Asian perspective [21]. Perhaps the social and cultural environment in Hong Kong influences young adults' career decisions due to the westernized nature of that society; most Chinese students and participants tend to listen to their parents in order to avoid arguments and misunderstanding [22].

### *Purpose of the Study*

The researcher has conducted research in the fields of social care, human resource management, school administration, and organizational psychology. The personnel shortage in the areas of social caring and nursing has been an issue for decades. Although many human resource professionals, policymakers, school administrators, leaders of social care departments, and researchers always establish plans to encourage potential medical professionals to work in the field. However, the results of these plans were not always effective. Due to the social and cultural expectations, the researcher wants to understand how the East Asian perspective influenced these issues. Currently, studies with a focus on nursing education and nurses' career development tend to explore participants' elements from a westernized perspective [7,11,13,16], in which individualism is the primary social norm and practice [4]. Therefore, this study aimed to understand the relationship between East Asian nursing students' sense of filial piety and their decision-making behind selecting nursing education as their major [23–34].

There are two purposes of this study. First, every year, a significant number of students decide to enroll in nursing education programs for initial training. Based on the literature review, various reasons for this have been found among these students [7]. However, many of them drop out of the programs or leave the nursing profession within the first few years of their career development [14]. Therefore, this study sought to study a group of traditional-age nursing students enrolled in a nursing education program but who will not enter the nursing profession after graduation. Second, the study sought to understand the factors that contribute to the relationship between East Asian university students' filial piety [20,35–37] and their decision-making behind selecting their university major and career development pathway in nursing [38]. The elements studied included outcome expectations, interests, and goals [38].

As a result, little empirical evidence exists for East Asia regarding the relationships between expectations, social-economic backgrounds, social environmental factors, and personal interests of young individuals and their selection of university subjects and career paths [8–11]. It is worthwhile to note that, unlike in Western cultures, Asian young people tend to highly respect their parents' suggestions, as well as the recommendations of their elders, such as parents, grandparents, teachers, and elder siblings, due to traditions of filial piety [8]. In short, this study serves as one of the first attempts to understand the relationship between filial piety and the decision-making of East Asian university students regarding their major selection and career development [23–34]. Unlike with Westerners, the values of filial piety, collectivism, and sense of family union are essential elements for East Asian people [39]. Therefore, this study will help provide opportunities for readers, university administrators, international school staff, educators, and international student service professionals to establish effective counselling services to assist this particular group of students.

### **2. Materials and Methods**

The employment of qualitative research methods [40] would be appropriate. Unlike quantitative research, qualitative research method allows the researcher to collect rich and in-depth data information from the participants. The researcher could access the information and lived stories which could not be answered by statistics and numbers [41]. Although the researcher had aimed to collect information from one single university counselling centre for a case study, the researcher believed the data information from one single source might not be able to express the holistic pictures of the current problem. Therefore, the researcher decided to employ the general qualitative approach [40,42–44], which may better apply to this study. Thomas [44] indicated that the general qualitative approach is an inductive approach that may meet most of the requirements for qualitative researchers. Moreover, without additional requirements, qualitative researchers could freely collect data information from the targeted

groups. As the current study might not fit other qualitative methods, such as case study, the general qualitative approach would be the selection [41,44].

### *2.1. Participants*

Fort-two traditional-age nursing students (i.e., six men and 36 women) at their final year of a bachelor's degree program in nursing were invited. All agreed to participate in this study. All the participants were local Taiwanese students. Five out of the 42 were indigenous Taiwanese. All were born and raised in Taiwanese families with the traditional East Asian conceptions. The age range was 18-22 years. The researchers invited these participants with the snowball sampling strategy [40,42,43]. In other words, the researcher invited participants based on the networking from other participants. Participants were enrolled at five different universities in Taiwan. The participants needed to meet the following criteria, which were:


### *2.2. Data Collection*

Two types of tools were employed, including individual and focus group activities [40,43,45]. The individual interviews were conducted to explore their understanding, life experience, and family issues under a private sharing environment [46–48]. The focus group activities were conducted to explore some similar background, social and family expectations, social bias, and interests. Both research tools were particularly useful in regard to listening to the voices by engaging them individually and collectively.

All participants have voluntarily participated in this study. The general inductive approach [44] was employed for qualitative data collection and analysis. The researcher was the collector of the data information. In order to seek meaningful data information, first, the semi-structured interview sessions were created. According to Seidman [49], individuals are less likely to share lived stories and personal background to others without any prior relationships. Therefore, in order to overcome this issue, the researcher decided to conduct two sessions of semi-structured, one-on-one, and private interviews in a private room at a community centre for each participant. Each interview lasted up to 40 min. One of the directions of this study was to explore the direct effect of filial piety on a career decision. Therefore, the protocol interview questions for both individual interview sessions tended to focus on the relationship between an East Asian perspective and the social context and issues. Appendix A shows the protocol interview questions for both interview sessions.

Second, after each participant completed the individual interview sections, all participants were invited into six focus group activities, each with seven participants. As the schedule of each participant was different, the focus group activity hosted seven members regardless of their university enrolments. The focus group activities were hosted at the same community centre. Appendix B shows the protocol interview questions for the focus group activities.

As it might be difficult for participants to share their understanding and experience in English language and a language other than their mother tongue, all the sections were conducted in Chinese Mandarin and translated into English. All the interview and focus group activities were audio-recorded and transcribed into written documents for data analysis. Member checking was done after the data analysis procedure. In order to protect the personal rights of each participant, all participants were given a pseudonym.

### *2.3. Data Analysis*

After the data collection procedure, 543 pages of written transcripts were created based on the semi-structured interviews and focus group activities. Qualitative researchers [40,43,45] advocated that large-size data information should be narrowed down to meaningful themes and patterns. Therefore, the researcher followed the general inductive approach (GIA) [44] for data analysis. First, the researcher used the open-coding procedure for the initial themes. After the initial themes were formed, the axial-coding procedure was employed for the second-level themes. As a result, two themes and seven subthemes were merged.

### *2.4. Human Subject Protection*

All the signed and unsigned agreements, personal contact, audio recording, written transcripts, computer, and related materials were locked in a password-protected cabinet. Only the researcher had the means to open it. After the study was completed, the researcher immediately destroyed and deleted all related materials for personal privacy.

Due to the agreement, the university information and place of origin was masked due to privacy. Most of the students were concerned that their university information would be disclosed. Due to the small population and closed professional networks, the researcher needed to protect the information of the participants. However, the participants allowed the researcher to show their gender for this study. Therefore, the following part will outline their gender with their discussion. It is worth noting that a small number of participants agreed to disclose their place of origin. The researcher reported this information with the discussion of the findings.

All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of The Youth Caring Association (2018/2019/SummerFall).

### **3. Findings and Discussion**

During each interview and focus group activity section, the participants answered the same general semi-structured questions that asked for their opinions and feedback. Although all participants had a similar family and social background in the same country, having come from one of the Taiwanese areas, their personal sharing, lived stories, and life experiences were not the same. Unlike in other studies, which have focused on westernized cultural perspectives, all participants reported that their major selection (i.e., nursing) was influenced by their family members, parents, and social expectations. In order to help answer the research questions, the findings were categorized into two superordinate themes and seven subthemes. It is surprising to note that all 42 participants were studying a university major that was not their own selection or interest. In other words, upon graduation, they would receive a degree, initial license, and potentially develop their long-term career pathways in a direction in which they have no strong interests. More importantly, many expressed that they will leave the nursing profession after graduation. Table 1 outlines the themes and subthemes of this study.


**Table 1.** A list of themes and subthemes for this study.

### *3.1. Influences from the Participants' Contextual Environment*

Before the session began, the researcher needed to indicate that the statements about I hate nursing. I dislike the medical career had been recorded 356 times. It was very surprising that the mismatching of major and human resources (i.e., students) was significant in the current Taiwanese university environment, particularly in the faculty of nursing. Although the current study could not represent the overall situation in Taiwan and the East Asian region, the significant results from this study may explore and contribute to an understanding of how these extreme situations [14] happen in the East Asian region [31].

By listening to the stories and life experiences of the participants, the researcher identified several elements that impacted their major decisions. The findings of this study discovered that all did not follow their personal goals and interests when selecting their major (i.e., nursing). The results reflected that the participants' behaviors also tended not to follow their own interests in choosing a career, which goes against a large number of previous findings [7,15,16]. It is worth noting that in most cases in East Asia, university students are not allowed to switch their major once they have been accepted. In other words, students are required to complete the major and degree that they submitted as their choice with their application. Therefore, the contextual and environmental factors that prevailed during their secondary school period highly influenced their life-long developments and career decisions [50]. A group of participants expressed that nursing was a choice of their parents during the application period, saying, "I can apply up to six universities and majors . . . But once I admitted with a school and major . . . I cannot switch . . . but the major application was written by my mother without my agreement . . . " (P#5, Male, Interview).

The results of this study supported the idea that youth have their own ways of thinking about career developments [7,51,52]. However, because the university's constraints on their university decisions and their overall contextual factors limited their opportunities, most were unable to exercise the career pathways that they might have decided on their own [14]. A discussion of the influences on the career decisions of the participants follows.

### 3.1.1. The East Asian Perspective on Occupational and Role Expectations

All 42 participants expressed that the East Asian perspectives about occupational and role expectations influenced their decisions about the major they were studying, their university enrollment, and their career decisions [53]. In the traditional East Asian perspective, medical practitioners are considered to be upper-class citizens with a high level of social status [54]. Although more than half of the participants expressed their desire to become an artist, reporter, chef, journalist, or photographer, none of them selected the majors associated with those occupations. One participant expressed that almost all artists and performers in the fine arts could not become famous until their death, saying, "Although I want to be an artist, I may become a nurse and do some part-time art activities . . . But I am not sure will I become a nurse become I hate this occupation now . . . " (P#2, Male, Focus Group). Another participant (P#4, Female, Interview) believed her hobbies in dancing could not be a long-term occupation, sharing,

*I chose nursing was because I think medical professionals are very smart in our society* . . . *I really enjoy the perspective of medical professionals* . . . *Although I want to become an artist with my wish and interests* . . . *both athlet[e]s and dancers can only work in their occupation until the early 30s. After that, the lucky one[s] can work as [a] coach. The others are living under the poverty line* . . . *.but now, I don't know* . . . *should I become a nurse? I hate this career* . . . *.* (P#4, Female, Interview)

It is worth noting that not only these two participants, but all 42 participants also expressed the ideas *I hate nursing and I dislike medical professions* in this category. Many expressed interests in other fields but had no chances and opportunities. Although the participants tried to escape from the major (i.e., nursing) they were not interested, they still needed to complete the degree and training due to the East Asian perspective and expectation from the society.

The lack of long-term career development in Taiwan always prohibited youths from joining their dream professions [55], particularly in the humanities and fine art occupations. A participant (P#24, Male, Focus Group) shared an idea:

*I like[d] botany and plant biology from [a] young age. I wish[ed] I c[ould] become a botanist from 8 years ago* . . . *[but] the long-term development of [a] botanist is unclear. No one can guarantee jobs after graduation, as there are more than 3,000 graduates in this field yearly. Also, Taiwanese people don't really consider Botanist as a professional. So if I can, I [will] select nursing instead. At least the general public will consider doctors as upper professionals* . . . *.* (P#24, Male, Focus Group)

It was not surprising to hear that in East Asia, university subject selections were not based on the students' own interests and desires for career development [23–34]. For example, the above participants wished to join botanical biology during her secondary school period. But due to the social context and social expectation [39], she needed to select the alternative (i.e., nursing) as her sixth choice on her application. One participant (P#36, Female, Interview) shared about enrolling in a nursing school instead of an electronic engineering program based on the gender-oriented bias and social expectation of gender-oriented occupations, saying,

. . . *my parents and the society* . . . *in East Asian countries believe engineers, doctors, pharmacists, and emergency medical technicians must be males. I love science and engineering* . . . *I want to become a medical engineer in the future* . . . *but I mother told me that in many Taiwanese universities, there are only a few female students in any of the engineering programmes* . . . *it is ugly to be the only girl* . . . *.* (P#36, Female, Interview)

Most described their selection of their university major as having been significantly influenced by the East Asian perspective, social context, and social expectations [23–34]. More importantly, the sense of collectivism [4] was highly represented in these groups of individuals. For example, several of the female participants refused to study one of the STEM subjects because of the social norms and social expectations of men in these areas. Female individuals, however, were expected to study and join the workforce in lesser industries, such as nursing, administration, elementary school education, and social work [56]. Due to the strong East Asian perspective, social context, and social expectation on selecting an academic major and career, it is worth noting that some young adults might give up on following their own interests merely on the basis of the enrollment of students of the opposite gender in a program for which they had prepared for more than four years [19].

### 3.1.2. Academic Results

Standardized exams and university admission placement tests are widely used in many countries [57]. In Taiwan, a widely used standardized exam is the General Scholastics Ability Test (GSAT). Although there are alternative pathways for enrollment, the GSAT is the most common admission exam for local Taiwanese students. Nearly 30 of the student respondents expressed that their low GSAT scores prohibited their enrollment in the university of their choice and also their selection of university subjects. Unlike in the American educational system, with the community college options, students are allowed to select an undecided major pathway. But Taiwanese students have to select their university enrollment and major during their final semester of 12th grade in secondary school. One participant (P#38, Female, Focus Group) expressed,

*I wish I c[ould] follow my will to become a medical doctor. But my low score* . . . *I can only apply for the nursing program* . . . *However, I cannot enrol at a community college or vocational college because there are no ways to build-up to the medical doctor's programme in the future* . . . *If I want to study but not re-take the exam for one more year, the only selection would be nursing* . . . *But again, I hate nursing* . . . *because it is not the same* . . . *.* (P#38, Female, Focus Group)

It is less likely for students to switch their university major once enrolled in many East Asian contexts due to unbalanced admission requirements and university policies. Unlike universities in the United States, students started their general education requirement during the first two-year of university education. Except for in special circumstances, students can switch and re-design their university major after the first or the second year of university education. The following participant (P#10, Male, Interview) shared his negative stories about the switching issue in Taiwan, saying,

*Once I studied nursing during my first year at university, I discovered that I don't like this subject. However, in order to change my university subject, I have to be [among] the top-rated students for the purpose of switching. Otherwise, I have to re-apply for the entire GSAT steps again* . . . *I hate nursing, I hate it so much* . . . *But I cannot switch it due to the administrative requirement* . . . *Also, the social expectation of my nursing career* . . . *What can I do* . . . *.* (P#10, Male, Interview)

In addition to their university enrollment and their selection of a major, the participants' academic results also limited their selection of university location. Some reported that they were forced by their parents and teachers to go to an urban university due to their high score of GSAT. One participant (P#11, Female, Interview) reported,

*I was partially willing to study nursing actually if I could stay in my hometown* . . . *In the southern part of Taiwan, there is a very good university and the nursing department. The enrolment of this university is my goal. But I received a high score in my GSAT. Therefore, my mother and school principal forced me to come to the capital city with a university that I dislike. I don't like the university, the social expectation of the university, and how people believed this university should create good nurses* . . . *So I hate nursing because of the social expectation* . . . *.* (P#11, Female, Interview)

It is worth noting that this participant (P#11, Female, Interview) was willing to study nursing if she could stay in her hometown. However, the East Asian perspective, social context, and social expectation [23–34] about her testing score, university, major, and potential career development destroyed her career perspective and beliefs. Although she wanted to become a nurse, she will now suffer due to the stresses and pressures from all different directions of her life.

Almost all described their GSAT scores, regardless of the level, as being a factor in their university enrollment, the decision about their academic major, and career development. Because the GSAT score was the most significant element, the decision about university subjects, the location of the university, and even the ability to switch university subjects after enrollment were not controlled by the participants. However, more importantly, a large number of participants expressed that based on their academic results, their parents, teachers, and even school leaders [35] expected them to enroll at top-tier universities and in a specific academic major. Almost all participants were forced to study in an academic major based on the recommendations from their parents and school staff. For example, a participant (P#32, Female, Focus Group) said, "I cannot cho[ose] which school and which location [where] can I study, I have to listen to my parents and teachers".

### 3.1.3. Financial Influence

Everyone experienced the stress of financial factors. Due to the East Asian traditions and cultural perspectives [34], children are responsible for taking care of their parents and even grandparents after their university graduation. Although the Taiwanese government does not regulate any policies about birth control, many contemporary Taiwanese families have fewer than two children.

Nearly two-thirds of the participants expressed that they would be the one who would take care of their parents and even grandparents after graduation, so they needed to select a career pathway that would allow them to make a significant amount of money. In fact, such East Asian perspectives on family engagements and responsibilities highly influenced the decision-making processes of these participants [31]. One participant (P#33, Male, Focus Group) told the researcher that being a nurse was almost the only way to take care of his parents and four grandparents after his graduation. In the same focus-group activity section, another participant (P#35, Female, Focus Group) echoed that situation with her own, saying,

*In many families, taking care of parents [is] expected. My parents expected me to take care of their late [years]* . . . *although I wanted to study music* . . . *I must study nursing, [in] which [I] can make money* . . . *But this must not be my wills* . . . *I really hate nursing* . . . *I don't want to become a nurse after university* . . . *I want my life back as an adult after university* . . . *I did not tell my mother about this yet* . . . *But I am sure I will not become a nurse* . . . *I dislike nursing not matter what* . . . *.* (P#35, Female, Interview)

Another Participant (P#25, Male, Focus Group) grew up in a rural community in central Taiwan. He also identified this family expectation and financial difficulties as a villager in the countryside. As mentioned above, many East Asian elderly were expected to be cared for by their children. Therefore, as a male child, the participant felt a need to send financial resources to his parents after university graduation. Consequently, this participant considered nursing as one of the occupations that could provide financial security, saying,

. . . *my parents sent me to university due to financial stress. I wish I c[ould] work in my family-run farm. But my parents want me to work in an upper social occupation* . . . *I need to listen* . . . *Moreover, my mother is a disabled person* . . . *I need to take care of her after I finished my university* . . . *But I want to continue my study in physical therapy in the future* . . . *I like the medical profession, but I surely dislike nursing as my life-long career development* . . . *I will explore a master's degree or another qualification soon* . . . *.* (P#25, Male, Focus Group)

It is worth noting that this participant has a very strong sense in the field of the medical profession. Although he took nursing as his alternative during his university application period, he will continue his career development in physical therapy due to these personal goals and financial considerations.

In addition to their concerns about post-graduation financial stress, participants also experienced pre-graduation financial stress due to their lower-income status. One participant (P#9, Female, Focus Group) used to plan to study in a culinary arts and bakery programs based on her goals and interests. However, most of the culinary arts and bakery program required higher tuition fees and supplemental fees. Because her family could not afford the extra fees, she had to give up her own goals and enroll in the nursing program. Another participant (P#8, Female, Focus Group) also shared similar stories, saying,

*I was planning to apply for the film major. But the department required [me] to have several expensive cameras and lenses, a high-quality computer, as well as the extra tuition fees for fine art lectures. I don't want to spend all the saving[s] of my family* . . . *I have to select a university major* . . . *which I hate* . . . *that can make money* . . . *.* (P#8, Female, Focus Group)

In short, the East Asian perspective, social context, and social expectation about the nursing and medical professions always limited their opportunities and career choices [6,25]. Unlike many studies conducted in the westernized societies and communities, many Taiwanese nursing students decided to enroll into one of the nursing programs based the social and cultural expectations [21]. Although university major and career development should be a selection of individuals, such preferences are not widely available in eastern societies. Furthermore, securing financial resources for their parents and elderly always forced them to enter the medical profession due to the stable salary and other financial considerations [29]. Although registered nurses may earn a reasonable salary for their family, it is unfair for both registered nurses and patients in this case in Taiwan. Although career mismatching always happens due to various reasons, this study discovered some significant findings which may gradually be detrimental to the medical and social health care system [11].

### *3.2. Influences from the Participants' Contextual Environment*

### 3.2.1. Parental Recommendations

"I do this degree for my parents" (P#13, Female, Interview). It is worth noting that the perspective of "I have to do this degree for my parents" was shared 320 times based on the transcripts.

Young adults and recent secondary school graduates usually do not have significant work and life experience from which to select a major that may, in turn, be an investment in their long-term career plans. One potential way to obtain recommendations is from their parents. Due to the ideas of collectivism, many East Asian people tended to conduct activities and behaviors based on the benefits of the groups and communities. Without the permissions from family members and group members, East Asian people usually conduct nothing further because of the ideas of respectfulness.

All advocated the ideas of collectivism with the traditional East Asian perspective [4] due to their respectfulness towards family members and elders in their cohort [25]. In fact, they consulted with their parents and even grandparents about university selection, academic major, and career development before they submitted their university application. One participant (P#20, Female, Interview) recalled a story about marking her major selections, saying,

*My mother asked me to study nursing as both of my parents are medical professionals in the national-level hospital. Perhaps I am not a good medical profession as I do not have any passion for my patients. But I have to listen to my parents as they are the one who took care of my life. I have to be respectful to them as a good child* . . . *This is traditional of us* . . . *Although I hate nursing* . . . *you don't know how much I hate nursing and the career development of a registered nurse* . . . *But I have to do this degree for my parents* . . . *.* (P#20, Female, Interview)

Another participant (P#28, Female, Interview) also shared a similar situation at the same focus group activity section, saying,

*My father is a doctor, and my mother is a nurse* . . . *they expected my sister and I [to] become medical practitioners* . . . *so, my sister was asked to enrol in a nursing school and so am I* . . . *my interest and career goal is to become a news reporter. But what can I do? Can I not listen to my parents? They spent almost half of their life for two of us* . . . *I cannot just say no to them* . . . *I hate nursing* . . . *But I am doing this degree for them* . . . *but for my own interests and goals* . . . *.* (P#28, Female, Interview)

From the above sharing, many participants selected their major and career development based on their parents' desires and recommendations instead of their own [29]. A study indicated that East Asian people tended to listen to their parents and cohort members as recommendations for their decision. Unlike people in the westernized society, East Asian people advocate the notion of respectfulness towards their parents, elders, and community members. Therefore, this study also reflected the practices of the current participants.

In addition to parental recommendations being an extension of the parents' original occupations, a large number of participants advocated that they were asked to study a major that their parents had not been able to achieve during their own youth. In fact, these expectations from parents are not uncommon in the East Asian perspective [27]. During the last century, most of the East Asian countries and regions were developing as third-world countries. Therefore, youth and students at that time did not have chances for university education. Therefore, parents of this generation and even the society have a higher-level of expectations in the current social context. One participant (P14, Female, Focus Group) was one of many with this case, saying,

*My mother was raised in a low-income family and wished to become a nurse during her childhood* . . . *but she couldn't* . . . *now, she asked me to complete her dream. But she did not ask me if I want to do so. As a daughter, I hope I can complete her dream for the purpose of respectfulness* . . . *however, I can tell you that I hate nursing* . . . *I studied this nursing degree for my mother* . . . *I will not join the nursing profession afterwards* . . . *absolutely no* . . . *.* (P14, Female, Focus Group)

Similarly, another participant (P#17, Female, Focus Group) also shared a case of a comparable situation, saying,

*My parents did not go to university during their young age* . . . *[so they] forced me to go to university* . . . *during my early teenage [years], they asked me to [study] medical biology. I have no interest* . . .

*but my mother liked this* . . . *I [studied]. Now, my parents want me to study nursing* . . . *I enrolled at my nursing programme now* . . . *because I want to be a good girl* . . . *but I am sure that I hate this nursing subject and I studies this degree for them obviously* . . . *I will not join this nursing profession afterwards* . . . *But I have to study this for my parents* . . . *.* (P#17, Female, Focus Group)

After P#17 shared her experience in a focus-group activity section, another participant (P#18, Female, Focus Group) echoed her similar case, saying,

*I am sure a lot of students selected their major based on some expectations of the East Asian perspectives* . . . *I am sure more than half of us [classmates]' majors were selected by parents or someone at home* . . . *at least four women in my dorm room were* . . . *I am so surprised that four girls [roommates] in our room were forced to study nursing because of our parents' decision* . . . *We won't join the nursing profession afterwards as we all studied that for our parents* . . . *We all understood that we hate nursing so much* . . . *.* (P#18, Female, Focus Group)

When the researcher asked questions about the interrelationships of financial responsibility, academic major, and career decision and development, one participant (P#42, Female, Interview) exclaimed that due to pressure and expectations from her parents,

*I absolutely want to become an early childhood teacher* . . . *but my mother just wanted me to study nursing in the capital city* . . . *if I cannot complete the degree, who is going to take care of her late [in] life? I am just a machine to take care of the elderly. But I have to take care of her, due to the filial piety* . . . *this makes me hate nursing so much* . . . *.* (P#42, Female, Interview)

One participant (P#25, Female, Interview) expressed that she had to listen to her parents' suggestion to attend medical school instead of the school of agriculture, due to financial considerations. Based on her life stories, the participant showed a solid awareness of her interests and career goals in the farming industry. However, her sense of filial piety limited her arguments because her parents' suggestions were a higher consideration than her own interests and career goals [31].

In short, all participants advocated that their major selections and career developments were chosen by their parents due to the East Asian perspective, social context, and social expectation, particularly parental recommendations and respectfulness [23–34]. The finding from this section was hardly found in much of the current literature with a westernized perspective. It is worth noting that all participants always respected their parents instead of their own interests and goals. Although respectfulness is encouraged, such mismatching may damage the health and social care system.

### 3.2.2. Teachers' Recommendations

In addition to the recommendations from parents, teachers also controlled the participants' decisions about their university enrollment, academic major, and career development. Nearly all participants revealed that their secondary school principal and homeroom teacher[s] had forced them to go to a nursing program because their enrollment would increase the reputation of the secondary school. One participant (P#23, Female, Interview) shared,

*My secondary school is located in a rural community* . . . *I am the first graduate who receive[d] a first-rated GSAT score* . . . *[my] school principal asked me to go to the capital city and stud[y] a bachelor's degree in nursing science, so the secondary school can promote my name and achievement in the city hall. I have asked my parents, family member[s], teachers, counsellors, and social workers about this. All of them advocated the relocation to the capital city and nursing school* . . . *It seems like I have no choice* . . . *but I want to study ocean studies* . . . *.* (P#23, Female, Interview)

Teachers' recommendations did not influence just a single student's decision, but a large number of participants. Questions about teachers' recommendations were asked during both the individual and focus-group activity sections, and nearly two-thirds of the participants shared their opinions on how teachers' recommendation influenced their decision. Several significant opinions are listed below. For example, a participant (P#39, Female, Interview) shared negative experience, saying,

. . . *my 12th grade homeroom teacher told me that the shortage of medical doctor[s] would be terminated within a decade. So I listened and obeyed her opinion and switch[ed] to nursing. This is the worst recommendation so far in my life* . . . *I want to be a doctor* . . . *not a nurse* . . . *I hate nursing, I like to be a medical doctor as my career development* . . . *It looks like I am studying for my teacher* . . . *I am so angry and upset* . . . *.* (P#39, Female, Interview)

Another participant (P#22, Female, Interview) also told her lived stories between her teachers and herself to the focus group participants, saying,

. . . *my teacher asked us to fill up the application form in front of her. She assigned us with the particular subject[s]. If we [did] not listen to her, I think that would be a little bit irresponsible? So I just listened to her. But this is certainly not my own will* . . . *So like, I am studying this degree for my teacher or what?* (P#22, Female, Interview)

In short, nearly all participants exhibited filial piety and respectfulness to their teachers and other school professionals [23–34]. Although some of them argued that their university enrollment and academic major, and their career decisions, were not their first choice, they felt that as good students they had to show the required filial piety and respectfulness.

3.2.3. Pressures from Family Members

Besides family and teachers who had daily interactions with the participants, other family members, such as cousins, uncles, and aunts, also revealed expectations about the participants' decisions regarding their university enrollment, academic major, and career pathway. Almost all participants expressed experience in this subtheme. One participant (P#3, Female, Focus Group) shared that her decisions were forced by her uncle, saying,

. . . *my uncle told my mother that he used to work in a hospital and believed the nurses can make a lot of money. So, he highly recommended [to] my mother* . . . *[that] I select this direction. My mother and my uncle sent me the nursing brainwashing messages everyday* . . . *respectfulness, listening, orders from the parents* . . . *I am sure I hate nursing now* . . . *but I am studying this degree exclusively for these two people* . . . *.* (P#3, Female, Focus Group)

Another participant (P#1, Female, Focus Group) also added to this theme and said,

. . . *my uncle is [in] upper leadership in chained clinics in the capital city. He always convinced my mother, and I worked for him. So, he called my mother every day and asked me to study nursing* . . . *nursing is a subject that I hated* . . . *But the pressure from my uncle and mother* . . . *I have to follow this pathway* . . . *for them* . . . *.* (P#1, Female, Focus Group)

During the same focus group activity section, Participant (P#29, Focus Group) shared a negative experience, saying,

*I was the only one who was forced by other family members* . . . *in the Taiwanese society, many people believe being a doctor or medical practitioner is an excellent occupation. I agreed. But this is not something that I want to do for my whole life. I always thought I was tricked by my family as well as the society* . . . *.* (P#29, Focus Group)

In short, many participants expressed negative opinions and experiences about the influence of family [23–34]. Because East Asian people tend to respect their elders' recommendations [29], nearly all participants followed the wills of their family members instead of their own personal interests. It is worth noting that not only parents but also related family members, had the authority to order the participant to choose a particular direction for study and career choice.

### 3.2.4. Comparison with Other Relatives

Many participants seemed to have had common experiences about how their parents compared their (i.e., the participants') university enrollment, major, and even GSAT scores with those of other children with similar backgrounds. When one of the participants (P#19, Female) was asked how she had decided to start her undergraduate study in nursing, she said that her parents compared her GSAT score and university application form with those of other children who were living in the same neighbourhood. The following remarks made by some of the participants seem to show similar situations of negative experiences. For example, a participant (P#16, Female, Interview) shared,

*My mother asked [about] other people's academic major selections in the community centre. Afterwards, she came back and asked me to study that nursing* . . . *I was very surprised that my career and major are decided and selected by a group of aunts in the community centre* . . . *I hate nursing for sure* . . . *But as a Taiwanese girl* . . . *I needed to listen to our cohort and members of my family* . . . *If I don't, I cannot show the respectfulness to the community* . . . *.* (P#16, Female, Interview)

Another participant (P#1, Male, Interview) also shared that his mother brought his application form to the community centre and asked opinions from other children. Besides sending their information and report cards to the community centre, some participants told the researcher their parents had even brought all their family members and relatives to their home for discussion. One participant (P#23, Female) shared her experience, saying,

*During the application period, my parents called everyone to my home for suggestions and discussions. My elder cousins provided more than 20 suggestions. My parents always compare[d] my scores, university enrolment, and major intentions to everyone* . . . *they compared my scores and major* . . . *my score was not excellent. Some of them even laughed* . . . *after they laughed, I still had to follow my mother's decision* . . . *.* (P#1, Male, Interview)

Another participant (P#26, Female, Interview) shared a similar situation, saying,

*My GSAT score was high enough to apply [for] most of the appropriate majors. I would like to go to a university in the southern part of Taiwan, as I wanted to escape from my family* . . . *but my parents wanted me to study near them. So I stayed* . . . *my parents wanted me to study medicine. Even if I want to study nursing, I have to follow their wills* . . . *because they are my parents* . . . *.* (P#26, Female, Interview)

Last but not least, the cohort and community-based collectivism always influenced how East Asian people behave. In this case, although the participants have their own goals and interests for major and career development, they did not have many choices due to the perspective of collectivism and respectfulness from their elders, family members, and members in their community. The results of this study outlined the relationship between the East Asian perspective, social context, and social expectation. The researcher used nursing students as a sample to explore and discover how this relationship exists in contemporary society. Without a doubt, many behaviors, ideas and perspectives were influenced by this relationship [23–34].

Attention should be paid to the directives given by the parents and family members. Although filial piety is a traditional aspect of the East Asian culture that may not change in the short term in Taiwan, the opinions and expressions of youth should be respected [36]. It is important to note that individuals' academic and career interests and goals significantly influence their career pathways and career development [58,59]. Parents and family members should avoid ordering their children to base their decisions about university enrollment and university subject selections on the desires of their elders. One participant (P#41, Female, Interview) indicated that playing a musical instrument as an academic major was her desire because that hobby was developed based on the decisions of her parents. However, she had no choices due to her parents' decision and application. Therefore, parents should also learn that they need to release more authority to their children and that the children should be taught career development navigation, such as vocational skills training [60].

### **4. Conclusions**

To the best of the knowledge, this is one of the very first nursing studies that is based on the approach of the relationship between an East Asian perspective, social context, and social expectation for East Asian university students' decision-making about their major selections and career developments. In some other countries, in which individuals do not have the significant sense of filial piety and obedience toward their parents and elders that East Asians do, individuals may follow their own interests and career goals for university enrollment and selection of their academic major, particularly in the field of nursing education [1,7,16].

In addition to the East Asian educational systems and structures that exist at the macro-level, at the micro-level filial piety and family negotiation continue to play a vital role for young adults in their decision-making about university enrollment, academic majors, and career progression, and that role could influence their long-term career investment and development [23–34]. Based on the data and experiences shared by this study's 42 participants, all students indicated that their decisions and intentions were solid but were made unwillingly. However, due to their sense of filial piety, almost all of them had tended to listen to and obey suggestions or orders from their parents, teachers, and elders [35].

Some readers may argue that parents accept opinions from the youth in contemporary society. However, because this study was conducted recently in Taiwan with several dozen young adults, it seems clear that their sense of filial piety in the decision-making process is not likely to change in the short run. In general, secondary school graduates are usually under the age of 18 years old. Therefore, their university applications and related forms must be signed by parents or guardians. Although their sense of filial piety would not change even if the students had reached the age of 18 years old, young university students in their early adulthood should have greater authority to decide their own career pathways. The study's findings indicate that some young adults choose to avoid the dictates of filial piety and instead pursue their own interests and career goals. For those interviewed here, the pursuit of their own interests and career goals combined with those directed by filial piety sometimes occurred [20,35–37]. For example, they may have pursued a double university subject major and minor study, to also satisfy their own desires.

### *4.1. Limitations*

Every research study has its limitations. Two limitations have been found in this study. First, this study was a qualitative research study with 42 traditional-age nursing students. However, as mentioned before, nursing education and career development are famous career selections for many university students and second-career changers. Therefore, future research may expand the population to non-traditional age students, second-career changers, returning students, and adult students.

### *4.2. Implementations*

In recent decades, some vocational higher education institutions have created vocation-oriented undergraduate degree programs for students who want to develop hands-on skills for their career pathways [61]. In addition to vocational undergraduate degree programs, some educational systems also allow individuals to take their first and second year of university education with a university subject as undecided. Both of those educational elements are significant blueprints for reforming Taiwanese and even all East Asian higher education institutions and universities, with the goal of expanding the current curriculum and policies for students with non-traditional backgrounds.

Instead of changing the sense of filial piety, the universities, departments of education, and related agencies could reform specific policies in order to avoid putting limitations on young adults. As discussed above, some participants expressed that the university policy did not allow them to switch their university subject. However, recent graduates of secondary school and young adults usually do not have enough life experience to choose appropriate career pathways during their late teenage

years. Thousands of university graduates do not participate in the industry or subject area in which they received their undergraduate degree. As a result, if universities would allow enrolled students to switch their academic major after their registration and enrollment, that potentially could increase the graduation rate and interest levels of students and contribute to their learning and enjoyment. It is also worth noting that the relevant agencies should discuss creating an undecided academic major for students who encounter difficulty deciding on a specific major.

Parents also can learn skills and techniques for sharing short-term, middle-term, and long-term career goals and academic interests with their children, while providing the children with the appropriate rewards, appreciation, encouragement, support, and involvement. In addition, parents can learn the significance of helping their children explore and discover their own academic interests and career goals, by bringing them on field trips to university campuses and organizations, introducing them to various cultural backgrounds and diversities, and increasing their understanding of particular subject matters. By conducting these parenting steps, the parents can help their children to exercise filial piety by a show of respectfulness toward their parents, while at the same time making their own decisions about university enrollment, university subject selection, and career development.

**Funding:** This research was funded by Woosong University Academic Research Funding 2020.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A**

Interview Protocol Questions: The First Interview


### **Appendix B**

Interview Protocol Questions: The Second Interview


### **References**


© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*

### *Article* **E**ff**ectiveness of Blended Learning in Nursing Education**

**María Consuelo Sáiz Manzanares 1,\* , María del Camino Escolar Llamazares <sup>1</sup> and Álvar Arnaiz González <sup>2</sup>**


Received: 15 January 2020; Accepted: 26 February 2020; Published: 1 March 2020

**Abstract:** Currently, teaching in higher education is being heavily developed by learning management systems that record the learning behaviour of both students and teachers. The use of learning management systems that include project-based learning and hypermedia resources increases safer learning, and it is proven to be effective in degrees such as nursing. In this study, we worked with 120 students in the third year of nursing degree. Two types of blended learning were applied (more interaction in learning management systems with hypermedia resources vs. none). Supervised learning techniques were applied: linear regression and K-means clustering. The results indicated that the type of blended learning in use predicted 40.4% of student learning outcomes. It also predicted 71.19% of the effective learning behaviors of students in learning management systems. It therefore appears that blended learning applied in Learning Management System (LMS) with hypermedia resources favors greater achievement of effective learning. Likewise, with this type of Blended Learning (BL) a larger number of students were found to belong to the intermediate cluster, suggesting that this environment strengthens better results in a larger number of students. Blended learning with hypermedia resources and project-based learning increase students´ learning outcomes and interaction in learning management systems. Future research will be aimed at verifying these results in other nursing degree courses.

**Keywords:** learning management system; higher education; nursing; data mining

### **1. Introduction**

In approximately the last decade there has been a marked interest in investigating ways of teaching other than traditional face-to-face. The incorporation of technological resources such as virtual platforms and hypermedia resources, combined with other innovative, methodological techniques such as project-based or problem-based learning, have revolutionized the teaching–learning process. The aim is to teach in the most efficient way possible and to make the most of resources while ensuring sustainability. These technological and methodological resources have been applied to different disciplines, especially in the field of health sciences (medicine, pharmacy, psychology, veterinary, etc.). However, in recent years these resources have been incorporated into nursing studies. Next, an approach will be made for the most relevant concepts of teaching in a virtual platform, which has been called blended teaching, as well as the implementation of methodological resources for project-based learning. Likewise, special importance will be given to its application for the formation of future nursing programs by analyzing the pros and cons of this form of teaching and learning in the society of the 21st century. For this reason, the most relevant concepts of these new forms of teaching and their

specific application to the nursing degree will be dealt with below. The final objective of this work is to study the effectiveness of different blended learning environments in the teaching of future nurses.

Twenty-first century society requires students and graduates to develop a series of skills related to two important leitmotifs: collaborative work and operation of information and communications technology (ICT). It is increasingly necessary to possess effective and rapid problem-solving skills and to develop digital competences [1]. The use of learning management systems (LMS) is, therefore, a reference in instructional practice, especially in higher education, as is the implementation of collaborative work in these methodological settings for the resolution of tasks and problems. A good example might be the use of project-based learning (PBL) methodology [2]. Recent investigations have confirmed that if such a methodology is accompanied by the use of hypermedia resources (e.g., flipped learning experiences, quizzes, use of wikis, on-line glossaries, etc.), then acquisition of deep learning is strengthened in students [3]. Deep learning is a concept developed in the framework of the taxonomy of Bloom [4]. It corresponds to the highest level of learning competences (comprehending, analyzing, summarizing, and evaluating their own learning). One of the currents of thought in LMS learning environments suggests that learning in these environments implies deeper learning from the point of view of cognitive and metacognitive complexity, as these facilitate self-regulated learning (SRL) and meaningful learning [5].

Likewise, LMS permit a more precise analysis of interactions which are logged in records (or logs). The logs represent units of information and registration that stores precise data on the frequency of user interactions and their duration [6]. LMS also facilitates the inclusion of hypermedia resources [7]. The use of these resources is especially relevant in health science degrees (nursing, medicine, pharmacy, etc.) since it implements practical assumptions in the work, which has been proven to reduce errors in the workplace [8].

There are various stages in this instruction process that will facilitate or inhibit the efficiency and depth of the learning process. One of them is the design of learning tasks in LMS [9,10]. Another essential element is that the teacher plans for process-oriented feedback [11].

### *1.1. Teaching through Learning Management Systems*

The teacher has to reflect, among other points, on the following points: (1) the aims of the subject module, (2) to whom it is addressed, (3) what previous knowledge is required for a successful approach to the subject matter, (4) the type of learning tasks that facilitate content acquisition, (5) the metacognitive skills of the students prior to the instruction, (6) the cognitive and the metacognitive skills in each task needed for its effective solution, and (7) when and where the teaching–learning process will be developed. Likewise, the teacher has to plan follow-up with both the individual student and the group behavior on the platform. As has been argued, solving problems in a collaborative way is one of the most demanding skills in 21st century society. These types of competences are key references in educational and technological areas and for entry into employment. Collaborative work facilitates the construction of deep and effective learning in the students [10]. A scheme for the preparation of pedagogic design in the LMS may be seen in Table 1.


**Table 1.** Preliminary elements to take into account for the design of learning activities.

Nevertheless, computational techniques are required to conduct a conclusive analysis of student behavior in LMS. As previously mentioned, at present a broad percentage of learning is done in virtual environments, in what is called blended learning teaching. A lot of data can be recorded by LMS and accessed through logs. However, educational data mining (EDM) [12,13] is needed to study them precisely. Machine learning techniques can be applied to EDM. Subsequently, possible applications of those techniques will be presented in the analysis of learning data in LMS environments.

### *1.2. Application of Artificial Intelligence Techniques to Analyze the Teaching and Learning Process*

Development of the internet and information and communications technology (ICT) has expanded learner access to information, and they have changed the way that information is taught and the way it is learned [14]. A learning management system (LMS) is an interactive learning environment that facilitates both teaching and learning. In addition, these software environments record all the actions performed by the teacher and by the students, under individual and group headings. However, those logs store a lot of data and learning analytics have to be used in order to study them in a flexible and accurate manner. These techniques can be simple, such as the ones usually found in LMS (descriptive statistics). However, more complex analytical techniques can be used, such as machine learning techniques (a subset of artificial intelligence). The latter are analogous to the computational thought of the human brain and operate with what is known as artificial intelligence. Machine learning techniques of classification and clustering [15] are among the most widely applied techniques for data analysis in educational environments. The use of these techniques for the analysis of both student and teacher behaviors will provide the teacher and those responsible for educational institutions with ideas to introduce improvements into the learning environment [11].

In brief, machine learning techniques are used, as these techniques are currently considered to provide the researcher with more data in the field of cognitive psychology and learning than traditional statistical techniques [16,17]. In particular, machine learning techniques permit personalized learning and provide individualized information on the development of student learning. Prediction techniques facilitate early detection of at-risk students and, therefore, personalized [18] help from the teacher [19,20]. Machine learning techniques also provide information on the effects of predicting the independent variable over each of the dependent variables in percent effects [21].

### *1.3. Design of the Blended-Learning Space in Nursing Instruction*

Blended teaching, increasingly present in educational scenarios, is done through a blend of face-to-face (F2F) and virtual learning on LMS, known as blended learning. However, there is no generalized agreement on the taxonomy of blended learning [22]. Nevertheless, its differences with blended learning are accepted; in the blended learning environment, the student completes 80% with LMS and 20% is F2F, hereafter referred to as Blended Learning type 1. In contrast, blended learning (80% interaction in the LMS) is a space where feedback is done 80% of the time through F2F and 20% through LMS [18,23], hereafter referred to as Blended Learning type 2. Recent investigations have found that the replacement blended environment accompanied by the use of active methodologies (e.g., PBL, use of hypermedia resources, flipped learning experiences, and quizzes, or all at once) improved the learning results of students [3,24]. These achievements are especially significant in university environments [25,26] because future graduates will have to develop collaborative work skills, problem-solving independence, and the use of new technologies. These skills are essential for good development of entrepreneurship.

Along these lines, recent studies have shown that [27] an educational intervention that applies blended learning methodology can easily be added into nursing curricula. This type of learning enhances learning in this field. Recent systematic research indicated that blended learning together with PBL is a methodology that ensures effective learning among nursing students [28]. This type of paradigm is more effective than traditional teaching such as face to face. The reasons are that students need to develop the knowledge and skills necessary in clinical practice. Several studies recommend nursing teachers to use multifaceted techniques (blended learning, learning based in projects, etc.) to promote effective learning beyond face-to-face teaching [29]. While these studies highlight the need to train teachers in these techniques [28], the main reason is that, traditionally, teaching has been done face to face, and an organized transfer towards the use of these methodological resources is needed. A recent systematic review showed that, since 2018, there has been a growing interest in the implementation of these experiences in nursing studies. However, an increase in these experiences and more research in this discipline of knowledge are needed [30].

Moreover, blended learning environments permit an evaluation of the whole teaching–learning process in a systematic and simple way. Thus, the suggestion is that there are different variables that influence successful learning in this line of investigation into e-evaluation models, especially the learning strategies employed by the students themselves [31,32], the environment in which the learning takes place [33], the teaching design that the teacher brings to the class [30], and the behavioral learning of the students in the LMS [23]. The prediction interval of these variables is situated around 56%-61% [34].

### *1.4. Extraction and Analysis of Information on the Teaching–Learning Process Recorded in LMS*

As we have mentioned earlier, development of the teaching through LMS will facilitate the student in learning recording and follow-up behaviours [35]. Many of these learning managers use supervised machine learning techniques, techniques such as multiple regression analysis (MRA), neural network, and SVM. Those techniques help with the detection and subsequent prediction of successful and risk behaviours. Behaviours of the students in LMS that have been related to successful learning are, among others [23]:


Therefore, the frequency and systematicness of student interactions and their interactions with LMS are directly related to effective learning [36]. Along these lines, recent investigations [23] have revealed differences in predicting learning results in relation to the variable "teaching methodology" (understood in terms of the pedagogic structure of the teaching, the evaluation procedures, and feedback). The type of activities and the evaluation tests (quizzes, tests, projects, presentations . . . ) are understood to determine the effectiveness of behavioural learning logged on the LMS.

As previously mentioned, application of machine learning techniques to study the logs will allow the teachers to analyze the behavioural learning of their students and to detect at-risk students. In these cases, early intervention will presumably improve student learning responses. Recent studies have confirmed [17,35] that following up with student behavioural learning in the LMS facilitates the identification of at-risk students with an explained variance of 67.2%.

In summary, the use of machine learning techniques will permit the study of behavioral learning of both students and teachers on the platform, which will facilitate the application of prediction techniques to the learning results [37]. Reviewing the investigations presented earlier, we consider it important to study the behavior of PBL in the LMS. As has been indicated, there are few studies in that field, and more information is needed that will help to improve teaching practices in these environments [38]. Project work and personalization of learning in LMS have been proven to have significant effects on the quality of learning. Particular relevance has been in Health Science degrees, such as nursing or medicine, etc., since it facilitates work on clinical cases in a collaborative way and optimizes the results applied to real learning contexts [39].

This research study was performed to analyze data of students' online and face-to-face (F2F) activity in a blended nursing learning course. We applied two types of blended learning: Blended Learning type 1 [in which the interaction between the teacher and students is 80% in the LMS and 20% Face to Face (F2F)] and Blended Learning type 2 [in which the interaction between the teacher and students is 20% in the LMS and 80% Face to Face (F2F)].

In light of the above, the hypotheses in this study were the following:

H 1: The types of blended learning (Blended Learning type 1 vs. Blended Learning type 2) used will predict student learning outcomes;

H 2: The types of blended learning (Blended Learning type 1 vs. Blended Learning type 2) used will predict the learning behaviours logged on the LMS; and

H 3: The type of clusters will be different for each type of blended learning used (Blended Learning type 1 vs. Blended Learning type 2)

### **2. Materials and Methods**

### *2.1. Design*

A quasi-experimental post-treatment design with an equal control group (in terms of metacognitive skill) was used. Likewise, learning outcomes (learning outcomes in the development of project-based learning; learning outcomes in exhibition of project-based learning; learning outcomes in the test; and learning outcomes total) and behavioral learning in the LMS were the dependent variables (access to complementary information; Access to guidance to prepare PBL; Access to theoretical information; Access to teacher feedback; and mean visits per day).

### *2.2. Participants*

A sample of 120 university students was assembled following the third year of their nursery degree in Spain (the degree has four years) during one semester (9 weeks): 63 followed the Blended Learning type 1 methodology and 57 followed the Blended Learning type 2 methodology (see Table 2).

The students were assigned to each blended learning group (Blended Learning type 1 vs. Blended Learning type 2) by means of convenience sampling. The work was developed in the subject of "Quality management methodology of nursing services."


**Table 2.** Group assignment and descriptive statistics for age, *<sup>a</sup>n* = 60. *<sup>b</sup>n* = 62.

Note. *Mage* <sup>=</sup> Mean Age; *SDage* = Standard Deviation.

### *2.3. Instruments*

a. *LMS UBUVirtual version 3.1*. A Moodle-based learning management system (LMS) was used that began with a constructivist approach and was developed through a modular system. It is a personalized Moodle-based LMS. An LMS is a modular learning environment that permits interaction and feedback between teacher and students, in many cases in real time, and in addition it facilitates the process of automated feedback.

b. The (ACRAr) Scales of Learning Strategies by Román & Poggioli [40]. This widely tested instrument identifies 32 strategies at different points in the information processing cycle. The reliability indicators on the scale were between α = 0.75 to α = 0.90 and the indicators of content validity were between r = 0.85 and r = 0.88. The subscale of metacognitive skills was applied in this study; this scale incorporated 17 strategies about the use of metacognitive skills into the problem solving tasks. A reliability index of α = 0.80 was obtained in this study; the reliability indicator on this subscale was α = 0.90 and the indicator of validity was r = 0.88.

c. *Student learning results: the results were recorded in the di*ff*erent evaluation procedures*. (1) Multiple-choice tests on the theoretical contents of the subject (test) were assigned a weight of 30% of the final grade. The test had 10 multiple-choice questions (four possible answers) with only one correct response. As well, five questionnaire-type quizzes were administered, one for each thematic unit. Cronbach´s Alpha reliability of the test was α = 81. (2) Development of PBL, with a weight of 25%, was measured with a rubric, which can be seen in Supplemental Material Table S1. (3) Likewise, the exhibition of the PBL, with a weight of 20%, was also measured with a rubric and can be seen in Supplemental Material Table S2. In the final mark, Cronbach´s Alpha reliability of PBL was α = 62. This result is lower because there was less dispersion among the scores in this type of evaluation test. Since the performance of the groups was quite uniform, this aspect can be checked in the results section and it is in accordance with the philosophy of PBL. Finally, the learning outcomes total covered the weighted scores of all the results (over 10 points). 4) The students solved five practices, and this part was 25% of the final grade. However, in this part all students had the highest qualification since the teacher reviewed the practices continuously, and if they were not correct the teacher ordered them to be repeated. Therefore, because it is not discriminate it has not been included in the analysis. Examples of the PBLs developed can be found at this link https://riubu.ubu.es/handle/10259/3753/discover.

### *2.4. Procedure*

Convenience sampling was followed for the choice of the sample. This was due to the possibility of working with this methodology by a specialist teacher who attended to both groups, and in this way the "type of teacher" effect was avoided. Before the instructional intervention, the two groups (Blended Learning type 1 vs. Blended Learning type 2) were scored on the metacognitive skills Scale of ACRAr [40], with the aim of establishing the similarities between both groups in terms of metacognitive skills.

As stated in the introduction, Blended Learning type 1 was applied to the experimental group, a learning environment in which the interactions between teacher and student were 20% F2F and 80% LMS. Likewise, Blended Learning type 2 was applied to the control group, a learning environment in which the interactions between teacher and students were 20% LMS and 80% F2F. In the experimental Group, hypermedia resources were used such as videos, and feedback was through the LMS. In contrast, classroom interactions between teacher and students and feedback in the control group were all F2F. In both groups, PBL methodology was followed. The difference, as has been pointed out, consisted of the type of blended learning in use (Blended Learning type 1 vs. Blended Learning type 2). Project development was done in both (the control and the experimental) groups in a collaborative way. The project work was completed in small groups of students of between 2 and 5 members.

### *2.5. Data Analysis*

The following statistical analyses were applied: (1) Analysis of asymmetry and kurtosis; (2) analysis of the variance of a fixed-effect factor (ANOVA); (3) multiple regression analysis (MRA) [appropriate Tolerance (T) values were considered close to one and, with respect to the variance inflation factor, the values were between 1–10]; (4) cluster analysis. Package for the Social Sciences (SPSS) v.24 was used to perform the different analyses [41]. Likewise, the Goodness-of-fit indices were measured by structural equation modeling (SEM) and was used to study the settings of the machine learning technique to predict the learning results. The calculations were performed with the Statistical Package for the Social Sciences (SPSS) AMOS v.24 [42]. (5) Finally, to visualize the results in a cluster analysis, RapidMiner Studio software [43] was used.

### *2.6. Ethical Considerations*

The research project was approved by the Ethics Committee of the University of Burgos. Previously, at the start of the project, the students were informed of the objectives, and their participation was at all times on a voluntary basis. Likewise, informed consent of each participant was recorded in writing.

### **3. Results**

### *3.1. Previous Statistical Normalcy Analysis in the Sample*

Before starting the research, the indicators of normality were studied. The results obtained from earlier statistical analyses with regard to the normality of the sample are presented below (values higher than |2.00| indicate extreme asymmetry, the lowest values indicate normality, and the values of between |8.00| and |20.00| suggest extreme kurtosis [44]). The results of metacognitive skills on the ACRAr subscale in both groups were acceptable for both indicators (see Table 3). Therefore, parametric statistics were used. Descriptive statistics are also shown in Tables A1 and A2 (see Appendix A).



Note. *M* = Mean Age; *SD* = Standard Deviation; *A* = Asymmetry*; K* = Kurtosis; *ASE* = Asymmetry Standard Error; *SEK* = Kurtosis Standard Error.

### *3.2. Previous Statistical Analysis of Homogeneity between the Groups before the Intervention*

Significant differences between both groups (experimental and control) in their use of metacognitive strategies were anlayzed before application of the different types of blended learning (Type 1 vs. Type 2). To do so, a single-factor ANOVA with fixed-effects was performed (blended learning type) on the results. No significant differences were found between both, so they can be considered similar groups (F1, <sup>119</sup> = 0.276; *p* = 0.601; η <sup>2</sup> = 0.002) in the ACRAr subscale of metacognitive skills.

Similarly, in order to study which type of supervised learning technique would be the most appropriate, the Goodness-of-fit indices were measured in the structural equation modeling (SEM) that was used to study the settings of the machine learning technique to predict the learning results. The calculations were performed with the Statistical Package for the Social Sciences (SPSS) AMOS v.24, as may be seen in Table 4, and no dependent relations between the observed values and the different prediction methods (LR, DT, RBFN, and kNN) were found for any of the four prediction models. Among these possibilities, the following were applied in the MRA.


**Table 4.** Goodness-of-fit indices.

Note. df = degrees of liberty; χ <sup>2</sup> = Chi squared; LR = Linear Regression; DT = Decision Trees; RBFN = Radial basis function network; kNN = k-Nearest Neighbor classification; NFI = normed-fit-index; RMSEA = Root-Mean-Square Error of Approximation; SRMR = Standardized Root-Mean-Square Residual; TLI = Tucker–Lewis index; CFI = comparative fit index; AIC = Akaike Information criterion; ECVI = parsimony index.

### *3.3. Hypothesis 1.*

MRA was performed to study the predictive value of the variable blended learning type applied to the student learning outcomes. An *R* <sup>2</sup> = 0.404 was found, which indicates that this variable explained 40.04% of the variance in the learning results. The Tolerance (*T)* values were within an interval of 0.106 and 0.336 and the Variance Inflation Factor (*VIF*) between 3.491 and 9.45, so none of the variables had to be removed. Likewise, the highest partial correlation was found in the Learning Outcomes Total (*r* = 0.586; *p* = 0.000), see Table A3.

### *3.4. Hypothesis 2.*

MRA yielded a figure of *R <sup>2</sup>* = 0.719 in the study of the predictive value of blended learning applied to student behaviors on the platform. This figure indicated that the blended learning type in use explained 71.19% of the variance in the learning behaviors of students on the platform. The Tolerance (T) values were situated within an interval between 0.136 and 0.539 and the Variance Inflation Factor (VIF) between 1.472 and 7.346, so that no variable had to be removed. The highest partial correlation was found in Access to Teacher Feedback (*r* = 0.448), see Table A4.

### *3.5. Hypothesis 3.*

A k-means clustering technique was applied in each type of blended learning in use (Blended Learning type 1 vs. Blended Learning type 2), as seen in Table 5. Three clusters are shown in Table 5 that were found in the two types of blended learning (Cluster 1, Sufficient; Cluster 2, Intermediary; and Cluster 3, Excellent. The classification of Cluster type was according to the maximum possible value in each learning outcome and number of accesses obtained). Higher values for performance were found in the Blended Learning type 1 rather than the Blended Learning type 2 in all three clusters, specifically in Learning Outcomes Total. Likewise, with regard to the learning behaviors developed by students in the type of blended learning in use (Blended Learning type 1 vs. Blended Learning type 2), as may be seen in Table 5, a higher number of log-ons to the platform in the Blended Learning

type 1 rather than the Blended Learning type 2 environment were found, except for student queries on theoretical information provided by the teacher (see Table 6).

**Table 5.** Centers of final clusters for the learning results variable in Blended Learning type 1 and type 2, Blended Learning type 1: <sup>a</sup>*n* = 1; <sup>b</sup>*n* = 45; <sup>c</sup>*n* = 17; Blended Learning type 2: A lost value is observed. *<sup>a</sup>n* = 9; *<sup>b</sup>n* = 30; *<sup>c</sup>n* = 18


Note. PBLD = Project-Based Learning Development; PBLE = Project-Based Learning Exhibition.

**Table 6.** Centers of final clusters and the variable behavioral learning on the LMS in Blended Learning type 1 and type 2. Blended Learning type 1: a lost value is observed. *<sup>a</sup>n* = 31; *<sup>b</sup>n* = 27; *<sup>c</sup>n* = 5; Blended Learning type 2: Two lost values were observed. *<sup>a</sup>n* = 36; *<sup>b</sup>n* = 16; *<sup>c</sup>n* = 6.


Note; PBL = Project-Based Learning Development.

Figure 1 shows the scores in the two groups: experimental group (red color) and control group (blue color). As can be seen, there was a greater homogeneity of higher scores in the experimental group for different types of Learning outcomes. Similarly, Figure 2 points to the distributions of LMS behavioral learning scores in different resources.

behavioral learning scores in different resources.

Blended Learning type 1: <sup>a</sup>*n* = 1; <sup>b</sup>*n* = 45; <sup>c</sup>

*bn* = 30; *<sup>c</sup>*

*n* = 18

Figure 1 shows the scores in the two groups: experimental group (red color) and control group (blue color). As can be seen, there was a greater homogeneity of higher scores in the experimental

**Table 5.** Centers of final clusters for the learning results variable in Blended Learning type 1 and type 2,

**Maximum Cluster <sup>1</sup>**

**Blended Learning type 1**  Learning outcomes in PBLD 2.50 1.75 2.00 2.34 Learning outcomes in PBLE 2.00 1.00 1.62 1.80 Learning outcomes in test 3.00 2.30 2.24 2.50 Learning outcomes Total 10 7.00 8.62 9.26 **Blended Learning type 2** Learning outcomes in PBLD 2.50 1.88 2.09 2.32 Learning outcomes in PBLE 2.00 1.50 1.59 1.87 Learning outcomes in test 3.00 1.70 1.82 2.39 Learning outcomes Total 10 6.08 8.00 9.08

Note. PBLD = Project‐Based Learning Development; PBLE = Project‐Based Learning Exhibition.

type 1 and type 2. Blended Learning type 1: a lost value is observed. *<sup>a</sup>*

Learning type 2: Two lost values were observed. *<sup>a</sup>*

Note; PBL = Project‐Based Learning Development.

**Table 6.** Centers of final clusters and the variable behavioral learning on the LMS in Blended Learning

*n* = 36; *bn* = 16; *<sup>c</sup>*

**Blended Learning type 1** Access to Complementary Information 0–14 9 14 14 Access to guidance to prepare PBL 0–6 10 9 6 Access to Theoretical Information 0–14 12 18 14 Access to Teacher Feedback 0–158 69 103 158 Mean Visits per day 0–7 2.48 3.40 4.51 **Blended Learning type 2** Access to Complementary Information 0–7 4 6 7 Access to guidance to prepare PBL 0–5 3 5 5 Access to Theoretical Information 0–14 12 18 14 Access to Teacher Feedback 0–66 5 30 66 Mean Visits per day 0–2 0.84 1.30 1.93

**Interval Cluster <sup>1</sup>**

*n* = 6.

**Sufficient**

**Sufficient**

*n* = 17; Blended Learning type 2: A lost value is observed. *<sup>a</sup>*

**Cluster 2 Intermediary**

*n* = 31; *bn* = 27; *<sup>c</sup>*

**Cluster 2 Intermediate** *n* = 9;

**Cluster 3 Excellent**

*n* = 5; Blended

**Cluster 3 Excellent**

**Figure 1.** Distribution of scores in the different types of Learning outcomes. Note. Development PBL = Development Project‐Based Learning outcomes; Exhibition PBL = Exhibition Project‐Based Learning outcomes; Test = Test Learning outcomes; Total LO = Learning outcomes Total. **Figure 1.** Distribution of scores in the different types of Learning outcomes. Note. Development PBL = Development Project-Based Learning outcomes; Exhibition PBL = Exhibition Project-Based Learning outcomes; Test = Test Learning outcomes; Total LO = Learning outcomes Total. *Int. J. Environ. Res. Public Health* **2020**, *17*, x FOR PEER REVIEW 10 of 16

**Figure 2.** Distribution of scores in the different types of behavioral learning in the LMS. Note. CI = Access to Complementary Information scores; CPBL = Access to guidance to prepare PBL scores; TI = Access to Theoretical Information scores; F = Access to Teacher Feedback; MVD = Mean Visits per **Figure 2.** Distribution of scores in the different types of behavioral learning in the LMS. Note. CI = Access to Complementary Information scores; CPBL = Access to guidance to prepare PBL scores; TI = Access to Theoretical Information scores; F = Access to Teacher Feedback; MVD = Mean Visits per day.

#### day. **4. Discussion**

techniques [12,13].

**5. Limitations**

**4. Discussion** In the blended learning environments, the type of teaching design appears to be a predictive factor in both the learning results and the learning behaviors that the students develop in the LMS. blended learning with 80% of interactions in LMS appeared to be more effective, both with respect to the learning results of the students and the effectiveness of the learning behaviors that they develop. This type of pedagogic design includes the use of hypermedia resources that strengthen teacher feedback in real time, which furthers the development of SRL strategies [24,35]. This aspect is of special relevance for teachers in nursing higher education, and the implicit message is that they would be well advised to design their materials for use in a blended learning environment [27–30], as those environments appear to have increased the effectiveness of active methodologies, especially PBL with hypermedia resources in LMS [38]. In addition, Blended Learning type 1 (80% the interaction in the LMS) strengthens students' use of learning‐based projects that have been considered more effective in the LMS [2,23]. These behaviors range from access to feedback given by the teacher to tasks carried out by the student or the collaborative groups and the average number of visits per day [23]. All of this indicates that the Blended Learning type 1 design increases the interaction of the student in the LMS and that interaction also facilitates student access to feedback from the teacher, as the LMS can be consulted as many times as necessary when learning, an aspect that is less feasible with F2F instruction [9,10,18]. In this way, the teachers can structure their help In the blended learning environments, the type of teaching design appears to be a predictive factor in both the learning results and the learning behaviors that the students develop in the LMS. blended learning with 80% of interactions in LMS appeared to be more effective, both with respect to the learning results of the students and the effectiveness of the learning behaviors that they develop. This type of pedagogic design includes the use of hypermedia resources that strengthen teacher feedback in real time, which furthers the development of SRL strategies [24,35]. This aspect is of special relevance for teachers in nursing higher education, and the implicit message is that they would be well advised to design their materials for use in a blended learning environment [27–30], as those environments appear to have increased the effectiveness of active methodologies, especially PBL with hypermedia resources in LMS [38]. In addition, Blended Learning type 1 (80% the interaction in the LMS) strengthens students' use of learning-based projects that have been considered more effective in the LMS [2,23]. These behaviors range from access to feedback given by the teacher to tasks carried out by the student or the collaborative groups and the average number of visits per day [23]. All of this indicates that the Blended Learning type 1 design increases the interaction of the student in the LMS and that interaction also facilitates student access to feedback from the teacher, as the LMS can be consulted as many times as necessary when learning, an aspect that is less feasible with F2F instruction [9,10,18]. In this way, the teachers can structure their help and prepare specific materials for each group.

and prepare specific materials for each group. In addition, machine learning techniques have been used in this study, in view of their effective use with what is known as data mining [13,18,36]. In particular, supervised and unsupervised machine learning techniques have been used (linear regression and clustering K‐means methods, respectively). Prediction and clustering studies, among others, can be conducted with these techniques, which help the teacher to gain knowledge of the learning characteristics of students and In addition, machine learning techniques have been used in this study, in view of their effective use with what is known as data mining [13,18,36]. In particular, supervised and unsupervised machine learning techniques have been used (linear regression and clustering K-means methods, respectively). Prediction and clustering studies, among others, can be conducted with these techniques, which help the teacher to gain knowledge of the learning characteristics of students and to predict at-risk students [23,34,38]. Even so, it is true that these techniques should be used throughout the whole

start, in the middle, and at the end of the study module will be analyzed with machine learning

This study has limitations, but the results of this study should, nevertheless, be given prudent consideration. Limitations include the following: methodological intervention was in one university, the students were from a specific country, convenience sampling was applied, the knowledge area of the students was specific, and the type of design (quasi‐experimental) was also specific. Although, it must be taken into consideration that there are few specific studies to test the effectiveness of this type of methodology in nursing students. Studies that have been carried out have similar

characteristics that are justified from the specificity of this research [28–30].

teaching process to be able to develop personalized actions for student learning [35,36]. In subsequent studies, therefore, development of the learning process among students at the start, in the middle, and at the end of the study module will be analyzed with machine learning techniques [12,13].

### **5. Limitations**

This study has limitations, but the results of this study should, nevertheless, be given prudent consideration. Limitations include the following: methodological intervention was in one university, the students were from a specific country, convenience sampling was applied, the knowledge area of the students was specific, and the type of design (quasi-experimental) was also specific. Although, it must be taken into consideration that there are few specific studies to test the effectiveness of this type of methodology in nursing students. Studies that have been carried out have similar characteristics that are justified from the specificity of this research [28–30].

Therefore, future studies will be directed at increasing the size of the sample and the diversity of the nursing degree course level. Therefore, this profession is subject to continuous theoretical and technological advances that require systematic research on how to teach better in order to learn more effectively.

### **6. Conclusions**

This research study has identified the characteristics to design an effective LMS in the nursing degree. The use of prediction and clustering techniques is very important to facilitate personalized learning and to analyze how resources are better utilized in the blended learning space. This type of analysis can be automatically generated in LMS environments, such as Moodle, and could be integrated in modules and plugins. These tools would facilitate rapid and straightforward generation of those analyses, which would be of great utility for the teacher and would assist with the early detection of at-risk students, as well as behavioral analyses of both the individual student and the collaborative groups of students, which would foreseeably increase the teaching quality and learning outcomes. This need has been underlined in such studies as those by Peña-Ayala [12] and Romero et al. [17], and they have to be approved by university management, but they will virtually be a necessity in 21st century teaching as we move closer to personalized on-line teaching, both in F2F teaching and in virtual learning environments. In summary, this teaching design is especially significant in the nursing degree since project work is a practice that has proved very effective in the training of future professionals.

Good results have been obtained in all assessment tests in the two types of blended learning. However, the type of blended learning that applied automated feedback and hypermedia resources obtained even better results (more percentage of work in the LMS) [28–30]. One explanation may be that the student can access information in the LMS at any time, which is not possible for F2F interaction, and this facilitates personalization of learning and motivates the student [7,8,39]. Therefore, incorporating these forms of work in teaching in the field of health is a very effective option.

The results obtained are in line with those found in the research of Oh & Lee [28]. The use of PBL methodology in blended learning environments empowers nursing students to acquire practical skills that are of great help for nursing work in real intervention environments [28]. This form of teaching is flexible [30] because it facilitates development and tests hypotheses in the resolution of tasks similar to those they will encounter in a working environment, and in addition, group work facilitates the acquisition of collaborative work skills, which they will also encounter in such working environments. All this increases the self-efficacy and critical thinking skills of these professionals. Recent studies recommend the application of this methodology within the nursing degree curricula [27].

In sum, it can be concluded that this way of teaching seems to be effective for nursing students. Although, more studies are needed in this field aimed at studying the effectiveness of blended learning in teaching in the nursing degree.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/1660-4601/17/5/1589/s1, Table S1: Rubric to evaluate development of Project-Based Learning, Table S2: Rubric to evaluate Exhibition of Project-Based Learning.

**Author Contributions:** Conceptualization, M.C.S.M., and M.d.C.E.L.; methodology, M.C.S.M.; software, Á.A.G.; validation, M.C.S.M., and Á.A.G.; formal analysis, M.C.S.M.; investigation, M.C.S.M., and M.D.C.E.L.; resources, M.C.S.M.; data curation, M.C.S.M.; writing—original draft preparation, M.C.S.M., and M.D.C.E.L. and Á.A.G.; writing—review and editing, M.C.S.M., and M.D.C.E.L. and Á.A.G.; visualization, Á.A.G.; supervision, M.C.S.M., and M.D.C.E.L.; project administration, M.C.S.M.; funding acquisition, M.C.S.M., and M.D.C.E.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the of Consejería de Educación de la Junta de Castilla y León (Spain) (Department of Education of the Junta de Castilla y León), Grant number BU032G19, and grants from the University of Burgos for the dissemination and the improvement of teaching innovation experiences of the Vice-Rectorate of Teaching and Research Staff, the Vice-Rectorate for Research and Knowledge Transfer, 2020, at the University of Burgos (Spain).

**Acknowledgments:** Thanks to all the students who participated in this study and the Committee of Bioethics of University of Burgos (Spain).

**Conflicts of Interest:** The authors declare no conflicts of interest.

### **Appendix A**


**Table A1.** Descriptive statistics for the Learning outcomes.

Note: *M* = Mean; *SD*= Standard Deviation; PBL = Project-Based Learning; α = Reliability index of Cronbach's Alpha.



Note: *M* = Mean; *SD* = Standard Deviation


*Int. J. Environ. Res. Public Health* **2020**, *17*, 1589

Access to Theoretical Information −0.01 0.003 −0.21 −3.49 0.001 0.06 −0.31 −0.17 0.68 1.47 Access to Teacher Feedback 0.01 0.001 0.60 5.37 0.00 0.82 0.45 0.26 0.19 5.25 Mean Visits per day 0.08 0.05 0.20 1.55 0.13 0.76 0.14 0.08 0.14 7.35

Note: Dependent variable: Blended Learning type; *VIF* = Variance Inflation Factor.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*
