Hybrid Learning Effects on Indonesian Students Majoring in Industrial Engineering for Understanding and Performance: A Case Study with an Experimental Design ^†

Abstract

Due to the COVID-19 pandemic, most schools and colleges have adopted hybrid learning. Hybrid learning, also called “blended learning,” mixes online and classroom instruction. Blended learning may become permanent as face-to-face and internet-based education become more accepted. We examined how hybrid learning affects the understanding of Indonesian students majoring in Industrial Engineering at Atma Jaya Catholic University. In this study, understanding matched learning efficacy. An experimental design was used to measure component influence. In strategic planning, strengths, weaknesses, opportunities, and threats (SWOT) analysis was used as an effective tool to examine an organization’s internal and external variables with a learning methodology design. A questionnaire survey was conducted to measure the understanding of the SWOT analysis results and the related strategy. A total of 96 participants were involved in this study. The mixed learning method, using the weakness–opportunity or mini–maxi strategy with the divestment–investment principle, was found to be effective.

1. Introduction

Information technology is a means to disseminate collective knowledge for educational innovations. Innovative technology-based educational tools such as mobile devices, smartboards, Massive Open Online Courses (MOOCs), tablets, laptops, simulations, dynamic visualizations, and virtual laboratories have significantly transformed the educational landscape in schools and institutions [1]. In the last ten years, digitalization has influenced the global educational system, garnering attention from professionals, scholars, and policymakers in order to enhance educational progress [2]. While technological education covers various areas and is tailored to individual characteristics, there is a lack of studies on the selection of themes and career education during free semesters. Consequently, professors are hesitant to accept hybrid learning systems due to the extra burden they entail [3]. The current frameworks of technological knowledge derived from the philosophy of technology have not been widely utilized in technology education, particularly in the design of curricula. This is likely because most of these frameworks were not originally developed for technology education [4].

The hybrid learning system is anticipated to provide a resolution and combine in-person learning experiences by optimizing the utilization of technology when students are required to study remotely or online due to the pandemic [5]. The hybrid learning system has advantages such as fostering a productive and streamlined learning experience, providing easy accessibility and flexibility, reducing expenses and energy consumption in education, and enabling students to achieve optimal independent learning [6]. Implementing a hybrid learning system impacts students’ metacognitive awareness [7]. Presently, there is significant progress in the advancement of hybrid learning, which is derived from the amalgamation of one or more models, methodologies, or media [8]. Indonesia is developing several hybrid learning methods, including face-to-face lectures, synchronous virtual collaboration, asynchronous virtual collaboration, and self-paced asynchronous learning [9].

There are several issues related to the quality of education in Indonesia. The main challenges faced in the education system include facilities, the inadequate quality of teachers, low student achievement, a lack of student interest in reading, the high cost of education, and the need for greater diversity tolerance (SARA) [10]. Hybrid learning has gained popularity in developed countries owing to the widespread availability of the internet. Extensive research has been conducted in developing countries, particularly in the ASEAN region. The disparity in the number of pupils and teachers occurs in developing and developed countries [11]. Indonesia’s education system has been characterized by a large quantity of education but lacking in quality, failing to meet the country’s aspirations for an education system at an international level. The causes for this are insufficient money, lack of human resources, flawed incentive systems, and ineffective management. However, the underlying cause is related to political dynamics and power struggles [12].

The Industrial Engineering Study Program of the Atma Jaya Catholic University of Indonesia has traditionally adhered to conventional learning methods known as teacher-centered learning until recently. Nevertheless, it has adjusted and adhered to the progress of education by using a student-centered learning approach. Consequently, the hybrid learning paradigm has been introduced in the Performance Measurement and Improvement course. Hybrid learning cultivates students to be engaged, self-reliant, accountable, creative, and proactive in their learning. In order to guarantee the effectiveness of the hybrid learning model, educational administrators must consider the following key factors: the integration of existing technologies, the provision of adequate student support facilities, motivation strategies, and the development of a well-designed and balanced online curriculum for students.

Currently, students in the Industrial Engineering study program at Atma Jaya Catholic University of Indonesia lack motivation and performance. This can be linked to the monotony associated with the traditional learning approach, which primarily emphasizes the role of professors. Consequently, a new instructional approach is needed to adopt hybrid learning. Hybrid learning is beneficial for stimulating students’ curiosity and enhancing their efficiency by motivating them to be updated with campus improvements and changes. This learning strategy also encourages students to engage with the materials to actively acquire knowledge [13]. The hybrid learning model is anticipated to enhance students’ motivation and performance, beyond the levels observed in the conventional learning paradigm. The effectiveness of implementing the hybrid learning paradigm has been confirmed by the results of multiple prior studies [14,15,16,17,18]. Hybrid learning is advantageous for students and their learning process. However, because of the diversity of the people involved, the outcomes of this program may vary.

We investigated the students’ understanding of the learning process implemented through a hybrid learning approach. Utilizing the principles of experimental design and the True Experimental Design Nested Factorial Experiment, we conducted a SWOT analysis to develop strategies and establish a hybrid learning strategy. Such a hybrid learning approach can be used to enhance the learning outcomes of individual students with diverse personal backgrounds. We also explored how the hybrid learning model effectively enhanced students’ academic performance and understanding.

2. Literature Review

2.1. Hybrid Learning

Virtual learning and hybrid learning facilitate learning and foster the development of digital literacy by promoting the utilization of digital resources as a means of communication [19]. In digital literacy, technology is the key to seeking and assessing resources and obtaining accurate and pertinent material for effective communication [20]. The authors of reference [21] examined how the combination of hybrid learning and critical thinking abilities affects the writing performance of undergraduate learners in Indonesia. The findings indicated that the implementation of hybrid Task-Based Language Teaching (TBLT) influenced the writing proficiency of the students.

Multiple researchers examined the influence of hybrid learning on digital literacy in language learning. The influence of hybrid learning on digital literacy was surveyed for fifteen undergraduate students enrolled in English Language Study Program Research [22]. Hybrid learning significantly enhanced the students’ proficiency in digital literacy. Hybrid learning impacted grammar and digital literacy, too [23]. The findings indicated that the implementation of hybrid learning improved students’ proficiency in grammar and digital literacy. In previous research, the effects of hybrid learning or the use of a single application on digital literacy in various academic fields were confirmed.

2.2. Experimental Design

The objective of the experimental research design is to ascertain the causal links between the independent and dependent variable(s). While there is ongoing discussion regarding the definition of a causal relationship [24], experts in organizational and behavioral sciences [25] agree that a cause–effect relationship is determined by three criteria: (a) a consistent relationship between the independent and dependent variables; (b) the independent variable changing before the dependent variable; and (c) ruling out other possible explanations for the observed relationship. Experimental study designs are crucial to reduce potential risks to internal validity. Internal validity refers to the level of confidence in the causal relationship between a spontaneous or manipulated change in the independent variable and the resulting change in the dependent variable [26].

2.3. Strengths, Weaknesses, Opportunities, and Threats (SWOT) Analysis

SWOT analysis was introduced in the early 1950s as a framework for examining organizational plans [27]. This framework has been extensively utilized in education for strategic planning and decision-making in complex situations that necessitate the perspectives of the talents of different individuals involved [28]. SWOT analysis offers a systematic framework for collecting information from different sources and presents a comprehensive assessment of the internal (strengths and weaknesses) and external elements (threats and opportunities) that may impact the incorporation of new technology in education [29]. SWOT analysis is employed to build plans based on internal and external elements that have the potential to impact decision-making within the business [30]. The advantages of using this method include the following: (a) providing a broad overview of the objective and establishing a framework for solving a problem by starting with a general approach and then moving to specific details; (b) being applicable at both the organizational and macroeconomic levels and addressing various decision subjects; and (c) highlighting opportunities and guiding decision-makers in addressing weaknesses and threats [31].

3. Methodology

3.1. Data Collection

We employed a probability sampling method (stratified random sampling) to recruit participants. Out of a population of 261 students majoring in industrial engineering at Atma Jaya Catholic University of Indonesia, aged between 18 and 23 years, 72 students were selected as participants. The example figures were calculated using the Slovin formula.

We gathered their performance data with factors using the Material Test, which was designed as part of the Measuring and Performance Improvement course material. The examination comprised 40 questions that were randomly organized with a variety of materials. For SWOT analysis, we collected data using qualitative and quantitative methods. The qualitative data were obtained by observing the learning activities of the participants. The quantitative data were gathered by a questionnaire on a 5-point Likert scale. The questionnaire consisted of items to measure the factors and indicators of learning approaches. The questionnaire responses in four sections were scored and weighted in SWOT analysis.

3.2. Research Variables

The dependent variable of this study was the efficacy of the learning method, while the independent variables were the aspects to influence the efficacy of the learning method. The independent factors including hybrid learning (M1), conventional learning (M2), and the performance rate (M3) were categorized into good (P1), average (P2), and poor (P3) groups. Gender was included as another independent variable.

3.3. Hypothesis

We employed an analysis of variance (ANOVA) and the F-value to assess the impact of the factor. Based on the computations, the null hypothesis (H0) and alternative hypothesis were tested at a certain significance level (α) (

Table 1. Research hypotheses of this study.

Purposes	H0	H1
To test the effect of the influence of learning method factors	H0: Mi = 0 (There is no effect of i-level learning method factors on the learning effectiveness, which is measured by the level of understanding)	H1: Mi ≠ 0 (There is an effect of i-level learning method factors on the learning effectiveness, which is measured by the level of understanding).
To test the effect of the performance rate factor, the factor level is nested in the learning method factor	H0: Pj(i) = 0 (There is no effect of the j-level performance rate factor nested in the i-level learning method factor on the learning effectiveness, which is measured by the level of understanding)	H1: Pj(i) ≠ 0 (There is an effect of the j-level performance rate factor nested in the i-level learning method on the learning effectiveness, which is measured by the level of understanding).
To test the effect of the gender factor which is the factor level nested in the learning method and performance rate factors	H0: Gk(ji) = 0 (There is no effect of the k-level gender factor nested in the j-level performance rate factor and the first-level learning method factor on the learning effectiveness, which is measured by the level of understanding)	H1: Gk(ji) ≠ 0 (There is an effect of the k-level gender factor nested in the j-level performance rate factor and the i-level learning method factor on the learning effectiveness, which is measured by the level of understanding).

).

3.4. Data Analysis

A post hoc test or ANOVA and the Scheffe and Tukey tests were used to determine the impact of hybrid learning and identify the links between the factors and response variables. SWOT analysis was conducted to ascertain the necessary strategy and offer recommendations for the Performance Measurement and Improvement course. We reviewed any deficiencies, barriers, and challenges encountered in this study.

4. Results and Discussion

4.1. ANOVA

An ANOVA was conducted on the data of the learning method, performance rate, and gender factors. The criteria for a significant effect were a p-value less than 0.05 and an F-value greater than 1.95. The learning method and performance rate factors showed p-values of 0.002 and 0.000 and F-values of 9.33 and 19.33. However, gender had no significant impact on the knowledge level response (p-value of 0.126 and F-value of 1.67) (Figure 1).

To enhance understanding and learning efficacy, the learning approach and performance metrics are essential. Students must attain comparable levels of learning effectiveness. These findings align with earlier studies which found that the greater the interactivity of a learning approach, the higher the student’s understanding of information. To optimize the advantage, a hybrid learning model needs to be deployed to attain all the learning objectives.

Figure 2 illustrates the division of the performance rate at three levels, each with its own categorization and values. These classifications and limits were determined based on the rating scale for all aspects of learning performance. A low performance rate was defined as a total value below 29, a medium rate as a value between 30 and 34, and a high rate as a value above 35. The performance rate component and the amount of knowledge demonstrated a direct correlation, whereby a greater performance rate corresponded to a better level of understanding and increased learning efficacy. Figure 2 also depicts the variations in the impact on the level of understanding at various levels of the performance rate. To obtain optimal learning effectiveness and understanding, students must have a high performance rate. Consistently attending learning sessions according to the directions and guidelines of the lecturer enhances the students’ success rate. The consistent pattern results in excellent performance to attain the best possible outcome [32]. An individual with a high level of performance is more open to receiving feedback and effectively enhances their learning process without facing obstacles.

Figure 3 depicts the correlation between the learning method and factors influencing the rate of performance. There is no interaction between them, as indicated by the lack of a line intersecting the two components. The impact created by the learning approach and performance were identical and varied with the level of understanding. Such results illustrated the efficacy of the mixed learning strategy with each learning method corresponding to an increased performance rate.

4.2. Post Hoc Test

Since the distinction between factors with less than three levels was evident, a post hoc test was conducted on variables that affected the response variable (i.e., the level of knowledge) at more than three levels, specifically the performance rate factor. A summary of the post hoc test results for the performance rate from the Scheffe and Tukey procedures is presented in

Table 2. Recapitulation of post hoc test results for performance rate.

Treatment Comparison Pair	Average Difference	Conclusion
Low–Average	−1.0103	Real Different
Low–High	−1.9011	Real Different
Average–High	−0.8908	Real Different

.

Factors at all levels exhibited variations and impacts on the knowledge level. The average difference value represents the discrepancy between the mean values of the subsets within each factor level. The average performance rate was 1.0103, which was higher than the average low performance rate. The average high performance rate was 1.9011, which was higher than the average subset of the low performance rate. The high-performance rate was 0.8908, which was higher than the average performance rate. There was a positive correlation between the degree of understanding and performance rate. The related factors encompassed the cognitive abilities, level of engagement, proactivity, and receptiveness of pupils in the learning process.

4.3. Qualitative SWOT Analysis

The SWOT strategy was developed by analyzing the external and internal environments of the Industrial Engineering study program and synthesizing the relevant conditions. The strategic focal points were categorized based on the SWOT components. The approach was categorized as S-T, S-O, W-T, and W-O strategies. The factors in the SWOT strategy were used to create a robust and effective strategy that aligns with an organization’s goals and fulfills its optimal requirements.

The S-O approach results in the maxi–maxi strategy. In this approach, the strategy is to be in equilibrium, where the organization’s strengths may be effectively utilized to capitalize on every opportunity and enhance its position with ease. The S-T approach consists of the maxi–mini method. This strategy remains robust with the ideal capacity to effectively neutralize any current dangers. In the W-O approach, challenges are faced in capitalizing on opportunities due to its internal weaknesses. However, these weaknesses can catalyze transformation and be used to take advantage of existing chances. The W-T or mini–mini strategy is used to integrate weaknesses and threats. This strategy is ineffective due to unfavorable conditions within the corporation, and this approach needs to be revised.

4.4. Quantitative SWOT Analysis

The Industrial Engineering study program had the W-O approach as the mini–maxi strategy or divestment–investment principle. Thus, it was necessary to change the strategy. The coordinate of (−2.32, 0.39) indicated the combination of negative and positive values in quadrant III. Thus, when implementing hybrid learning, the Industrial Engineering study program had flaws to be addressed. Hence, it is imperative to modify the existing strategy to address the flaws and capitalize on emerging opportunities. Even with vulnerabilities, possibilities for divestment were found. Therefore, suggestions to adopt this integrated learning approach were made for immediately and in the long term. In the short term, it is necessary to enhance the preparedness of current resources, while long-term implementation is demanded to establish a learning system in online education.

5. Conclusions

A suitable educational approach leads to a productive educational setting, thereby fostering engaged and student-focused learning. Based on the results of this study, the hybrid learning method with the W-O or mini–maxi strategy based on the divestment–investment concept can be a successful approach to learning. For the introduction of hybrid learning, the Industrial Engineering Department must develop a performance-tracking tool for students and establish a website as a resource for literacy. Once hybrid learning is firmly established at Atma Jaya Catholic University of Indonesia, additional research can be conducted to assess the long-term impact on student performance and understanding.