A Comparative Study on Teaching Methodologies Applied in Engineering and Manufacturing Process Subjects during the COVID-19 Pandemic in 2020 and 2021

Abstract

Specific disciplines in engineering, such as manufacturing processes, require students in their academic stage to pay special attention, given the possible changes that may affect the acquisition of competencies. In an environment of uncertainty, such as a global pandemic, teaching must adapt without losing the effective delivery of content to students. The health and safety measures applied during the first months of the pandemic led to a different type of teaching to that which had customarily been applied, such as synchronous and asynchronous methodologies defined by the university’s governing bodies, where face-to-face and online methodologies coexisted in the same academic year. All of this avoided interrupting the academic year. This paper studies the results achieved in this uncertain environment, extends them and compares them with the following year, where only the face-to-face methodology was applied to the students enrolled in Manufacturing Processes 2 at the Centro Universitario de Mérida within the Bachelor’s Degree in Design Engineering and New Product Development (Grado en Ingeniería en Diseño y Desarrollo de Nuevos Productos -GIDIDP-). An analysis of variance (ANOVA) was applied to the data obtained to locate the significant differences between the samples taken in the first year with online and face-to-face teaching methodologies and those taken in the second year with an exclusively face-to-face methodology. When comparing the results, maintaining face-to-face teaching proved essential, as it contributes towards achieving better marks or maintaining the level. However, online methodologies also help as an additional tool to acquire other knowledge and specific skills in these technical engineering subjects, specifically those dealing with the manufacturing processes addressed in this study.

1. Introduction

The advancement of information and communication technology (ICT) has opened up a viable solution for university teaching in the uncertain environment of the COVID-19 pandemic. The market offers several devices, such as smartphones and tablets, among others, which provide permanent connectivity for their users, and are becoming increasingly popular in the educational field [1].

The use of these technological devices in education has given rise to the concept of mobile learning or m-learning, which benefits from new technologies and mobile devices [2]. Electronic devices and technology-supported teaching could enhance student learning [3], and modern smartphones have more data processing power than the big computers of a few years ago. The internet and cloud storage have profoundly changed the way many professionals worldwide work, creating a new way of working in schools and universities. It is advantageous for students to be taught from anywhere and at any time; it could help in acquiring competencies, as younger adults/university-age individuals use a greater breadth of technologies [4]. As Sevillano and Vázquez-Cano state in their paper [5], mobile devices (e.g., smartphones) belong to users, and are available to them around the clock, favoring the adaptation and access to content according to individual needs and competencies. However, with this study, the authors intend to confirm that technology is an additional tool to support traditional face-to-face teaching, providing better results as per the authors’ experience.

There is an added difficulty in specific engineering disciplines, i.e. manufacturing processes, which usually require extra effort by the students [6]. They tend to be subjects with extensive content, usually applied practically to solve real industrial problems. The measures that contribute toward acquiring competencies include improving online content and assessment, which reaffirms that teachers of this type of subject focus more on face-to-face teaching, although this way of learning and teaching is becoming increasingly obsolete [7,8].

At a professional level, and according to [9], there are numerous regulated professions equivalent to current engineering studies, which base their regulations on acquiring technological competencies and skills. Based on this premise, mobile devices must be included in the teaching and learning process, opening up a range of educational improvements that must be considered [10,11,12,13], whether face-to-face or virtual methodologies are applied.

Among other things, class time would be optimized, and student motivation would increase [14].

In addition, technologies can simplify assessments by providing teachers with more immediate progress indicators, which are necessary in technical subjects or subjects with theories that need to be translated into practice for problem-solving (e.g., the engineering subjects discussed in this paper). Thus, many mobile applications on the market facilitate flexibility, communication, access to information, and the ability to create and assess content by students and teachers. This extensive development of educational applications is constantly evolving due to the demand generated by users [15].

The authors carried out this comparison study based on their experiences, from two academic years, where all teaching methodologies proposed and permitted by the university were applied simultaneously, motivated by the COVID-19 pandemic. The authors analyzed two consecutive academic years (2019–2020 and 2020–2021) to obtain sufficient data and to affirm that there is a certain correspondence between the assimilation of competencies by the students measured through their results and the methodologies applied in teaching during the academic year. During the first year of this study, the methodologies used were predominantly technological and based on online learning and teaching due to the COVID-19 pandemic.

2. Background

Numerous studies have analyzed [16] the relationship between learning strategies and learning outcomes in university students. Some authors link the learner’s personality to their results [17,18], others to motivation [19], and others to skills, such as self-regulation of learning [20,21] or gender, and its differential impact on the strategies used [22,23,24].

In this paper, and for specific engineering subjects (e.g., manufacturing processes and the like), face-to-face teaching dominates, but has been partially replaced by online methodologies with the COVID-19 pandemic. For decades, these subjects have been taught face-to-face with some student success, but there is no substantiating data on how other methodologies influence students and their final marks. Although online activities have been incorporated periodically through virtual university campuses, and lately, increasingly more educational projects [25] have been trending toward other types of online teaching or monitoring, part of them still require high classroom attendance.

This change has led to a renewal of traditional teaching methodologies and “reflections” by teachers (on a personal level) and the teaching-learning process [26]. In this sense, the work by Jenaro et al. [16] casts doubt on the unanimous evidence, and other authors mention that there is no clear correspondence between traditional methodologies, the most innovative methodologies, and the qualifications and results achieved by students [27]. For example, two studies can be highlighted [28,29], in which university students value the usefulness of lecture sessions versus other more active learning methods. Contrary to expectations, students highlighted lecture sessions as valuable sources of learning, independent thinking, and problem-solving skills, which are typically found within the competencies to be developed by engineering students. Other studies reveal no significant differences between active vs. passive methodologies, student satisfaction, and final results [30].

However, universities and teachers are gradually paying more attention to virtual learning platforms that are useful as a repository of learning materials and elements and allow for proper organization, planning, and monitoring of subjects, and easier planning and assessment, in regard to competencies [31]. This is probably why a learning management system (LMS) has become a common and essential tool in education [32]. Most schools worldwide use these systems to achieve better quality education and to provide new “learner-centered” teaching [33].

This trend accelerated in the first quarter of 2020 due to the coronavirus disease 2019 (COVID-19) global pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). At the beginning of March 2020, the government ordered the confinement of the population (e.g., to remain in their homes) to help stop the spread of the virus. This measure was approved via the state of alarm of 14 March 2020 [34] and subsequent measures of 20 March, 2020 [35] that strengthened the confinement measures, paralyzing non-essential activity in the country, including educational activity [36].

Suspending face-to-face teaching has forced quick and immediate adaptations to other types of teaching and evaluations. This step, from face-to-face learning to online learning, was followed by teachers and students immediately with no prior planning. In the same month of March, the Spanish Network of University Quality Agencies (REACU) informed all quality assessment agencies that, in their assessments, they should recognize the changes that each university had to introduce to adapt teaching and assessment during the pandemic, under the European, national, and regional premises [37].

Later, in April, REACU published new communication guidelines, to adopt assessment methodologies using the resources available and align them with the quality standards of the European Higher Education Area (EHEA) [38], based on these general criteria:

• Use of different assessment methods, based on continuous assessment techniques and individual tests;
• They must allow assessing the acquisition of skills and learning outcomes in the subjects;
• Assessment criteria and methods, including the assessment criteria, should have been made public beforehand [37] and included in the teaching guides of the subjects as addenda [39].

With this background, the students’ previous learning experiences prove to be significant. They had to shift from a face-to-face method to a completely virtual one. Although currently, most students are knowledgeable about technology (Generation Z) [40], a communication channel was needed to explain the instructions to follow the off-campus teaching method.

According to studies based on objective data, such as abandonment rates or the percentage of students who complete studies [41], disciplines related to engineering are challenging and more difficult for students and teachers. The age groups of these students (in these disciplines) have increased; technological development could be an element that essentially defines the characteristics of the labor market [42].

Due to the urgency of the situation, the change (to remote learning) had to be implemented in the middle of the semester, which meant greater efforts by teachers and students. Moreover, there was no planning regarding technological means. Therefore, acquiring other digital skills to address the appropriate educational levels were implemented from the onset, which could translate into, according to some authors, the increase of the following three inequalities [43]:

• Access to electronic devices and internet connection;
• Time of use and the quality of it (sharing device by several family members);
• Teachers and student competencies for the proper use of digital platforms for educational purposes [25].

Many authors see assessment as one of the difficulties that the teaching team encounters; it has become a critical point for online teaching [44].

In this scenario, full of difficulties and challenges, the teachers adapted the content to be taught synchronously and asynchronously and created new content or adapted existing content to a virtual environment. The interaction between students and teachers is the first concern in this new purely digital approach since it is necessary to supplement the advantages that human interaction offers in face-to-face classes [45].

For these reasons, this paper demonstrates—with objective data obtained in off-campus teaching methodologies—that students have successfully acquired the skills. We compare face-to-face teaching from the first and the second academic years, which only had face-to-face teaching.

The conclusions and results of this paper will be used to give a voice to the authors who believe virtual teaching will be the next teaching–learning model. According to Torrecillas [46], due to the pandemic and the forced introduction of online classes, Spanish public universities did not have the "experience factor" incorporated into their backgrounds. In fact, in percentage terms, only 3.45% of public universities used an off-campus teaching method in the 2018–2019 academic year, compared to 62.54% of private universities [46].

This paper shows the methodology and the marks students achieved in the final assessment after applying the three methodologies in the first academic year of this comparison (2019–2020), and the methodology applied in the following year (2020–2021) in Manufacturing Processes 2 (MP2), to establish a comparison on whether there are significant differences between the first year and the second (for 39 students in the first year and 28 in the second).

3. Materials and Methods

3.1. Selection of Participants

The study participants were three lecturers from the Centro Universitario de Mérida, currently teaching in the Bachelor’s Degree in Design Engineering and New Product Development (Grado en Ingeniería en Diseño y Desarrollo de Nuevos Productos -GIDIDP-) program, with the following distributions of the teaching of the subjects under study:

• First teacher: 40% of MP2 (Topics 1, 2, 3 and 4);
• Second teacher: 20% of MP2 (Topics 5 and 6);
• Third teacher: 40% of MP2 (Topics 7, 8, 9 and 10).

In order to keep "stable" the effects that the participation of the three professors (i.e., in teaching the subjects) could have–from the beginning of the project in the 2019–2020 academic year, an agreement was reached among them, to maintain the teaching carried out by each professor from one academic year to the next; that is, to maintain not only the same subjects taught by each professor, but also the content of the subjects.

The sample size per academic year is distributed as follows (see

Table 1. Sample size by academic year.

Year 2019–2020	Year 2020–2021
39	28

):

3.2. Methodologies Used

Given the need to assess the specific and transversal skills related to engineering and manufacturing processes, the teachers–participants determined that immediate data on understanding and following a topic through technological resources would be part of the immediate feedback. In addition, these tests could also be used to assess other educational values, such as subject-specific and transversal skills.

Another relevant issue was acquiring skills in using the platforms enabled to monitoring the teachers, and the virtual campus became the primary communication tool in the first year of this comparison. Fortunately, the widespread use of this tool facilitated rapid adaptation of the content and assessment activities.

The requirements approved by the University of Extremadura on 12 April and included in the “Addendum of Academic Criteria for the Adaptation to Off-Campus Teaching during the State of Alarm due to COVID19” had to be met for the first academic year of this comparison (2019–2020), and to apply the new methodologies. This document specifies the need to transform the teaching activity into an off-campus teaching format, adapting teaching methodologies and assessment systems (as well as “acquiring skills” [47]). The three methodologies proposed in the addendum above were applied:

• Face-to-face teaching. Teaching activity that requires the student be in the classroom, where learning is directed by teachers, who, in their most traditional roles, explain, clarify, and communicate ideas and experiences [47].
• Synchronous virtual class. Teaching activity developed through an interaction between teachers and students, which requires the coincidence of both simultaneously (synchronous presence), using technological communication tools, such as, for example, chat, videoconference, among others [25,47].
• Asynchronous virtual class. Teaching activity in which teachers and students interact flexibly at different times. Some teaching activities in this category are: reading documents, viewing videos of theoretical content, viewing tutorials on problem-solving and/or practical cases [47].

For the second academic year, the return to normality and the elimination of some of the restrictions imposed by the pandemic led to the reinstatement of face-to-face teaching, while maintaining some online and synchronous activities, such as constant communication and the presentation of certain practical activities, seminars, and follow-up activities that carried weight in the final assessment.

Manufacturing Processes (MP2) is divided into 10 topics (see

Table 2. Distribution of the topics of MP2.

Teaching	Year 2019–2020	Year 2020–2021
Face to face	1, 2, 3, 4	1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Synchronous	5, 6	-
Asynchronous	7, 8, 9, 10	-

for classification) to apply the different methodologies described above. The first topics were already covered by face-to-face teaching; the rest were divided between synchronous and asynchronous teaching during the first academic year, while in the second year, all teaching was face-to-face.

Classes have been taught traditionally through face-to-face teaching, with the teacher and students in the classroom, making the presentations and providing the appropriate explanations directly, for topics 1, 2, 3, and 4 in the first year, and all topics in the second year. With the synchronous teaching method, replacing face-to-face teaching with online teaching was suggested, with direct interaction between the student and the teacher, through a virtual classroom and by videoconference, for topics 5 and 6 (see

Table 3. Off-campus teaching methodologies.

Teaching	Interaction
Synchronous virtual classes	Live lecture classes Specialized presentations Problem resolution Virtual practices Collaborative group work coordinated by the teacher
Asynchronous virtual classes	Reading documents Creation of short videos uploaded to the virtual campus Generation of explanatory PDF with questionnaires

). Moreover, in asynchronous teaching, students were asked to access the content and explanations on demand through the virtual campus of the subject. Each student could follow the teaching on the day and time most convenient for them in topics 7, 8, 9, and 10. It is worth noting that, in the part of the course, which was taught asynchronously and synchronously during the first year of the study, the use of videos, animations, and demonstrations as subject matter was very helpful.

The authors paid particular attention to the first year of this comparison as the criteria for assessment and valuation of students’ skills had to be altered due to the three methodologies being applied. Adjustments had to be made to adapt them to the new proposed methodologies and teaching activities to meet the criteria and assessment requirements approved by the UEx. In this sense, alternative assessments activities were proposed to be carried out electronically through the virtual campus. With the final tests in the continuous assessment mode, the content taught in person had not reached 50% (see

Table 4. Percentages of teaching methodologies.

Type of Activity	Percentage 2019–2020	Percentage 2020–2021
Activities related to the synchronous virtual class: Solving questionnaires, attendance, etc.	20%	0%
Activities related to the asynchronous virtual class: viewing videos, reading documents, solving questionnaires, etc.	20%	0%
Evaluation activities developed in the previous face-to-face training	20%	40%
Final test	40%	60%

), so the final tests should not exceed this percentage in the final mark. For global assessments, the final tests would be 100%. For the second year, the criteria used before the pandemic were applied in a 100% face-to-face mode.

All these changes were notified to the students through the recommendations contained in the document “Planning of the MP2 students”, which also included the programming of the subject for the rest of the semester, so that the students could clearly know how to proceed in non-face-to-face teaching.

Topics 7, 8, 9, and 10 in PDF format and videos with detailed explanations of their content would be provided on the virtual platform. These last topics would be reinforced with assessment questionnaires, which later served to contrast the results to analyze the impact of each teaching methodology applied.

Due to the uniqueness of the entire process, two exam protocols were drawn up for the June and July calls. These were collected in two similar documents entitled “Examination protocols for MP2”, but are not relevant for this paper, because they have no relevance to the discussion of results.

The results obtained in this comparative study are from the final exam of the subject in each academic year, as this is the most objective and reliable tool we have to draw conclusions.

Finally, during the second academic year of this study, the level of the final test was raised since the teachers wanted to assess not only the importance of the teaching methodology applied, but also the level of demand regarding these methodologies. A bank of exam questions (see

Table 5. Sample question bank according to the level of difficulty.

Topic	Difficulty Levels
Topic 1. Introduction to plastic metal forming	High difficulty (16) Medium difficulty (16) Low difficulty (12)
Topic 2. Volumetric forming by plastic deformation.	High difficulty (8) Medium difficulty (12) Low difficulty (12)
Topic 3. Sheet metal fabrication.	High difficulty (8) Medium difficulty (12) Low difficulty (8)
Topic 4. Powder forming (sintering).	High difficulty (8) Medium difficulty (12) Low difficulty (8)
(...)	(...)
Topic 10. New technologies in industrial production.	High difficulty (8) Medium difficulty (12) Low difficulty (8)

) was obtained from the teaching staff, classified into low, medium, and high difficulty questions. For the development of the exam in the first year, questions of low to medium difficulty were used; for the exam in the second year, questions of medium to high difficulty were proposed. For example, in topic 1 of the subject, the teachers–participants had 16 high difficulty questions, 16 medium difficulty, and 12 low difficulty.

It is necessary to make an objective analysis of the difficulty of the exam and its questions in order to make an objective comparison in the results and discussions with the success rate, to see if the difficulty really had an influence on the assimilation of competences by the students. The criteria of difficulty according to the classification is shown in the

Table 6. Rating of difficulty.

Difficulty Levels	Difficulty Criteria
High difficulty	+14%
Medium difficulty	+7%
Low difficulty	+0%

.

These percentages of difficulty have been extracted from previous years and exams taken by students according to the types of answers presented in the final exam.

In this environment, and although the results will be discussed in-depth in the following section of this paper, the results achieved in the final test, dividing the answers into correct answers, blank answers, and incorrect answers or errors, are shown (see

Table 7. Summary of results year 2019–2020.

Topics	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10	Total
Correct	65%	76%	75%	65%	56%	65%	74%	54%	45%	43%	63%
Blank	11%	7%	5%	12%	17%	16%	6%	22%	26%	33%	15%
Fail	24%	17%	20%	23%	27%	19%	20%	24%	29%	24%	22%
											100%

and

Table 8. Summary of results year 2020–2021.

Topics	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10	Total
Correct	61%	68%	60%	37%	59%	68%	65%	35%	54%	51%	56%
Blank	10%	9%	13%	28%	9%	8%	16%	21%	12%	16%	14%
Fail	29%	23%	27%	35%	32%	24%	19%	44%	34%	33%	30%
											100%

).

The tables above show a clear trend towards lower results in their success rate in the second year, due to the lower number of correct answers per subject. The sample for the second year is also slightly smaller, as mentioned in the introduction to this paper, with 46 students in the first year and 27 in the second year. Therefore, the total number of questions answered was 1950 for the first year and 1080 for the second year.

The results obtained in the previous tables show a decrease in the success rate of students from the first year 2019–2020 to the second year 2020–2021, being 63% in the first year and 56% in the second year, a decrease of seven percentage points. If we take a closer look at the evolution in each academic year, and by subject, we can also see a lower success rate, as can be seen in the graphs below (see Figure 1).

4. Results and Discussion

4.1. Comparison of Teaching Methodologies

The study of results begins by comparing the three teaching methodologies, two online and one face-to-face, applied in the first year. With this, relevant data about whether it significantly influences acquiring skills or students’ final marks were gathered. At the same time, these results are contrasted with those achieved in the following year of fully face-to-face teaching. The table below shows the overall success rate of the subject in the two years of the study (see

Table 9. Final test results and success rate.

Categories	Year 2019–2020	Year 2020–2021
Students with grades > 5	25	11
Students with grades < 5	14	16
Success rate (>5)	64.10%	44.74%

).

A lower success rate is observed in the 2020–2021 academic year (44.74%) due to the increased difficulty of the final test of the subject by selecting more difficult questions than in the previous year, as mentioned in the methodology section. The following table shows how the individual scores for the questions and topics vary downwards from one year to the next (see

Table 10. Sample of grades by topics and questions. Year 2019–2020.

	Topic 1 Question 1	Topic 2 Question 6	Topic 3 Question 14	(...)	Total	Total T1–T4	Total T5–T6	Total T7–T10
Student 1	0.2	−0.06	0.2	(...)	6.68	3.39	1.74	1.55
Student 2	0.2	0.2	−0.06	(...)	7.69	3.36	1.74	2.59
Student 3	0.2	−0.06	−0.06	(...)	6.88	2.58	1.48	2.82
Student 4	−0.06	−0.06	0.2	(...)	5.44	1.86	1.22	2.36
Student 5	0.2	0.2	0.2	(...)	8.21	3.88	1.74	2.59
(...)	(...)	(...)	(...)	(...)	(...)	(...)	(...)	(...)
Student 39	−0.06	0.2	−0.06	(...)	5.38	2.79	1.04	1.54

and

Table 11. Sample of grades by topics and questions. Year 2020–2021.

	Topic 1 Question 1	Topic 2 Question 6	Topic 3 Question 14	(...)	Total	Total T1–T4	Total T5–T6	Total T7–T10
Student 1	−0.04	−0.04	0.25	(...)	4.11	2.49	0.17	1.46
Student 2	−0.08	−0.08	−0.08	(...)	3.28	0.04	0.81	2.44
Student 3	0.25	0.25	−0.08	(...)	6.46	2.93	1.63	1.90
Student 4	0.25	−0.04	−0.08	(...)	7.83	3.32	1.24	3.27
Student 5	0.25	0.25	−0.08	(...)	8.31	3.32	1.24	3.32
(...)	(...)	(...)	(...)	(...)	(...)	(...)	(...)	(...)
Student 28	0.25	0.25	0.25	(...)	5.26	2.14	1.28	1.84

). These tables reflect the scores per question. The 0.2 and 0.25 values correspond to correct answers, −0.03 and −0.04 values correspond to blank answers, and −0.06 and −0.08 values correspond to incorrect answers.

The score differences are determined by the number of questions in the exam, e.g., if we have an exam with 40 questions, as in 2020–2021, the correct answer will be +0.25 points; and if the exam has 50 questions, the correct grade will correspond with a weighting of +0.2. The same would apply to incorrect answers (−0.06 and −0.08) and blank answers (−0.04 and −0.03). Although the tables are not shown in their entirety as they are extremely long, in the extract presented in this article, it can be seen how this scoring system provides a limited penalty, and is perfectly compatible with obtaining high marks and few failures, as can be seen in the TOTAL column of the following tables.

The tables below (

Table 12. Final test results by topics Year 2019–2020.

	Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic 7	Topic 8	Topic 9	Topic 10
	Face-to-Face Teaching				Synchronous		Asynchronous
Success	126	236	146	100	88	151	116	106	86	66
Blank	16	16	6	6	23	2	6	26	25	37
Failure	47	51	38	38	39	42	29	45	55	37
Success rate	66%	77%	76%	69%	58%	77%	76%	59%	51%	47%

and

Table 13. Final test results by topics Year 2020–2021.

	Topic 1	Topic 2	Topic 3	Topic 4	Topic 5	Topic 6	Topic 7	Topic 8	Topic 9	Topic 10
	Face-to-Face Teaching				Face-to-Face Teaching		Face-to-Face Teaching
Success	66	73	65	40	64	73	70	38	58	55
Blank	11	10	14	30	10	9	17	23	13	17
Failure	31	25	29	38	34	26	21	47	37	36
Success rate	68%	60%	37%	59%	68%	65%	35%	54%	51%	68%

) contain the total summaries of correct, blank, and incorrect answers by subject, methodology, and academic year. The success rate is also shown in such a way to reveal how some subjects yield more success than others. Specifically, in the 2019–2020 year, the face-to-face subjects stand out, followed by the subjects that correspond to the synchronous methodology, and finally the last subjects that correspond to the asynchronous methodology. In the year after, i.e., 2020–2021, there is an apparent equality that will be studied in the ANOVA analysis applied below.

The results tables show that more than half of the answers were correct, thanks to the face-to-face method in practically all cases. Only topic 10 of the first year had a success rate of 47%, and topics 3 and 7 of the second year, with rates of 37% and 35%, were below average.

4.2. ANOVA Analysis

In the statistical analysis, the ANOVA test was used to determine if the populations had the same mean value or if there were differences between them. For this purpose, as stated in Section 3.1, the data obtained during the academic years 2019–2020 and 2020–2021 have been used. The results obtained are shown in

Table 14. Number of questions per subject in the final examination in both academic years.

Academic Year	Sample Size	Means	Standard Deviation
2019–2020	39	5.38	2.22
2020–2021	28	5.20	1.92

and

Table 15. Descriptive statistics from the data experiment for different methodologies.

Methodologies	Academic Year	Sample Size	Means	Standard Deviation
Face-to-face vs. face-to-face	19–20	39	2.79	1.00
	20–21	28	2.15	0.88
Face-to-face vs. synchronous	19–20	39	1.04	0.55
	20–21	28	1.26	0.53
Face-to-face vs. asynchronous	19–20	39	1.54	0.96
	20–21	28	1.81	0.92

.

Table 14 shows no significant differences comparing the two academic years studied, which have very similar values for the averages achieved by the participants, 5.38 for the first year and 5.20 for the second (see Figure 2).

With the data obtained, a more in-depth comparison of the methodological blocks applied proved interesting. In the comparison by methodologies, and as seen later in the ANOVA study, there are specific differences between the means (see Table 15), mainly in comparing the block corresponding to the face-to-face methodology (see Figure 3). The face-to-face and synchronous mean values offer similar values (1.04 and 1.26); the face-to-face and asynchronous averages did not offer significant differences, either (1.54 and 1.81), but they did in the face-to-face and face-to-face comparison (2.79 and 2.15). These differences will be examined in more detail below.

Table 16. ANOVA results for a 0.05 level of significance for the means per academic year.

Origin of Variation	DF	Sum of Squares	Mean Square	F-Value	Prob > F	F Critical
Model	1	0.52	0.12	0.12	0.73	3.99
Error	65	287.09	4.42
Total	66	287.62

and

Table 17. ANOVA results for a 0.05 level of significance for the means for the different methodologies per academic year.

Methodologies	Origin of Variation	DF	Sum of Squares	Mean Square	F-Value	Prob > F	F Critical
face-to-face vs. face-to-face	Model	1	6.78	6.78	7.47	0.01 *	3.99
	Error	65	58.96	0.91
	Total	66	65.74
face-to-face vs. synchronous	Model	1	0.77	0,77	2.59	0.11	3.99
	Error	65	19.25	0.30
	Total	66	20.02	p
face-to-face vs. asynchronous	Model	1	1.19	1.19	1.34	0,25	3.99
	Error	65	58.49	0.89
	Total	66	59.68

* Population means are taken to be significantly different at the significance level 0.05.

list the p-values obtained in the analysis of variance. The values of p must be compared with the significance level to assess the null hypothesis to determine whether the differences between the mean values are statistically significant. A significance level $\alpha$ = 0.05 has been set for this study. If the p-value is less than or equal to the significance level, the null hypothesis is rejected, and it can be concluded that not all population means are equal. Otherwise, if the p-value is greater than the significance level, there is insufficient evidence to reject the null hypothesis that the population means are all equal.

The results obtained from the one-way ANOVA for the analysis of the mean scores obtained by each class can be seen in Table 16. The ”F” value of the sample (2019–2020 and 2020–2021) is 0.12 and lower than the critical F of 3.99; p-value = 0.73 (p < 0.05). Therefore, the null hypothesis of equality of means is not rejected, and there is no difference between the groups.

As seen in Table 17, if the “face-to-face vs synchronous” methodology is considered, the ANOVA analysis of variance shows no significant differences, obtaining a p = 0.11. The same is true for the dependence of the face-to-face vs. asynchronous methodologies with a value of p = 0.25, which means that there is no direct relationship between the types of methodologies concerning the academic results obtained in the academic years 2019–2020 and 2020–2021.

Methodologies Face-to-Face vs. Face-to-Face Per Academic Year

The results obtained from the one-way ANOVA for the analysis of the mean scores obtained by each class can be seen in Table 17. The ”F” value of the sample (2019–2020 and 2020–2021) is 7.47 and higher than the critical F of 3.99; p-value = 0.01 (p < 0.05). Therefore, the null hypothesis of equality of means is rejected, and there is a difference between the groups. Subsequently, a multiple comparison analysis (post hoc test) was carried out using Fisher’s exact test (see

Table 18. Fisher test for means comparison with significance level 0.05.

Academic Year	diff	Test Statistic	p-Value	Sig.
2019–2020 vs. 2020–2021	0.64	2.73	0.01	Yes

) to examine differences in depth. As a result, a direct correlation was observed between the face-to-face methodologies in the different academic years.

The results obtained in Table 18 show the differences when comparing the face-to-face methodologies in the two years of the study. These differences, as seen above, do not appear in the other methodologies applied. With the same number of topics, the same teaching staff, teaching the thematic block and the same methodology—the only difference between one year and the next is the increase in the difficulty of the questions posed in the final test, as explained in Section 3.2. Methodologies used (see Table 5).

If we compare the data obtained in Table 7 and Table 8, extracting only the data on the percentage of success in the first four topics, where the significant difference is obtained, and this is compared with the difficulty criteria in Table 6, we can see that there is an average difference of 13.75 percentage points, which coincides with the added and estimated difficulty of 14% in the final exam according to Table 6.

5. Conclusions

The conclusions drawn by the authors of this study are several. The first is that the students adapted quickly to the different methodologies, achieving similar marks during the first academic year in which they were applied and passed the course satisfactorily, acquiring the corresponding skills.

Based on the results of the first year, there are minor deviations between the methodologies applied, mainly due to the interest shown by the students in the face-to-face methodology and subjects traditionally used in previous years. The deviations that occurred in the last topics of the course on asynchronous methodology were corrected by assessment questionnaires during this period. Of the 39 students analyzed, 25 passed the final test satisfactorily (64.10%), having participated in the three methodologies offered by the teachers.

In the second year of this comparison, 11 of the 28 students tested passed the final test satisfactorily. There is a clear decrease in the success rate from 64.10% in the previous year to 40.74% due to an increase in the final test level, due to the added difficulty of the questions. However, in comparing the blocks by methodologies, no significant differences are observed beyond the part corresponding to classroom teaching, revealing the importance of this methodology to ensure the transmission of content and assimilation of competencies by the students.

Therefore, in the circumstances, the authors do not recommend fully and abruptly replacing face-to-face teaching in engineering subjects with other methodologies in an off-campus nature. Other authors [48] coincide and add “unless supported by complementary activities to web-based learning, and in constructing a learning environment that fosters communication between students and teachers, connecting learning with experience, enhancing teamwork and taking advantage of the information and knowledge that ICTs provide”.

However, the global trend is to include hybrid teaching between traditional, face-to-face and technological teaching, including synchronous and asynchronous methodologies. Therefore, the conclusions presented in this study could be affected or would require further approximation or study with larger populations.

The authors will continue to collect data for further comparisons, while incorporating new LMS learning tools and teaching methodologies in engineering-specific subjects. To arrive at a future where the teaching methodology can be applied online in its entirety, provided it does not impair the students’ knowledge and assimilation of skills.