Evaluating Competency Development and Academic Outcomes: Insights from Six Semesters of Data-Driven Analysis

Abstract

Competency-Based Education (CBE) has been widely studied since the 1970s, yet it remains innovative due to its challenges across disciplines and cultures. Tecnológico de Monterrey, a Mexican private institution, implements CBE through its Tec21 model, which emphasizes challenge-based learning to develop disciplinary and transversal skills. Since its launch in 2019, Tec21 has generated extensive data, offering an opportunity to assess its performance and ensure quality. This study analyzes data from six academic semesters in the School of Engineering and Sciences to address key quality assurance questions. First, we evaluate whether initially enrolling in a generic area before selecting a specific program improves long-term student outcomes. Second, we examine competency development, identifying challenges in achieving certain skills and their links to dropout rates and module difficulty. Third, we explore the relationship between final grades, module credit allocation, and Tec weeks to assess curriculum alignment with academic performance. Using data from over 550,000 evaluations of 4500+ students, our analysis provides robust quality metrics. Findings suggest that students who start in generic areas perform better long term and highlight the complex interplay between competencies, module characteristics, and academic success. These insights deepen the understanding of CBE implementation and offer recommendations to improve educational strategies and quality assurance within competency-based frameworks.

1. Introduction

Although Competency-Based Education (CBE) has been studied since the 1970s (Spady, 1977), it is still innovative due to the challenges CBE raises in terms of disciplines and cultures (Liu, 2023; Miço & Cungu, 2023; Sa’dijah et al., 2023). An early work published by Spady (1977) defines CBE as a “data-based, adaptive, performance-oriented set of integrated processes that facilitate, measure, record and certify within the context of flexible time parameters the demonstration of known, explicitly stated, and agreed upon learning outcomes that reflect successful functioning in life roles”. Spady identified six crucial components for creating CBE programs: outcomes, time, instruction, measurement, certification, and program adaptability. Implementing and integrating these components is not straightforward (Spady, 1977), and so most recent research is focused on the design, implementation, measurement, and certification of competencies for specific disciplines (Nursing (Liu, 2023), Maths (Sa’dijah et al., 2023), and Foreign Languages (Rustamov, 2023)), prompting universities with CBE programs to continuously evaluate their performance. Although there is plenty of research in CBE (Liu, 2023; Miço & Cungu, 2023; Olivares et al., 2021; Rustamov, 2023; Sa’dijah et al., 2023; Spady, 1977), little has been carried out for providing insights about competency evaluation from large numbers of data (Vargas et al., 2024).

Tecnológico de Monterrey is a Mexican private institution strongly engaged with the evolution of education and its social impact. It has implemented a new CBE pedagogical model for all its academic programs at the bachelor level. This model, called Tec21 (Olivares et al., 2021), makes use of challenge-based learning to have students acquire knowledge and develop skills in their studied discipline, as well as the soft transversal skills necessary for professional flourishing. Under Tec21, students have several different possibilities for developing competencies, which can be carried out at various levels of proficiency. Further, at the moment of enrollment, students may opt to register into a specific subject or into a generic one. For example, a student may enroll directly, say, in the BSc in Computer Science and Technology, or, instead, she could start in the area of Computer Science and Information Technologies, before choosing one of the specific subjects offered in such an area.

Tec21 was launched in August 2019. Since then, it has been a rich source of information. Tec21 has given rise to a huge data repository containing facts about students, instructors, courses, and their outcomes. This repository comes along with a data dictionary and a data teaser. It has been developed and maintained by both the Living Lab and the Data Hub Center at Tecnológico de Monterrey. Having collected millions of data on student competency assessment, these units have launched a request for research proposals, named “Fostering the Analysis of Competency-based Higher Education”1, calling on the scientific community for help in obtaining a general understanding and an in-depth insight about Tec21’s performance. In this paper, we report on the results of some exploratory research that we conducted as an answer to that call. It is based on data science tools.

The aim of this study is to analyze the outcome of the Tec21 model using part of the dataset made available through the call mentioned above. In particular, we use data collected over six academic semesters in the School of Engineering and Sciences. We aim to provide an insight into three fundamental issues: First, whether delaying the choice of enrolling in a long-term program (going through a generic area instead) is beneficial for students or not. Second, whether there exist sub-competencies that are harder to develop, and, if so, how they relate to dropout and the inherent difficulty of the modules where they are meant to be developed. And, lastly, whether there exists a relation among final grades, the number of credits allocated to a formation unit, and whether a unit is part of a specific shorter training period—called a Tec week—or not.

The remainder of this paper is organized as follows: Section 2 gives an overview of existing analyses of CBE. Next, Section 3 details the materials and methods used in our study, including a comprehensive description of the dataset and the feature engineering processes we carried out in previous data analysis. Then, Section 4 presents the study performed in order to validate our three starting hypotheses and discusses the results. Finally, Section 5 describes the conclusions drawn from our investigation and suggests directions for further research.

2. Related Work

Competency-Based Education has been researched for several decades in higher education (HE). CBE provides a useful model for both curriculum and assessment (Marcotte & Gruppen, 2022). However, critical issues still arise from its implementation and evaluation (Henri et al., 2017). Competency evaluation is one such issue at the core of CBE (Vargas et al., 2024).

The assessment of the competencies acquired by students requires a complex analysis that encompasses multiple factors. Such factors vary depending on the subject, competency, and knowledge area (Henri et al., 2017; Moonen-van Loon et al., 2022). Therefore, some authors have published analyses of students’ competency evaluations for particular competency types (Sánchez-Carracedo et al., 2021; Talamás-Carvajal et al., 2024; Thanachawengsakul & Wannapiroon, 2021) and knowledge areas (Moonen-van Loon et al., 2022; Vargas et al., 2024).

Research on competency evaluation usually involves samples of fewer than 1000 students due to the complexities of evaluating large samples and collecting and processing the results (Rasmussen et al., 2016). For example, Larre-Bolaños-Cacho et al. (2020) analyzed the results of training and evaluating the transversal competencies of 11 engineering students throughout a Data Analytics and Cloud Computing course. Similarly, Sangwan et al. (2022) surveyed 30 freshly graduated engineers about their competencies to propose a set of relevant competencies for 21st-century jobs. Also, the research work carried out by Thanachawengsakul and Wannapiroon (2021) presented a proposal that evaluated the competencies of 30 digital entrepreneurs using an online knowledge repository developed in the same work.

Some authors have proposed competency evaluation methodologies and have also reported results for relatively small datasets. A methodology proposed by Sánchez-Carracedo et al. (2021) evaluated the sustainability competencies of engineering students. The methodology assessed students’ self-perception through a questionnaire with 34 questions posed as statements with a four-point Likert scale representing possible responses. The authors collected 221 answers from Informatics Engineering students, finding differences between the students’ improvement perception among different sustainability competencies.

Navarro et al. (2023) analyzed some transversal competencies of 23 Master’s students in Civil Engineering Planning and Management through a sustainability-related case study. The authors proposed and used a methodology based on case studies to assess students’ competencies through the students’ judgments in the field of sustainable design.

Vargas et al. (2024) published a tool that implements a competency assessment framework, facilitating learning analysis and course monitoring. Such a framework implements formulas for evaluating students’ competencies. As a result, it aids professors in assigning traditional numeric and competency evaluations. However, it creates a correlation between the numerical and competency evaluations that need to be analyzed and corroborated with additional studies.

Two research works have analyzed large datasets about competency evaluations, but in a knowledge area different to engineering (Moonen-van Loon et al., 2022; Talamás-Carvajal et al., 2024). Moonen-van Loon et al. (2022) used text-mining techniques to analyze the narrative data of the competency-based portfolios of 1500 students on a Master’s program in Medicine. They implemented topic modeling and sentiment analysis algorithms. Also, Talamás-Carvajal et al. (2024) published a descriptive and correlational analysis of the complex thinking competencies of HE students. The authors used a dataset with 15,903 records corresponding to the evaluations of competencies and their components (sub-competencies). The authors found that critical thinking is crucial in developing complex thinking and some of its components such as scientific thinking and systemic thinking. This work isolated the competency evaluation from the traditional numeric evaluation.

Most of these research works separated the competency evaluation from the traditional numeric evaluation (Larre-Bolaños-Cacho et al., 2020; Navarro et al., 2023; Sangwan et al., 2022; Sánchez-Carracedo et al., 2021; Talamás-Carvajal et al., 2024; Thanachawengsakul & Wannapiroon, 2021). However, analyzing both the traditional numeric evaluation and its corresponding competency evaluation provided by the same professor bridges the gap between both evaluation approaches. Furthermore, having larger datasets with thousands of competency evaluations about multiple students, academic programs, and professors in various semesters and cohorts avoids biases and provides valuable insights about competency evaluation.

3. Materials and Methods

3.1. The Tec21 Pedagogical Model and the Structure of Tec’s STEM Programs

The Tec21 educational model integrates the development and assessment of competencies across all courses. Each course outlines the specific competencies that students are expected to develop and be evaluated on. During the program, students develop three different types of competencies, namely, transversal, area, and discipline competencies. Transversal competencies are essential for all students at the university, not just those in STEM fields. These include skills such as complex thinking, communication, and digital transformation. Area competencies are specific to STEM students and encompass abilities like problem-solving and data analysis. Discipline competencies are tailored to the long-term program a student is enrolled in.

Each competency is a broad, main skill or ability that students are expected to develop. Competencies are composed of multiple sub-competencies. Sub-competencies break down a broader competency into manageable, actionable components, providing a clear, precise framework for instructors to assess. They are, thus, very specific, and, together, they are expected to account for the mastery of the corresponding competency.

The development of each sub-competency is structured into three levels of proficiency: Level A (basic), Level B (intermediate), and Level C (advanced) (Olivares et al., 2021). The summative evaluation of each course includes a competency assessment, where students are required to demonstrate evidence of achieving the proficiency level that the course is designed to develop. As students progress through different academic periods, the model guides them to develop competencies through the sub-competencies, starting from Level A in the initial semesters and progressing to Level C in the final ones.

Figure 1 illustrates the structure of the STEM academic programs, which are organized into three five-week periods per term. During each period, students engage in formative units, which go beyond traditional courses to focus on the development of competencies. These formative units are divided into two main types: subjects and blocks. Subjects resemble typical courses but are specifically designed to facilitate competency development. Blocks, on the other hand, are formative units centered around a challenge-based educational strategy. In these blocks, students tackle a well-designed challenge that requires them to develop the competencies assigned to the unit while acquiring and applying the knowledge taught in the block’s modules.

Programs are structured into three stages: exploration, focus, and specialization. The exploration stage spans three semesters, during which students take formative units that allow them to explore various STEM disciplines and decide on a long-term program to pursue. The focus stage constitutes the core of the program, where students primarily engage in formative units alongside peers from their chosen discipline. Finally, the specialization stage includes two semesters: the Tecsemester, which offers opportunities for internships, research projects, or studying abroad, and the last semester, which is dedicated to integrating all the competencies developed throughout the program.

All skills are worked on continuously over the course of the program, but their focus varies depending on the stage of the educational journey. In this model, students have the option to choose their long-term program from the beginning. However, for those who are undecided, the exploratory stage provides a valuable opportunity to explore different STEM disciplines. Figure 2 illustrates the STEM programs offered at the university.

Students can start their university experience by selecting a long-term program, such as Food Engineering, or by opting for an academic area (entry program) like Bioengineering, which includes five related programs. Within an academic area, students develop the same competencies, including both transversal and area competencies. During the exploration stage, these are the prevalent competencies. As students progress to the focus stage, they transition into their chosen long-term program and receive formative units tailored to that program, alongside other electives and general courses. In this stage, students continue to develop transversal and area competencies while beginning to focus on discipline-specific competencies. Appendix A provides a comprehensive list of all the competencies that STEM academic program students developed between 2019 and 2022.

3.2. Database and Sample Description

The database published by Tecnológico de Monterrey is distributed as a collection of text files, separated by academic programs due to its large size. The raw database contains $5,940,006$ records corresponding to evaluations of $16,500$ STEM students at the School of Engineering and Sciences of Tecnológico de Monterrey from August–December 2019 (abbreviated to 2019AD) to February–June 2022 (abbreviated to 2022FJ). This dataset encompasses the initial three academic courses implementing the Tec21 pedagogical model divided into six academic periods.

The database includes 44 raw features that collect information about the student, the group, the formative unit, and the competency and sub-competency evaluated. In our data analysis problem, we employed the public data transformation steps reported in Valdes-Ramirez et al. (2024) for obtaining a dataset. However, we retained all competencies and sub-competencies instead of the sustainability competencies.

According to the raw dataset, the ICT and IBQ academic programs exhibit the closest values in terms of the number of students and database rows collected. Figure 3 presents two heatmaps illustrating the differences between entry programs regarding the number of students and database rows. In these heatmaps, darker gray tones represent larger differences. To mitigate bias associated with academic programs, we selected the ICT and IBQ entry programs for analysis, as they demonstrate the smallest differences in both metrics. Additionally, we included data from the full academic programs associated with these entry programs, namely, IAG, IAL, IBT, IDS, IQ, ITC, IRS, and ITD.

For cohort selection, we included the first two cohorts to enable a comprehensive evaluation of their academic progress. The dataset for the 2019 cohort encompasses six academic periods (from August–December 2019 to February–June 2022), whereas the 2020 cohort includes four semesters (from August–December 2020 to February–June 2022). To refine the dataset, we applied filters to exclude records of subjects not listed in the official program guides published on the Tec website. These exclusions address instances where students might have taken subjects outside their programs due to the flexibility of the academic model, resulting in a reduced sample size for evaluating performance.

After applying these filters, the dataset comprised $557,675$ records for 4566 students, each representing a unique sub-competency evaluation in terms of the student, formative unit, semester, and competency (refer to Appendix B). The dataset includes evaluations of 60 competencies, with 50 associated with IBQ-related programs and 33 with ICT-related programs. Furthermore, it contains assessments of 150 sub-competencies, of which 111 are linked to IBQ-related programs and 69 to ICT-related programs. Similarly, evaluations span 351 formation units, with 285 belonging to IBQ-related programs and 242 to ICT-related programs.

Regarding gender distribution, male students $\left(2682\right)$ constitute nearly two-thirds of the sample. Students are distributed across campuses, which are grouped into five regions (Figure 4): RM (Monterrey Region), RCS (South/Central Region), RCM (Mexico City Region), RO (West Region), and DR (Regional Development Region). Notably, the Monterrey Region and Regional Development Region represent the highest and lowest numbers of students, respectively, while the other regions have relatively balanced student distributions. Some students have moved between regions, resulting in their total counts exceeding the actual number of students. The sunflower chart in Figure 5 details the number of students by entry program and corresponding long-term programs. Among long-term programs, IAG $\left(66\right)$ and ITD $\left(166\right)$ have the fewest students enrolled under the IBQ and ICT entry programs, respectively, while IBT $\left(1110\right)$ and ITC $\left(1113\right)$ have the largest number of enrollments.

Although the majority of students in the dataset are Mexican ($93.47\%$), Tec de Monterrey academic programs consistently attract international students, enhancing their global reputation and value. Figure 6 depicts the distribution of the remaining $6.53\%$ of students by country. The most represented foreign countries include the USA, Ecuador, Venezuela, Honduras, and Guatemala, each accounting for over $5\%$ of the international students. However, nationality data are incomplete for $1.3\%$ of the foreign students, who are listed without specified country information.

4. Results and Discussion

In this section, we describe the findings of our study. First, we will present the results of the analysis about whether delaying the choice of enrolling on a long-term program is beneficial for students or not. Next, we will describe whether there exist sub-competencies that are harder to develop and how they relate to dropout. Then, we will show whether there exists a relation among final grades, the number of credits allocated to a module, and whether a module is part of a Tec week or not.

4.1. Entry Program or Long-Term Program? An Analysis of Student Performance

To perform this analysis, we first generated a dataset view with records of students enrolled in the programs under study. In particular, we needed to separate active students from those who abandoned the School of Engineering and Science (dropouts). Also, we needed to identify students’ choices (whether they continued opting for an entry program or switched to a specific one). We proceeded as follows: If there is no record of a student as of Fall 2021 for the 2019 cohort and Spring 2022 for the 2020 cohort, she is considered to have left the school. Otherwise, the student is allocated to the last program in which she was enrolled. Next, we computed the total number students (and the associated percentage) in each subject. The three most relevant long-term programs are selected based on the school’s programs (see Figure 2). Students who did not drop out or who were not enrolled in any of these three programs were grouped into the Other category.

Table 1. Percentages of ICT/IBQ students per destination program chosen.

Program	Cohort	ITC	IRS	ITD	Other	Dropout STEM
ICT	2019	57.096	15.385	8.234	6.176	13.109
	2020	57.500	15.326	8.696	7.065	11.413
		IBT	IQ	IDS
IBQ	2019	40.015	20.503	10.823	14.482	14.177
	2020	43.776	20.217	10.526	12.865	12.615

shows the percentages of IBQ/ICT students per destination program chosen based on their cohort.

Figure 7 and Figure 8 convey the academic progression of students who enrolled in the ICT and in the IBT entry programs, respectively, starting in the academic year 2019AD. They track their movement through various programs until the academic year 2022FJ. The identification of the new entrants in each year is based on the column enrollment_period.id, which indicates the period in which the student first enrolled at Tec. New entrants for a given year are those whose enrollment_period.id corresponds to that year. Each node represents a specific academic program and year, while the bands connecting the nodes indicate the flow of students transitioning between these programs over time.

Key observations from Figure 7 include the following: a majority of the students who initially enrolled in the ICT program in 2019AD remained in the ICT program through to 2020FJ; from 2020AD onwards, there is a noticeable transition of students into the Engineering in Computer Technologies (ITC) program, with many continuing in ITC up to 2022FJ; some students diverted to other programs such as Robotics and Digital Systems Engineering (IRS), Digital Business Transformation Engineering (ITD), and various other engineering disciplines, as indicated by the thinner bands extending to these nodes; a substantial portion of students who initially enrolled in the ICT program in 2020AD remained in the ICT program through to 2021FJ; from 2021AD onwards, there is a significant transition of students into the Engineering in Computer Technologies (ITC) program, with many continuing in ITC up to 2022FJ; some students diverted to other programs such as Robotics and Digital Systems Engineering (IRS), Digital Business Transformation Engineering (ITD), and various other engineering disciplines, as indicated by the thinner bands extending to these nodes.

Key observations from Figure 8 include the following: almost all students that were in IBQ 2019AD are in the IBQ program in 2020FJ; there is a notable transition of students from IBQ to IBT in 2020AD and subsequent years; some students diverted to other programs such as Chemical Engineering (IQ), Environment Engineering (IA), and various other engineering disciplines, as indicated by the thinner bands extending to these nodes; a large majority of students who initially enrolled in the IBQ program in 2020AD remained in the IBQ program through to 2021FJ; from 2021AD onward, there is a notable shift of students transitioning to the Biotechnology Engineering (IBT) program, particularly in 2022FJ, and there are also transitions to other programs such as Chemical Engineering (IQ), Food Engineering (IAL), and various other engineering disciplines, as shown by the thinner bands extending to these nodes.

Comparison of Academic Performance

To test the hypothesis that a longer permanence in entry programs, which allows students to delay their choice of degree, benefits their performance in subsequent semesters, we created two new datasets: one with students who did not complete the entry program (stayed one or two terms), and another with students who did otherwise (stayed three terms). As can be seen in Table 1, the dropout rate decreased by nearly two percentage points from the 2019 cohort to 2020 cohort. However, it is still significant that a considerable percentage of students, 27.94% in ICT and 18.89% in IBQ (see

Table 2. Percentages of students that attended one, two, three, four, five, or six semesters in an entry program.

Program	Cohort	1	2	3	4	5	6
ICT	2019	17.01	79.52	2.60	0.54	0.22	0.11
	2020	12.39	27.94	58.04	1.63	-	-
IBQ	2019	11.88	83.85	3.43	0.69	0.15	0.0
	2020	12.10	18.89	67.44	1.57	-	-

), continued studying for only two terms before transitioning to a long-term program.

Considering only the number of semesters taken by both cohorts, without differentiating by cohort, the reduction in the percentage of dropouts is even greater. In the case of ICT, it is reduced by up to four percentage points (from $7.87\%$ to $3.04\%$), and by up to five points (from $9.33\%$ to $3.73\%$) in IBQ, as seen in

Table 3. Percentages of students per entry program, long-term program chosen, and semesters completed.

Entry Program	Semesters Completed	Dropout STEM	IRS	ITC	ITD	Other
ICT	1	48.34	11.07	20.30	3.69	16.61
	2	7.87	24.52	55.70	7.97	3.94
	3	3.05	1.61	80.29	12.01	3.05
			IBT	IDS	IQ
IBQ	1	60.26	7.62	10.60	4.30	17.22
	2	9.33	40.26	16.33	21.22	12.87
	3	3.73	57.51	2.21	25.15	11.41

. High dropout rates are observed for those who only studied one term, as observed in other studies carried out for STEM careers (Casanova et al., 2023).

These findings suggest that the flexibility of entry programs generally benefits students’ academic performance in the following academic years. Continuing with this line of analysis, we examined whether students who completed the three-term entry program demonstrated better academic outcomes compared to those who completed only part of it.

Table 4. Summary of average grade and approved sub-competencies ratio by campus region and set of students for IBQ dataset.

Campus Region	Avg Grade	Set	Approved Ratio
DR	93.429467	Complete	0.950221
DR	92.646617	Partial	0.949588
RCM	94.022748	Complete	0.978515
RCM	92.055155	Partial	0.946551
RCS	93.549451	Complete	0.959267
RCS	91.987231	Partial	0.914206
RM	93.855562	Complete	0.950907
RM	91.955947	Partial	0.952878
RO	94.796176	Complete	0.932837
RO	91.636012	Partial	0.926436

shows the IBQ dataset, which confirms that students who took the complete entry program had a better ratio of approved sub-competencies and better grades than those who completed part of it. Although the differences are not significant, the trend is repeated on the different Teccampuses. Furthermore, students who took the complete entry program had a better ratio of approved sub-competencies and better grades in the different types of formative units than those who completed it partially in IBQ (see

Table 5. Summary of average grade and approved sub-competencies ratio by subject type and set of students for IBQ dataset.

Subject Type	Avg Grade	Set	Approved Ratio
Block	92.121894	Complete	0.946806
Block	90.153518	Partial	0.947961
Subject	92.382627	Complete	0.974444
Subject	90.170732	Partial	0.947870
Tec week	99.301826	Complete	0.902334
Tec week	99.194415	Partial	0.702470

), except for the sub-competencies of the ‘block’-type formative unit, where the ratio for the complete program is 0.947 and for the partial program is 0.948. However, the difference is practically insignificant. Even more, the performance of the A-level sub-competencies was higher in the students of the full entry program than those of the partial one, with both groups having a similar size of about 500 students (see

Table 6. Summary of approved sub-competencies ratio and unique student counts by sub-competency level and set of students for IBQ dataset.

Sub-Competence Level	Approved Ratio	Set	Unique Student Count
A	0.964370	Complete	501
A	0.930930	Partial	480
B	0.965177	Complete	501
B	0.966134	Partial	479
C	1.000000	Partial	1

).

Apart from that, the ratio of approved sub-competencies, including both area and general education sub-competencies, is higher in students from the full entry program than in those from a specific one, as shown in

Table 7. Summary of approved sub-competencies ratio and unique student counts by competency type and set for IBQ dataset.

Competence Type	Approved Ratio	Set	Unique Student Count
Area	0.963002	Complete	501
Area	0.951998	Partial	480
Disciplinary	0.883252	Complete	491
Disciplinary	1.000000	Partial	1
General education	0.958172	Complete	501
General education	0.908418	Partial	480

.

The analysis conducted on the ICT and IBQ entry programs demonstrates that providing students with the opportunity to delay their choice of specific degree pathway can positively impact their academic performance. Only IBQ results are shown in order to not make the article too long. By examining the academic progression and outcomes of students who either completed or partially completed the entry programs, we observed that those who completed the entire entry program generally achieved better academic results, both in the traditional numeric evaluation and its corresponding competency evaluation.

4.2. An Analysis of Competency Difficulty

Next, we carried out a competency study from different perspectives with the aim of (i) identifying formative units and sub-competencies that are more difficult to pass; (ii) checking if dropout could be a cause of the difficulty in overcoming these harder sub-competencies; (iii) verifying whether sub-competency evaluation is consistent when it is performed by professors in different formative units; and, finally, (iv) uncovering the number of observations needed on average to acquire a sub-competency.

To observe which formative units are more difficult to pass, the dataset available for the ICT entry program was filtered, selecting only subjects that are gathered in these programs according to the Tec website. Despite being compulsory formative units, Tec weeks do not form part of the final grade of a subject; hence, we discarded them for this particular study on competency analysis. Next, the mean and standard deviation of the grades for each subject were calculated only considering students who studied them in the semester in which they were planned. Then, the sub-competencies taught in each subject were included.

Observing those sub-competencies taught in subjects with a greater dispersion (SD) and number of failed students (see

Table 8. Statistics of 2019 cohort from ICT program subjects (top 10 with the highest dispersion). SD and Avg represent the standard deviation and average grade; the Fail and Pass columns collect the number of students who failed or passed the subject; sub-competencies include all the sub-competencies that are involved in the evaluation of the subject.

Subject	Subject Type	SD	Avg	Fail	Pass	Sub-Competencies
Object-Oriented Programming	Subject	15.94	88.48	34	663	Problem evaluation, decision making, cutting-edge technologies
Object-Oriented Computational Thinking	Subject	15.50	86.12	67	679	Application of standards and norms, demonstration of the functioning of systems in engineering and sciences, cutting-edge technologies, problem evaluation, decision making, implementation of actions
Mathematical Thinking	Subject	14.66	83.39	78	680	Explanation of the functioning of systems in engineering and sciences, scientific thought, understanding others’ code, problem evaluation
Computational Thinking for Engineering	Subject	13.27	88.11	40	762	Application of standards and norms, demonstration of the functioning of systems in engineering and sciences, cutting-edge technologies, problem evaluation, decision making, implementation of actions
Computational Modeling Applying Conservation Laws	Block	13.12	85.64	42	609	Scientific thought, demonstration of the functioning of systems in engineering and sciences, explanation of the functioning of systems in engineering and sciences, systemic thinking, written language, problem evaluation, decision making, implementation of actions
Computational Biology Analysis	Subject	13.09	90.49	26	687	Scientific though, understanding others’ code, determination of patterns, application of sustainability principles
Fundamentals of Biological Systems	Subject	12.91	85.54	3	46	Explanation of the functioning of systems in engineering and sciences, demonstration of the functioning of systems in engineering and sciences, scientific thought, digital culture, wellness and self-regulation
Ecological Processes for Human Development	Subject	12.91	85.32	2	45	Application of standards and norms, scientific thought, digital culture, recognition and empathy, application of sustainability principles
Foundation of the Structure and Transformation of Matter	Subject	12.21	84.44	62	761	Explanation of the principles of systems in engineering and sciences, scientific thought, problem evaluation
Computational Modeling of Movement	Block	11.85	86.17	44	594	Scientific thought, demonstration of the principles of systems in engineering and sciences, explanation of the principles of systems in engineering and sciences, systemic thinking, written language, problem evaluation, decision making, implementation of actions

), it can be concluded that learners struggle more with subjects that involve learning new paradigms and problem solving, such as Mathematical Thinking I and Object-Oriented Computational Thinking.

Once the most difficult subjects were identified, we tested if the sub-competencies that challenge students the most are also assessed in the subjects which reflect lower ratings in the evaluation. For this purpose, we examined which sub-competencies had the highest number of ‘non-observations’ (failed competency assessments), and also weighting this value by the total number of observations (ratio).

By examining the sub-competencies within the previously identified difficult subjects and comparing them with the sub-competencies shown in

Table 9. Top ten sub-competencies that posed the greatest difficulties to the students of the 2019 cohort from the ICT program.

Sub-Competency	Level	Failed	Approved	Ratio
Problem evaluation	A	594	6689	11.261
Demonstration of the functioning of systems in engineering and sciences	A	255	4771	18.71
Scientific thought	A	546	4692	8.593
Decision making	A	293	3896	13.30
Explanation of the functioning of systems in engineering and sciences	A	316	3612	11.43
Implementation of actions	A	185	3305	17.865
Cutting-edge technologies	A	131	2947	22.496
Understanding others’ codes	A	221	2273	10.285
Critical thinking	A	200	2017	10.085
Explanation of the functioning of systems in engineering and sciences	B	103	2013	19.544

, a clear relationship emerged: the sub-competencies that are the most challenging to approve are the same ones that are evaluated in these subjects. A ratio is also added, as a quotient of the number of approvals by the number of failures, showing an optimal comparison.

Also, by analyzing the evaluation of these subjects according to gender, it is possible to observe that women tend to fail less than men, as can be seen by comparing the percentage of enrollments to the percentage of failures between the two genders in

Table 10. Comparison of both genders between the percentage of enrollments and failures, revealing a lower proportion of failures among females on ICT program subjects for the 2019 cohort.

Subject	Men	Women	Fail Men	Fail Women	Women (%)	Fail Women (%)
Object-Oriented Programming	563	135	31	3	19.34	8.82
Object-Oriented Computational Thinking	603	149	55	12	19.81	17.91
Mathematical Thinking I	606	157	69	9	20.58	11.54
Computational Thinking for Engineering	639	163	34	6	20.32	15.00
Computational Modeling Applying Conservation Laws	526	129	36	6	19.69	14.29
Computational Biology Analysis	571	142	26	0	19.92	0.00
Fundamentals of Biological Systems	38	11	3	0	22.45	0.00
Ecological Processes for Human Development	36	11	3	2	23.40	100.00
Foundation of the Structure and Transformation…	654	169	54	8	20.53	12.90
Computational Modeling of Movement	517	121	41	3	18.97	6.82
Intermediate Mathematical Modeling	563	140	25	1	19.91	3.85
Statistic Analysis	567	140	15	1	19.80	6.25
Engineering and Science Modeling	680	172	22	2	20.19	8.33
Mathematics and Data Science for Decision Making	43	10	1	0	18.87	0.00
Computational Modeling of Electrical Systems	512	132	16	0	20.50	0.00

.

Using the datasets created, we can identify the IDs of students who leave the technological branch (TEC). Now, by knowing the subjects and sub-competencies that pose the greatest challenges, we can examine if there are any common academic features in this group. For this purpose, we calculated the average grade for the subjects performed achieved by those students who dropped out of the STEM branch, as well as the number of passes and failures in each subject. As shown in

Table 11. Average results for ICT program subjects achieved by the students of the 2019 cohort who dropped out.

Subject	Avg	Fail	Pass	Sub-Competences
Foundation of the Structure and Transformation of Matter	73.62	32	75	Explanation of the functioning of systems in engineering and sciences, scientific thought, problem evaluation
Mathematical Thinking	67.51	29	45	Explanation of the functioning of systems in engineering and sciences, scientific thought, understanding others’ codes, problem evaluation
Object-Oriented Computational Thinking	69.96	26	43	Application of standards and norms, demonstration of the functioning of systems in engineering and sciences, cutting-edge technologies, problem evaluation, decision making, implementation of actions
Computational Modeling of Movement	71.58	26	41	Scientific thought, demonstration of the functioning of systems in engineering and sciences, explanation of the functioning of systems in engineering and sciences, systemic thinking, written language, problem evaluation, decision making, implementation of actions
Engineering and Science Modeling	80.12	20	95	Explanation of the functioning of systems in engineering and sciences, problem evaluation, self-knowledge
Computational Thinking for Engineering	75.82	20	76	Application of standards and norms, demonstration of the functioning of systems in engineering and sciences, cutting-edge technologies, problem evaluation, decision making, implementation of actions
Computational Modeling Applying Conservation Laws	70.59	15	40	Scientific thought, demonstration of the functioning of systems in engineering and sciences, explanation of the functioning of systems in engineering and sciences, systemic thinking, written language, problem evaluation, decision making, implementation of actions
Object-Oriented Programming	62.59	9	13	Problem evaluation, decision making, cutting-edge technologies
Computational Biology Analysis	68.73	7	16	Scientific thought, understanding others’ codes, determination of patterns, application of sustainability principles
Intermediate Mathematical Modeling	79.04	5	20	Problem evaluation, understanding others’ codes, explanation of the functioning of systems in engineering and sciences

, there is a notable increase in the number of failures compared to the number of passes. Additionally, there is a notable decrease in the average grade, particularly in several subjects previously identified as the most challenging (e.g., Mathematical Thinking I).

Next, the relationship between the difficulty of sub-competencies and how much emphasis is placed on their assessment was analyzed. Attending to the tuple of subject, sub-competency, and required level, the number of times the sub-competency is evaluated was determined, ordering the sub-competencies by this value. Then, the ranking of the sub-competencies that are most complicated to pass was calculated (the lower the number, the greater the difficulty), according to the study carried out previously (Table 9). Observing the results shown in

Table 12. Ranking of most difficult sub-competencies and number of times that they are evaluated (based on the 2019 cohort of the ICT program, 15 examples). The difficulty value in the ranking is obtained by the difference between the number of approvals and fails for the sub-competency weighted by the total number of observations for each one.

Sub-Competency	Level	Evaluations	Ranking
Systemic thinking	A	16	11
Scientific thought	A	13	3
Written language	A	10	13
Understanding others’ codes	A	10	8
Problem evaluation	A	10	1
Critical thinking	A	10	9
Demonstration of the functioning of systems in engineering and sciences	A	8	2
Cutting-edge technologies	A	7	7
Decision making	A	7	4
Explanation of the functioning of systems in engineering and sciences	A	7	5
Digital culture	A	6	21
Implementation of actions	A	6	6
Explanation of the functioning of systems in engineering and sciences	B	4	10
Application of standards and norms	A	4	12
Wellness and self-regulation	A	3	22

, it can be concluded that there are sub-competencies for which the number of evaluations could be reduced in favor of others that pose a greater barrier for the students, such as systemic thinking or written language.

Next, the following analysis checked whether the assessment of sub-competencies is consistent when it is evaluated in different formative units and by different teachers. This means that, once the sub-competency has been observed (approved) in one formative unit, it is also passed in the subsequent ones.

Table 13. Number of evaluations in which a sub-competency is marked as failed after having been approved for the 2019 cohort in the ICT program (discarding cases equal to 0).

Sub-Competency	Level	Failed
Problem evaluation	A	367
Scientific thought	A	297
Demonstration of the functioning of systems in engineering and sciences	A	196
Explanation of the functioning of systems in engineering and sciences	A	192
Decision making	A	182
Implementation of actions	A	90
Cutting-edge technologies	A	89
Critical thinking	A	79
Systemic thinking	A	67
Understanding others’ codes	A	56
Written language	A	37
Explanation of the functioning of systems in engineering and sciences	B	34
Application of standards and norms	A	26
Application of sustainability principles	A	3
Innovation	A	1
Implementation of actions	B	1

shows the number of instances where the previous assertion was not fullfilled, i.e., where failed evaluations were found after having been passed, such as in systemic and scientific thinking or problem solving. However, the number of such cases is significantly lower compared to the frequency with which these sub-competencies are evaluated. This suggests that an appropriate criterion (evaluation rubric) is being used to assess whether sub-competencies are observed or surpassed.

By identifying the subjects that evaluate each competency, it is possible to visualize the weight given to each competency within the program (Figure 9). Some transversal competencies such as communication or reasoning for complexity appear in a large number of subjects in which they are evaluated. These mainly correspond to transversal competencies in optional subjects, which is directly related to the number of subjects offered by the university.

Based on the previous analysis of the most complicated sub-competencies to surpass, it is also possible to check whether the competencies include them and group sub-competencies of a similar difficulty, and check if they are great differences among them, easily by ordering them by means of a difficulty ranking (lower ratio means higher difficulty). As can be seen in

Table 14. Grouping of sub-competencies by competencies in which they are evaluated (based on the 2019 cohort of the ICT program).

Competencies	Evaluated Sub-Competencies	Ranking
Application of international standards	Application of standards and norms—A, application of sustainability principles—A, application of sustainability principles—B, application of standards and norms	12, 14, 36, 37
Commitment to sustainability	Application of standards and norms—A, application of sustainability principles—A	12, 14
Communication	Understanding others’ codes—A, written language—A, oral language—A	8, 13, 38
Data analysis in engineering and science	Interpretation of variables—A, development of scenarios—A, development of scenarios—B, interpretation of variables—B	17, 19, 20, 18
Development of business architectures	Requirements analysis—A, incorporation of technology—A	31, 33
Digital transformation	Digital culture—A, cutting-edge technologies—A	21, 7
Embedded systems	Managing technological design projects—A	39
Ethical and citizen commitment	Recognition and empathy—A, citizen commitment for social transformation—A	23, 40
Foundation of computer systems	Demonstration of the functioning of systems in engineering and sciences—A, explanation of the functioning of systems in engineering and sciences—A, explanation of the functioning of systems in engineering and sciences—B, demonstration of the functioning of systems in engineering and sciences—B	2, 5, 10, 27
Foundation of engineering and science systems	Explanation of the functioning of systems in engineering and sciences—A, demonstration of the functioning of systems in engineering and sciences—A, explanation of the functioning of systems in engineering and sciences—B	5, 2, 10
Generation of computational models for data analysis	Determination of patterns—A, determination of patterns—B, interpretation of variables—B, development of scenarios—B	16, 34, 18, 20
Innovative entrepreneurship	Innovation—A	26
Problem solving	Problem evaluation—A, decision making—A, implementation of actions—A	1, 4, 6
Problem solving with computing	Problem evaluation—A, decision making—A, implementation of actions—A, problem evaluation—B, decision making—B, implementation of actions—B	1, 4, 6, 30, 29, 25
Reasoning for complexity	Scientific thought—A, systemic thinking—A, critical thinking—A	3, 11, 9
Self-knowledge and management	Wellness and self-regulation—A, self-knowledge—A	22, 15
Social intelligence	Diversity—A, collaboration—A	24, 41
Software systems development	Definition of software requirements—A, application of software methodologies—A, computer project management—A	28, 32, 35

, there are competencies such as problem solving with computing that integrate sub-competencies that contrast with each other in difficulty, while other competencies collect sub-competencies that are more level with the rest.

Finally, we aimed to investigate the relationship between the frequency of attempts at a sub-competency and the corresponding performance outcomes. Specifically, we wanted to determine how many attempts a student needs to either approve or fail a sub-competency, knowing that the same student encounters the same sub-competency at different formative units throughout their program.

The analysis reveals that, on average, sub-competencies that failed in the most recent evaluation had been assessed 0.221876 times previously. In contrast, those that were approved had undergone around 1.266455 prior evaluations. This suggests that sub-competencies passing the final evaluation tend to have been evaluated more frequently compared to those that failed. Furthermore, it is more common for a student to fail a sub-competency when it is being evaluated for the first time.

4.3. Analysis of Competency Development Against Type and Grade of the Formative Unit

Finally, a subject-centered analysis was conducted to determine whether variations in the number of credits allocated to different subjects significantly impacted final grades, and whether subjects related to Tec weeks exhibited different evaluation patterns.

To assess the effort students dedicated to each subject, the grades in the dataset were weighted by the number of credits. This enabled us to observe if notable differences existed between the classic and weighted averages at the program level. However, no substantial differences beyond 1 point out of 100 were observed in either program (see

Table 15. Comparison between arithmetic and weighted mean by number of credits for the 2019 and 2020 cohorts of the ICT and IBQ programs.

Program	Semester	Mean 2019	Weighted Mean 2019	Mean 2020	Weighted Mean 2020
ICT	1	85.197	85.512	89.671	90.185
ICT	2	89.697	89.870	89.986	89.874
ICT	3	88.488	88.583	90.345	90.578
ICT	4	70.502	70.788	82.094	81.707
ICT	5	66.775	66.570	-	-
ICT	6	85.485	83.921	-	-
IBQ	1	86.377	86.889	90.355	90.810
IBQ	2	89.926	89.430	91.154	91.099
IBQ	3	86.524	85.662	89.409	88.580
IBQ	4	83.256	82.572	81.973	82.249
IBQ	5	83.535	82.228	-	-
IBQ	6	86.731	86.944	-	-

). This reflects the fact that the customization of the number of credits thus varied does not really have a significant impact in the overall assessment of a student’s record. Furthermore, the weighted average grade for females is 90.565 and for males 88.525, revealing no notable differences by sex.

In the case of Tec week subjects, a particular behavior was observed with respect to other types of subjects. Even means of 100.0 are appreciated for these subjects. However, the sub-competency self-knowledge in the entry program (

Table 16. Evaluation of Tec weeks for the 2019 cohort of the ICT entry program. SD and Avg represent the standard deviation and average grade; the Fail and Pass columns collect the number of students who failed or passed the subject.

Subject	SD	Avg	Fail	Pass	Sub-Competences
Week 18 Engineering—1	18.33	96.52	29	805	Self-knowledge
Week 18 Engineering—2	13.69	98.09	15	769	Self-knowledge
Entrepreneurship with Purpose	17.22	96.94	21	665	Conscious entrepreneurship
Turn Your Wellness On	10.51	98.88	4	353	Wellness and self-regulation
My Selfie Today	17.81	96.74	6	178	Self-knowledge
A Trip to My Interior	19.99	95.86	6	139	Self-knowledge
Me, You, Others, Us	12.04	98.54	2	135	Diversity
Meaningful Theater	14.67	97.81	2	89	Innovation
Induction to the Social Service	28.81	91.30	2	21	Recognition and empathy
18th Week All Entries	47.05	69.57	7	16	Self-knowledge
Week 18 Multientry Exploration—3	22.36	95.00	1	19	Self-knowledge

), which is the most assessed, with a high rate of ‘no observations’ (835 failed vs. 1196 approved), appears in subjects where only this sub-competency is assessed and still the mean is 95-100.0. Seeing the high scores and the sub-competencies being assessed, there seems to be a lower correlation between the assessment of the sub-competencies and the grade for the Tec week subjects. Something similar happens in the ITC program (

Table 17. Evaluation of Tec weeks for the 2019 cohort of the ITC program (long duration).

Semest	Subject	SD	Avg	Fail	Pass	Sub-Competences
4	Week 18 Engineering ICT—4	6.50	99.57	2	466	Self-knowledge
5	Week 18 Focus ITC—5	15.12	97.66	10	418	Self-knowledge
3	Induction to the Social Service	13.77	98.07	4	203	Recognition and empathy
5	Next Level: Competitive Programming Expert	10.19	98.95	2	189	Collaboration, computational algorithm optimization
3	CS Tool—Mastering Programming	0.00	100.00	0	139	Application of standards and norms
5	Web World Connecting	12.72	98.36	2	120	Cutting-edge technologies, software component development
5	Diversity in a Globalized World	20.07	95.83	5	115	Diversity
6	One Click Away from your Professional Life	12.86	98.33	2	118	Self-knowledge
4	CS Tool—Mastering Programming	16.01	97.39	3	112	Application of standards and norms
4	CS Tool—Mastering Analytics	9.76	99.05	1	104	Interpretation of variables
6	Web World Connecting	10.23	98.95	1	94	Software component development, cutting-edge technologies
4	Ikigai: Building your Dreams	14.51	97.87	2	92	Wellness and self-regulation
6	Diversity in a Globalized World	27.80	91.67	7	77	Diversity
5	CS Tool—Mastering Programing	10.98	98.80	1	82	Application of standards and norms
3	CS Tool—Mastering Analytics	26.35	92.59	6	75	Interpretation of variables
3	Diversity in a Globalized World	11.32	98.72	1	77	Diversity
4	Multigenerational Leadership	19.73	96.00	3	72	Effectiveness in negotiation
4	The Art of Emotion	11.55	98.67	1	74	Understanding others’ codes
5	CS Tool—Mastering Analytics	11.55	98.67	1	74	Interpretation of variables
3	Multigenerational Leadership	0.00	100.00	0	75	Effectiveness in negotiation
4	The Role of Integrity in Practicing my Profession	16.44	97.26	2	71	Integrity
4	Diversity in a Globalized World	20.69	95.59	3	65	Diversity
4	Me, You, Others, Us	12.22	98.51	1	66	Diversity
6	CS Tool—Mastering Programing	24.58	93.65	4	59	Application of standards and norms

).

In addition, it was observed that, in the entry program, hardly any sub-competencies that are related to the ‘block’- or ‘subject’-type subjects are evaluated in the Tec weeks. These are sub-competencies that are more generic or involve more personal knowledge, as seen in

Table 18. Sub-competencies assessed in the Tec weeks of the ICT entry program.

Sub-Competency	Level	False	True	Total
Self-knowledge	A	835	1196	2031
Conscious entrepreneurship	A	24	666	690
Wellness and self-regulation	A	10	361	371
Diversity	A	6	150	156
Innovation	A	3	105	108
Recognition and empathy	A	2	27	29
Application of standards and norms	A	4	11	15
Interpretation of variables	A	0	8	8
Collaboration	A	0	7	7
Understanding others’ codes	A	0	7	7
Effectiveness in negotiation	A	1	6	7
Digital culture	A	1	2	3

. In the long-term program, they appear to a greater extent, although only one or two sub-competencies are evaluated per week (

Table 19. Sub-competencies assessed in the Tec weeks of the ITC long-term program.

Sub-Competency	Level	False	True	Total
Self-knowledge	A	491	933	1424
Diversity	A	16	562	578
Application of standards and norms	A	20	547	567
Wellness and self-regulation	A	13	414	427
Collaboration	A	7	360	367
Interpretation of variables	A	6	294	300
Effectiveness in negotiation	A	5	239	244
Computational algorithm optimization	B	1	225	226
Recognition and empathy	A	9	216	225
Software component development	A	3	214	217
Cutting-edge technologies	A	3	214	217
Understanding others’ codes	A	4	208	212
Citizen commitment for social transformation	B	4	158	162
Integrity	B	5	146	151
Conscious entrepreneurship	A	10	63	73
Wellness and self-regulation	B	2	32	34
Digital culture	A	0	8	8
Innovation	A	1	6	7
Self-knowledge	B	0	2	2

).

Student Workload Versus Failure Rate

Knowing that students fail a learning unit if their final grade is below 70, we analyzed whether enrolling in more credits than usual negatively affects IBT students. Our analysis revealed a very weak negative correlation, indicating no strong or direct relationship between the number of credits a student enrolls in and the number of subjects they fail. The near-zero correlation value suggests there is no clear evidence that a heavier course load impacts the number of failed subjects. This comparison is illustrated in

Table 20. Correlation matrix between full-time equivalent (FTE) and suspensions count.

	student.fte	suspensions_count
student.fte	1.000000	−0.035677
suspensions_count	−0.035677	1.000000

.

Next, we aimed to determine if there is a relationship between the number of credits taken and course grades. In the IBQ program (see

Table 21. Comparison of student final grades by FTE category. This table shows the average final grades of students after adjustments, categorized by their FTE status.

fte_category	student_grades.final_numeric_afterAdjustment
Above 1	87.186581
Below 1	88.042601
Exactly 1	90.308793

), students who enrolled in more credits than they were entitled to received the lowest grades. This group was followed by students who enrolled in fewer credits and those who enrolled in the exact number of credits they were entitled to. Although the difference in grades is only 3 points out of 100, it suggests that enrolling in fewer credits is slightly better for academic performance in the IBQ program than exceeding the standard enrollment.

4.4. Limitations

We must point out the limitations of this study. First, we have a dataset that does not include the performance of one or more complete cohorts of students. Thus, a study of performance at the end of the studies was not feasible. This also hampered the analysis of competencies, which was limited to those observed in the first semesters. Thus, our findings stress the importance of data quality and quantity in research, suggesting that more robust and complete datasets are essential for making definitive conclusions about the efficacy of CBE models.

There was also no contextual information available that could provide answers to certain questions. For example, we could not compare students who took the long-term program without going through the entry program. This seems to be a decision established by the university. Additionally, one cohort, 2019, was affected by the pandemic produced by COVID-19, and we are not aware of any information regarding academic changes between the different cohorts; therefore, the study treats all records equally. On the other hand, the sub-competencies are not evaluated with a numeric grade; thus, the skill level acquired cannot be ranked.

5. Conclusions and Directions for Further Work

In this study, we analyzed the Tec21 dataset, released by Tecnologico de Monterey for the evaluation of the Tec21 competency-based education pedagogic model. Unlike the study of Sánchez-Carracedo et al. (2021) or Talamás-Carvajal et al. (2024), ours involved millions of records and did not separate the traditional numeric evaluation of a subject from the assessment of competency development.

The findings of our study suggest that the structure and flexibility of the entry programs, available in the Tec21 educational model, give a student more opportunities for success. This is because these programs allow students to explore their interests and build a solid academic foundation before committing to a specific subject. We have seen that not only does this approach enable increased student performance in subsequent semesters, but it also reduces the dropout rate, contributing to students’ long-term academic and professional development.

In particular, students who completed the full three-semester entry program had higher average grades and a better ratio of approved sub-competencies across different Teccampuses. This trend was consistent for both the ICT and IBQ programs, despite only IBQ’s results being shown, as well as across various types of formative units such as ‘block’ and ‘subject’. The complete-program students also showed a lower dropout rate, indicating that extended exposure to the entry program helps in retaining students within STEM fields.

Additionally, the performance in sub-competencies was notably higher for students who completed the full program. These students showed a higher approval ratio in both area and general education competencies compared to those who transitioned to a specific degree program after only two semesters. Interestingly, students who failed after completing the partial program tended to struggle more with ‘subject’ formative units, whereas those who completed the full program found ‘block’-type units more challenging.

Our study also shows that students find it more difficult to successfully complete subjects that involve learning new paradigms for problem solving. Coincidentally, such modules also involve observing competencies that students found very challenging to develop. Example competencies of this kind include mathematical thinking and computational thinking. Students who dropped out of school had particularly poor marks in these modules. Overall, our results further support the work of Talamás-Carvajal et al. (2024), who found out that critical thinking is crucial for developing both complex thinking and other competencies.

Our study suggests that academic coordinators should review each subject’s syllabus to balance the number of times a (sub-)competency is evaluated depending on its level of difficulty, and to balance the level of difficulty in the sub-competencies that compose a competency, while keeping in mind the number of observations that on average are needed for a student to acquire a sub-competency.

Our study also concludes that there are no significant differences in the assessment of competencies due to the teacher or campus where the student is enrolled. Also, in our findings, we noticed that the number of credits allocated to a formative unit has no significant impact on the final grade obtained (on average) by students, and yet students who enroll in full program perform better that those who follow more or fewer training units than scheduled for the academic period, though only to a slight extent.

Having the competency evaluations for each student and academic period, our future work encompasses longitudinal studies analyzing the evolution of the students regarding their competencies evaluation through the academic periods by program, geographical region, and gender. Further work also involves conducting a contrast pattern mining approach so as to characterize interesting behavior, for example, discovering what distinguishes a student who successfully develops complex competencies from those who do not.