Evaluating the Impact of an Educational Intervention Using Project-Based Learning on Postpandemic Recovery in Rural Colombia

Abstract

This study evaluates the impact of a Project-Based Learning (PBL) intervention on postpandemic educational recovery in rural Colombia, focusing on student competencies in mathematics, language, science, and 21st-century skills. Conducted in rural schools, the intervention aimed to address significant learning gaps exacerbated by the COVID-19 pandemic by providing teacher training and direct student support. A pretest–posttest single-group design was used to assess the effectiveness of the intervention, with standardized tests measuring academic competencies and an analytical rubric evaluating 21st-century skills. The results indicate significant improvements in math, language, and science test scores, with notable gains in problem-solving, collaborative work, communication, and critical thinking. However, a decline in creativity scores highlights the need for a stronger emphasis on fostering creativity within the PBL framework. Gender differences were observed, with female students generally outperforming males, suggesting the need for tailored instructional approaches. This study’s limitations, including the absence of a control group, nonrandom sampling, and the use of subjective assessment methods, are acknowledged, with recommendations for future research to address these issues. Despite these limitations, the findings underscore the potential of PBL to enhance student learning outcomes in rural settings, offering valuable insights for educators and policymakers aiming to support educational recovery and development in similar contexts. Further research is recommended to explore the long-term effects of PBL and to refine the intervention for broader implementation.

1. Introduction

In the wake of the COVID-19 pandemic, education in Latin America and the Caribbean has faced a severe crisis, reflecting a loss of more than a decade of progress in learning. Even before the pandemic, over one-third of the region’s children and adolescents (35 million) were not reaching the minimum proficiency level in reading, and more than half (50 million) were failing to meet learning standards in mathematics [1]. The pandemic has only exacerbated these challenges, making the need of effective educational interventions more urgent than ever.

The 2022 PISA assessment highlights the ongoing challenges in Colombian students’ academic proficiency, showing both progress and areas where improvement is still needed. Although there have been gains since 2006 in mathematics, science, and reading, significant gaps persist when compared to OECD averages. In mathematics, Colombia’s average score increased by 13 points, yet it continues to fall substantially below the OECD benchmark. Similarly, while there have been improvements in science, recent trends suggest that progress has stalled, with scores still not reaching the OECD average. Reading scores have improved but continue to lag behind the standards set by the OECD [2].

These statistics underscore the formidable challenges faced by the educational system. In response to these learning gaps, the Center of Science and Technology of Antioquia (CTA), a nonprofit organization based in Medellín, Colombia, with funding from Fundación Fraternidad Medellín, developed and implemented an educational intervention called Alianza por la Educación con Calidad y Equidad—AECE (Alliance for Education with Quality and Equity). The intervention ran from 2010 to 2023 and focused on improving educational outcomes in rural areas by offering training to primary school teachers and a series of remedial sessions specifically tailored for third graders.

The teacher training covered the design, implementation, and evaluation of PBL activities aimed at enhancing children’s competencies in Spanish language, mathematics, science, and 21st-century skills [3], and the remedial activities were designed to elevate students’ performance to grade-level standards in Spanish language, math, and science. By focusing on both teacher training and direct student support, the intervention sought to address educational deficiencies holistically, fostering an environment where students could recover lost ground and develop essential academic skills. This dual approach not only equipped teachers with the necessary tools and methodologies to implement effective PBL but also provided immediate, targeted support to students who had fallen behind due to the disruptions caused by the pandemic.

This article focuses on the results from the 2022 cohort of third graders, who were the first group to return to in-person learning following the prolonged school closures caused by the COVID-19 pandemic. The primary purpose of this study is to evaluate the impact of Project-Based Learning (PBL) on student performance in the abovementioned areas and the development of 21st-century skills within this unique postpandemic educational context. Specifically, this study seeks to address the following research questions:

• How does Project-Based Learning (PBL) impact children’s competencies in Spanish language, mathematics, science, and 21st-century skills in the postpandemic context?
• What changes in student performance across these subjects and skills are observed between the pretest and posttest after the implementation of PBL?
• To what extent does gender influence the effectiveness of PBL in enhancing these competencies?

Assessing the effectiveness of educational interventions is essential for enhancing the basic competencies of children in rural areas and for informing decisions about resource allocation and policy development [4]. This cohort presents a critical opportunity to examine how PBL can address learning losses and facilitate skill development during a period of recovery and adjustment. The analysis aims to contribute to the existing literature on the effectiveness of PBL, particularly in fostering academic resilience and adaptability in the face of unprecedented educational challenges.

2. The Intervention

The AECE intervention was designed to improve the pedagogical practices of primary school teachers in rural areas of the Antioquia region of Colombia, with the goal of strengthening student competencies in the core subjects such as mathematics, Spanish, and science. Over the 13 years of the program’s implementation, it reached approximately 10,500 students and 2500 teachers. The intervention focused on training and supporting teachers in designing, implementing, and evaluating PBL strategies in their classrooms. Additionally, the program promoted the development of 21st-century skills to enhance the quality of education in rural schools.

2.1. Pedagogical Foundations of the Intervention

The educational intervention was founded on an active learning approach [5,6,7] that emphasized the development both of foundational competencies in language, mathematics, and science and of essential 21st-century skills. This dual focus aimed to prepare students not only with the academic knowledge needed for success but also with the soft skills critical for navigating the complexities of modern life. Central to the intervention’s design and implementation was PBL, a pedagogical strategy that actively engages students in real-world projects. PBL encourages learners to take ownership of their education by fostering critical thinking, problem-solving, and collaboration skills, which are integrated with core academic content [8,9,10].

Research on PBL has demonstrated its effectiveness in improving foundational skills across subjects such as language [11,12], mathematics [13,14], and science [15,16]. PBL engages students in real-world projects and has been associated with increased student engagement and improved learning outcomes [17,18]. A key element of PBL is its alignment of assessment strategies with project objectives, allowing students to be evaluated on both academic understanding and their ability to apply these skills in practical contexts [19,20,21].

Similarly, the integration of 21st-century skills into education has become increasingly recognized as essential. These skills, which include critical thinking, collaboration, and digital literacy, are necessary for students’ personal development and their ability to succeed in an ever-changing world [3,22,23,24]. The incorporation of these competencies into educational frameworks aligns with the goals of PBL, as both approaches emphasize the importance of preparing students to meet the complex challenges and opportunities they will face in various aspects of life. Both PBL and 21st-century skills share a focus on preparing students not only academically but also in developing the practical abilities needed for lifelong learning and problem-solving. [23]. PBL supports the acquisition of 21st-century skills by promoting an active learning environment where students are encouraged to think critically, collaborate effectively, and apply their knowledge in diverse situations [25].

However, integrating 21st-century skills into classrooms presents challenges. There is often a gap between recognizing their importance and fully incorporating them into practice [26,27]. Traditional educational approaches, which tend to compartmentalize subjects like mathematics and science, can hinder interdisciplinary learning and limit real-world application opportunities [28].

In contrast, PBL offers a holistic approach, embedding these skills into thematic learning that fosters both academic and interpersonal growth. It encourages students to engage in collaborative projects that require critical thinking, problem-solving, and effective communication, all essential to 21st-century competencies [27]. Research has shown that PBL not only enhances student motivation and engagement but also improves their ability to work collaboratively and think critically [29]. By linking theory to real-world applications, PBL can bridge the gap between knowledge and practice [30].

2.2. Evaluative Foundations

The evaluation of learning outcomes in the intervention draws upon the Evidence-Centered Design (ECD) framework developed by Mislevy and colleagues [31,32,33,34], which provides a systematic and structured approach to educational assessment. This framework has been adapted to meet the unique requirements of users of the program. At the core of the ECD framework are three interconnected models, collectively forming what is referred to as the Conceptual Assessment Framework (CAF): the proficiency model, the task model, and the evidence model (Figure 1) [32].

The proficiency model forms the basis of the assessment strategy, identifying the knowledge, skills, and abilities (KSA) targeted by the intervention. These competencies are aligned with national standards for math, reading, and science education [35]. The proficiency model breaks down these competencies into specific skills and subskills, providing a clear framework for what students need to achieve. This structure guides the design of teacher training and student remediation activities and the subsequent evaluation of student learning.

The task model translates latent competencies into observable behaviors by defining the key features of assessment tasks and items. It ensures that these tasks effectively elicit student behaviors that reflect their competencies. In this intervention, the task model followed Norman Webb’s Depth of Knowledge (DOK) framework [36,37,38], which categorizes cognitive demands into various levels and guides task creation. This approach ensures assessments address a range of cognitive processes, including recall, reasoning, application of skills, and deeper conceptual understanding. For example, tasks designed to assess data interpretation from graphs focus on both recalling relevant information and applying interpretation and problem-solving skills.

The evidence model connects the latent competencies defined in the proficiency model with the observable behaviors elicited by the task model. It encompasses key elements of measurement, including the development of tests, rubrics, scoring methods, and statistical analyses for establishing students’ performance. The tests and rubrics are designed to align with the competencies outlined in the proficiency model and the tasks specified by the task model. These assessments incorporate multiple item types—such as multiple-choice, short answer, and performance-based tasks—to provide a comprehensive evaluation of student abilities. Clear criteria for evaluating student responses were established to ensure consistency and objectivity by defining specific performance levels. As shown in Figure 2, the intervention assessed students using two instruments: a standardized test measuring competencies and cognitive processes in language, mathematics, and science, and an analytical rubric [39] assessing 21st-century skills. For teachers, an observation rubric was used to track the implementation of PBL-based learning strategies in the classroom.

2.3. Implementation of the Intervention

The implementation of the intervention was designed to be comprehensive, involving both teacher development and student support. This approach was organized around three distinct stages. The program ran from 2010 to 2023, with a new cohort of teachers and students participating each year. This study specifically focuses on the 2022 cohort, offering insights into the program’s impact during this period, particularly in the context of postpandemic educational recovery. A detailed timeline of the intervention stages is included in Appendix A.

2.3.1. Stage 1: Planning

In the planning phase, participating schools were identified and engaged to establish the foundation for the intervention. The intervention was designed using a Theory of Change (TOC) framework [40,41], which provided a structured roadmap linking specific inputs, activities, and outputs to intended outcomes, as summarized in the logic model shown in Figure 3. This TOC approach enabled the identification of key performance indicators and milestones, allowing for effective tracking of progress and adjustments to improve program management. By clearly defining how each component contributed to the overall goals, the TOC model ensured that the intervention remained focused on measurable outcomes aligned with student and teacher development objectives.

The TOC was then refined to align with the program’s educational goals, and assessment instruments were developed and piloted to measure both student competencies and teacher practices accurately. Instructional materials were created to address the specific needs of the students and teachers participating in the intervention. The selection and training of educational advisors supported the implementation process. Additionally, remediation materials were revised to target specific learning gaps among the students, ensuring the intervention addressed their educational needs.

2.3.2. Stage 2: Baseline Assessment

In the second phase, baseline assessments of students’ competencies in math, science, and the Spanish language were conducted using cognitive instruments that were specifically designed to align with the intervention’s learning objectives. These baseline assessments, while similar in structure to the posttest instruments, included different items to maintain test validity while measuring the same competencies and standards targeted in the intervention. Like the posttest, 21st-century skills were assessed using a structured rubric to capture students’ proficiency across critical skills like problem-solving and collaboration.

Additionally, teachers’ instructional and evaluation practices were evaluated through a multimethod approach, including structured interviews, classroom observations, self-reported questionnaires, and samples of teaching plans and student work. These assessments provided a comprehensive understanding of both students’ and teachers’ starting points, enabling more accurate measurement of progress and targeted support throughout the intervention.

2.3.3. Stage 3: Delivery

The delivery phase encompassed three main components: professional development sessions for teachers, the implementation of PBL in classrooms, and targeted remedial sessions for students facilitated by CTA advisors. This phase began with eight professional development sessions for teachers, focusing on PBL. These sessions covered both theoretical and practical aspects of designing, implementing, and evaluating PBL-based teaching and learning activities within instructional units. Topics included curriculum alignment, project design, student engagement strategies, formative assessment techniques, and integrating 21st-century skills into classroom activities.

Following the initial training, teachers implemented the PBL methodology over a period of approximately nine months, structuring instruction into two or three PBL cycles. Each cycle included a series of steps designed to engage students in real-world, interdisciplinary projects that reinforced core academic skills and competencies, as explained below:

Initial project selection and planning. Teachers began each cycle by identifying themes that integrated math, science, and language content. Themes were aligned with curriculum standards and chosen based on their relevance to the students, allowing for meaningful learning objectives in each subject.

Launching the projects. To spark curiosity and highlight the interdisciplinary nature of each project, teachers introduced projects with a “driving question.” For example, a question on environmental conservation, “How can we protect local plants and animals?”, framed learning in science, math (data analysis), and Spanish language (reporting and presentations).

Guided inquiry and research. Teachers facilitated structured inquiry activities, guiding students through research to deepen their understanding of the project topics. This phase was supplemented with classroom resources and field trips, with teachers helping students form hypotheses and explore critical questions from multiple disciplinary perspectives.

Collaborative work sessions. Students were allocated collaborative work periods to pursue project goals. Each subject was addressed in targeted sessions—math for data collection, science for experiments, and language for written or oral presentations. Teachers supported students by providing guidance, addressing questions, and promoting diverse approaches to problem-solving.

Regular formative assessments. Throughout each project, formative assessments allowed teachers to observe and provide feedback on students’ academic and skill development. Peer and self-assessments were also used to track growth in critical areas, such as collaboration and communication.

Reflection and self-assessment. Teachers regularly set aside time for students to reflect on their progress, discuss challenges, and identify areas for improvement. This step reinforced students’ sense of ownership over their learning and encouraged self-assessment skills.

Final presentations and evaluations. At the conclusion of each project, students presented their work, integrating math, science, and language elements. Teachers evaluated students’ final outputs and interdisciplinary skills using structured rubrics.

Continuous adjustment and support. Teachers refined each PBL cycle based on observations of student progress. Six in situ follow-up sessions with PBL facilitators and three meetings with school administrators provided additional guidance and alignment support throughout the year, ensuring teachers could effectively tailor their strategies to meet the needs of their students.

The remedial sessions for students were designed to address specific learning gaps in reading, math, and science, aiming to bring participants up to grade-level standards. These sessions were structured to offer targeted support through individualized instruction, group activities, and problem-solving exercises that reinforced core academic concepts. The curriculum focused on strengthening foundational skills that had been disrupted due to the COVID-19 pandemic, with a particular emphasis on interactive learning and real-world applications.

A total of 27 remedial sessions were conducted, with 24 sessions planned and facilitated by experienced pedagogical advisors hired by the CTA. These sessions employed a combination of direct instruction, hands-on learning activities, and formative assessments to monitor student progress. The remaining three sessions were designed and implemented by the teachers who participated in the professional development program, allowing them to apply their newly acquired skills in PBL. Throughout these sessions, students were encouraged to engage in collaborative tasks, critical thinking, and applied learning to help them build both subject-specific competencies and 21st-century skills. The remedial program aimed not only to improve academic performance but also to foster greater confidence and engagement in the learning process.

Based on the program’s Theory of Change (Figure 3), the inputs and activities were expected to increase literacy, numeracy, and science skills among students. Additionally, teachers were expected to become more aware of and skilled in using PBL and active methodologies in their classrooms, leading to improved instructional and evaluation practices (Figure 2).

At the end of the intervention, students and teachers were assessed to identify the improvements in student competencies and the effectiveness of the professional development for teachers. These assessments aimed to measure the overall impact of the program and to gather insights for future educational initiatives.

3. Materials and Methods

3.1. Participants

As mentioned before, this study focused on the 2022 cohort of third-grade students from the Antioquia region of Colombia. The program engaged a diverse group of participants, comprising a total of 287 students, with 150 boys and 137 girls from nine rural schools across seven different municipalities. This distribution ensured a balanced representation of gender and a wide geographic reach, aligning with the program’s goal of improving educational outcomes in underserved rural areas. To maintain ethical standards, informed consent was obtained from all participating students’ parents or guardians. The names of the schools have been anonymized to protect the privacy of participants and institutions, as shown in

Table 1. Anonymized distribution of program participants by location, school, and gender (n = 287).

Municipality	School	n	Percentage (%)
Municipality A	School A	31	10.8
Municipality B	School B	58	20.21
Municipality C	School C	59	20.56
Municipality D	School D	29	10.1
Municipality E	School E	17	5.92
	School F	22	7.67
Municipality F	School G	9	3.14
Municipality G	School H	24	8.36
	School I	38	13.24
	Total	287	100

.

3.2. Instruments

A multifaceted assessment strategy was implemented to evaluate program outcomes, measuring students’ mastery of core competencies and cognitive abilities in language, mathematics, and science (

Table 2. Selected standards addressed by the program [35].

Area	Standard	Evidence	Number of Items
Language	Reading comprehension and critical analysis	The student can understand the content of a text by analyzing its structure and employing inferential and critical reading processes.	12
	Text production and composition	The student can produce various types of texts (expository, narrative, informative, descriptive, argumentative) while considering grammatical and orthographic conventions.
Science	Understanding abiotic and biotic interactions	The student can explain the influence of abiotic factors (light, temperature, soil, air) on the development of biotic factors (fauna, flora) within an ecosystem.	20
	Ecological relationships and survival	The student can understand and explain the intra- and interspecific relationships between organisms and their environment, emphasizing their importance for survival.
Mathematics	Estimation and mathematical strategies	The student can propose, develop, and justify strategies for making estimates and performing basic operations to solve problems effectively.	12
	Data interpretation and problem-solving	The student can read and interpret information from frequency tables, bar graphs, and pictograms with scales to formulate and solve questions related to real-world situations.

). To align assessments with the program’s learning objectives, an evidence model was developed based on the competencies outlined in the proficiency model and their corresponding observable behaviors in the task model [32]. This model incorporated a suite of assessments, including standardized pre- and posttests, a rubric for assessing 21st-century skills, scoring methodologies for each instrument, and statistical analysis to determine the competency level of each student (the instruments used in the intervention are available at https://github.com/ricardophd/cta-pbl-evaluation/blob/main/PBL-assessment-instruments (accessed on 17 September 2024)).

Language assessment focused on inferential and critical reading, using a 12-item test to evaluate comprehension, analysis, and critical thinking skills. The science assessment centered on ecological concepts, with a 20-item test assessing students’ understanding of abiotic and biotic factors and their interactions. The mathematics assessment measured students’ ability to estimate, calculate, and justify their reasoning through 12 questions evaluating computational skills, problem-solving, and logical reasoning (Table 2).

Test items were aligned with Webb’s Depth of Knowledge (DOK) framework [36,37,38], encompassing recall, application, and strategic thinking levels. Recall items tested basic knowledge, application items assessed practical application, and strategic thinking items evaluated the students’ capacity to analyze, synthesize, and make decisions based on their understanding. By integrating these components, the evidence model created an assessment system that provided a detailed picture of student learning and competencies.

To ensure the reliability and validity of these assessments, two key processes were implemented: cognitive laboratories and pilot testing. Cognitive labs involved engaging students with similar characteristics to the target population to understand how they interpreted and responded to the test items [42,43]. Researchers observed their thought processes and provided feedback on potential ambiguities or misinterpretations, ensuring the questions were clear and effectively measured the intended skills. Following the cognitive labs, pilot tests were conducted with a broader group of students. This allowed for the analysis of item difficulty and discrimination, further refining the tests to enhance reliability and validity [44,45,46].

To complement the quantitative data from standardized tests, the program incorporated an assessment of 21st-century skills, prioritizing problem-solving, collaboration, creativity, communication, and critical thinking (

Table 3. Twenty-first-century skills addressed by the intervention [3].

Skills	Description
Problem-solving	The ability of the student to identify a contextual problem and propose and implement alternative solutions.
Collaboration	The ability of the student to take on established roles, listen to peers, and contribute ideas in an organized manner.
Creativity	The ability of the student to represent the solution to the problem through a product, using various resources for its creation and strategies for its presentation.
Communication	The ability of the student to convey ideas or opinions clearly and coherently, both verbally and nonverbally.
Critical thinking	The ability of the student to analyze, evaluate, and synthesize information in a thoughtful and systematic way. It involves the capacity to question assumptions, identify biases, assess evidence, and consider alternative perspectives.

), as outlined in the Partnership for 21st-Century Skills framework [3]. These competencies are important for navigating the complexities of contemporary society.

A structured analytical rubric [39,47] was used to assess these skills in real-world contexts (

Table 4. Twenty-first-century skills assessment rubric.

21st-Century Skills	Assessment Criteria	Score (from 0 to 3)
Problem-solving	☐ Identifies a problem
	☐ Searches for possible solutions	Score
	☐ Implements a solution
Collaborative work	☐ Assumes established roles
	☐ Listens to peers	Score
	☐ Contributes ideas in an organized manner
Creativity	☐ Represents the solution through a product
	☐ Utilizes various resources in product creation	Score
	☐ Applies strategies to present the product
Communication	☐ Conveys ideas verbally or nonverbally
	☐ Communicates clear ideas	Score
	☐ Expresses coherent solutions
Critical thinking	☐ Analyzes information
	☐ Interprets arguments	Score
	☐ Supports solutions

Score definitions: 0 = Beginning; 1 = In Progress; 2 = Achieved; 3 = Extended.

), providing a comprehensive framework for evaluating student performance across the five skill dimensions. To ensure reliability, calibration sessions were conducted with trained evaluators, establishing inter-rater reliability (IRR) by standardizing scoring and minimizing subjective variation [48,49]. While the calibration process ensured consistency, the final application of the rubric was performed by single raters in each group. This pragmatic approach balanced the consistency gained through calibration with the efficiency needed for large-scale assessments, a common practice when resources and logistics limit the use of multiple raters. Research supports that careful calibration can mitigate biases in single-rater assessments, maintaining acceptable reliability levels [39,49].

The integration of these assessment components provided a comprehensive perspective on student capabilities, extending beyond cognitive skills to include essential competencies for modern learning. This approach aligns with the program’s objectives, emphasizing the development of critical thinking, problem-solving, and collaborative abilities as prerequisites for success in education, the workforce, and life.

3.3. Data Collection Procedures

The intervention employed a pretest–posttest single-group design, incorporating standardized tests and classroom observations. For the language component, n = 264 students completed the pretest, n = 226 took the posttest, and n = 196 had both sets of results for analysis. In science, n = 265 students completed the pretest, n = 221 took the posttest, with n = 190 providing both pretest and posttest results. For mathematics, n = 267 students completed the pretest, n = 233 took the posttest, and n = 201 had both results available for analysis. The tests were administered by CTA advisors in a controlled environment to ensure consistency and reliability. Students were provided with clear instructions, monitored for compliance, and given breaks as needed. The tests were administered in a structured, uniform manner, with strict adherence to time limits.

Additionally, n = 268 students were observed at the beginning of the intervention, and n = 273 were observed at its conclusion, with n = 238 having both pre- and postintervention observation results. These observations were conducted by CTA advisors who filled out analytical rubrics to assess the application of 21st-century skills during the implementation of PBL activities. The difference in the number of students between the pretest and posttest was due to factors such as absences, changes in enrollment, or logistical challenges that prevented some students from completing both assessments.

Upon completion, tests and rubrics were collected and entered into LimeSurvey [50] for data streamlining, management, formatting, and creation of data dictionaries.

3.4. Data Preparation and Analysis

3.4.1. Cognitive Tests

Student responses were processed to generate a global score for each student in each subject—language, mathematics, and science—using a weighted approach based on the cognitive demands of the assessment items. Each item was categorized as measuring recall, application, or strategic thinking, with weights assigned to reflect the complexity of each item type. After applying these weights, the scores were normalized using z-scores to standardize the distribution of responses.

Subsequently, a global score for each student in each subject was calculated. To facilitate comparison across subjects, these global scores were converted to a t-scale with a mean (M) of 50 and a standard deviation (SD) of 10 [51,52]. This transformation ensured consistency in measuring student performance across subjects and provided a standardized metric for evaluating individual student achievement within the intervention.

In addition to global scores, a measure of competency level was calculated based on students’ performance on each task or item within the tests. The criteria used to determine each performance level are shown in

Table 5. Established performance levels based on students’ task completion.

Performance Level	Probability Threshold	Description
Beginning	Probability of answering recall-level questions correctly is less than 50%.	Students at this level are just starting to grasp basic concepts and often struggle with foundational knowledge.
In Progress	Probability of answering recall-level questions correctly is greater than 50% and application-level questions correctly is less than 50%.	Students at this level show some understanding of basic concepts but need further development in applying their knowledge to practical scenarios.
Achieved	Probability of answering application-level questions correctly is greater than 50% and strategic thinking-level questions correctly is less than 50%.	Students at this level have met the standards for their age and grade, demonstrating a good grasp of both basic and applied concepts.
Extended	Probability of answering strategic thinking-level questions correctly is greater than 50%.	Students at this level exhibit a strong understanding of both basic and advanced concepts, applying their knowledge effectively and engaging in higher-order thinking tasks.

. These competency levels were established to offer a clearer understanding of students’ progression and areas of improvement, enabling the identification of specific strengths and weaknesses and informing targeted interventions and support.

Changes in global scores between pre- and posttests were evaluated using paired t-tests for each subject area (reading comprehension, mathematics, and science) to determine whether there were differences in mean scores. To account for potential non-normality in the data, Wilcoxon rank-sum tests were conducted as complementary analyses to the t-tests. These nonparametric tests verified the results obtained from the t-tests and provided additional robustness to the findings. Additionally, a repeated measures ANOVA was implemented to assess overall score changes and explore potential gender-based differences in performance trajectories. This analysis provided insights into the general trends in score changes and identified any significant interactions between gender and measurement conditions. All data analyses and visualizations were conducted using R [53].

3.4.2. Evaluation of Students’ 21st-Century Skills

The 21st-century skills rubric scores were used to generate a global score through a multistep process. First, raw scores were calculated for each of the five dimensions: problem-solving, collaboration, creativity, communication, and critical thinking. These raw scores were then standardized into z-scores to allow comparability across the different dimensions.

Subsequently, each z-score was converted to a 0–10 scale using a normalization function that adjusted values based on the minimum and maximum scores within each dimension. This ensured that all dimensions were on the same scale, facilitating direct comparisons between them.

Following this, an overall score for 21st-century skills was calculated by summing the standardized scores for each dimension. To enhance interpretability, the overall score was also converted to a 0–10 scale, aligning it with the individual dimension scores. Additionally, students’ performance levels for each 21st-century skill dimension were determined by first aggregating individual assessment items into total scores for each competency and then assigning a performance level based on the total score: “Beginning” for a score of 0, “In Progress” for a score of 1, “Achieved” for a score of 2, and “Extended” for a score of 3.

4. Results

4.1. Results of Pretest and Posttest Assessment

This section presents the results of statistical tests conducted to evaluate changes in student performance between pretest and posttest conditions. The analyses include paired t-test and Wilcoxon rank-sum tests to determine significant differences in global test scores, as well as repeated measures ANOVA to examine overall score changes and any gender-based differences in performance.

4.1.1. Language

The assessment results indicate an increase in language test scores from the pretest to the posttest. For girls, the mean score increased by approximately 2 points, rising from 52.1 (SD = 10.2) on the pretest to 54.2 (SD = 8.9) on the posttest. Similarly, boys’ mean scores increased from 47.6 (SD = 10.5) on the pretest to 49.6 (SD = 9.3) on the posttest (Figure 4).

Table 6. Paired samples t-test language test scores.

Test	Statistic	z	df	p	Effect Size	SE Effect Size
Student	−3.551		19	<0.001	−0.254	0.057
Wilcoxon	5440.000	−3.161		0.002	−0.277	0.087

Note. For the Student t-test, effect size is given by Cohen’s d. For the Wilcoxon test, effect size is given by the matched rank biserial correlation.

below presents the results of the paired samples t-test comparing the language test scores between the pretest and posttest conditions.

The paired t-test produced a t-statistic of −3.5515 with 195 degrees of freedom and a highly significant p-value (<0.001). The effect size, as indicated by Cohen’s d, is −0.254 with a standard error of 0.057, suggesting a moderate effect size.

The Wilcoxon test produced a test statistic of 5440.000 with a z-value of −3.161 and a significant p-value (0.002). The effect size for the Wilcoxon test, given by the matched rank biserial correlation, is −0.277 with a standard error of 0.087, also indicating a moderate effect. Overall, the results of both statistical tests for the language test suggest a significant change in test scores between the pretest and posttest conditions.

The results of the analysis of variance (ANOVA) assessing the impact of gender and measurement condition (pretest–posttest) on language test scores are presented in

Table 7. Impact of gender and measurement on language scores.

Source of Variation	Df	Sum of Squares	Mean Square	F-Value	p-Value
Error: id	1	112	112
Error: id: measurement	1	1.999	1.999
Error: within	386	36,851	95.5
Gender	1	2035	2034.8	21.313	0.0000 *
Type of test	1	396	396.0	4.148	0.0424 *
Gender: type of test	1	1	1.5	0.015	0.9014

The asterisk (*) in the table indicates statistical significance at p < 0.005.

.

The analysis revealed statistically significant main effects for both gender and type of measurement condition on language test scores. Specifically, the gender factor demonstrated a significant impact, with an F-value of 21.313 and a p-value of 0.0000, indicating that gender significantly influenced test scores. Similarly, the type of measurement exhibited a significant main influence, with an F-value of 4.148 and a p-value of 0.0424, signifying its substantial impact on scores. However, the interaction between gender and type of test was not statistically significant (F = 0.015, p = 0.9014), suggesting that the way the scores changed from pretest to posttest did not differ significantly between genders.

4.1.2. Mathematics

The assessment results indicate an increase in math test scores from the pretest to the posttest (Figure 5). For girls, the mean score increased by approximately 2 points, rising from 52.1 (SD = 10.2) on the pretest to 54.2 (SD = 8.9) on the posttest. Similarly, boys’ mean scores increased from 47.6 (SD = 10.5) on the pretest to 49.6 (SD = 9.3) on the posttest

As shown in

Table 8. Paired samples t-test math test scores.

							95% CI for Effect Size
Test	Statistic	z	df	p	Effect Size	SE Effect Size	Lower	Upper
Student	−6.44		200	<0.001	−0.454	0.068	−0.599	−0.308
Wilcoxon	4355	−5.824		<0.001	−0.494	0.085	−0.609	−0.358

Note. For the Student t-test, effect size is given by Cohen’s d. For the Wilcoxon test, effect size is given by the matched rank biserial correlation.

, The paired t-test yielded a t-statistic of −6.44 with 200 degrees of freedom and a p-value of <0.001. This result provides evidence of a significant difference in mean test scores between the two measurement conditions. The Wilcoxon rank-sum test yielded a test statistic (W) of 4355 and a p-value of <0.001. This result confirms the results of the paired t-test and provides strong evidence of a significant difference in test scores between the two measurement conditions

The repeated measures ANOVA test showed that the type of test (pretest or posttest) had a significant impact on math test scores. The F-value of 19.312 and a p-value of less than 0.001 mean that the difference in scores before and after the intervention is statistically significant (

Table 9. Impact of gender and measurement on math scores.

Source of Variation	Df	Sum of Squares	Mean Square	F-Value	p-Value
Error: student	1	1.779	1.779
Error: student: measurement	1	49.96	49.96
Error: within	396	38,003	96.0
Gender	1	192	191.8	1.999	>0.05
Type of test	1	1853	1853.3	19.312	<0.001
Gender: type of test	1	189	189.4	1.974	>0.05

). This result indicates that the intervention had a meaningful effect on the students’ math performance.

4.1.3. Science

The assessment of scientific skills showed a significant change between pretest and posttest scores (Figure 6).

The analysis produced a test statistic (W) of −2.903 with 189 degrees of freedom and a p-value of 0.004135. The p-value indicates a significant difference in mean test scores between the two measurement conditions. Both the Wilcoxon rank-sum test and the paired t-test provide evidence of a significant change in science test scores between the pretest and posttest conditions (

Table 10. Summary of test statistics, effect sizes, and confidence intervals for math test scores.

							95% CI for Effect Size
Test	Statistic	z	df	p	Effect Size	SE Effect Size	Lower	Upper
Student	−2.903		189	0.004	−0.211	0.085	−0.354	−0.067
Wilcoxon	6322	−2.277		0.023	−0.197	0.086	−0.354	−0.03

Note. For the Student t-test, effect size is given by Cohen’s d. For the Wilcoxon test, effect size is given by the matched rank biserial correlation.

).

The results of the analysis of variance (ANOVA) indicate a significant shift in science test scores between the pretest and posttest conditions, as evidenced by a highly significant F-value (F(1, 49.96) = 17.997, p < 0.001). Specifically, there is a significant variation in scores between the two measurement conditions (

Table 11. Summary of ANOVA results for science scores.

Source of Variation	Df	Sum of Squares	Mean Square	F-Value	p-Value
Error: studentid	1	1.892	1.892	0.077	0.781
Error: studentid: test	1	49.96	49.96	17.997	2.77 × 10−05 *
Error: within	390	37,800	96.9
Gender	1	7	7.5	0.077	0.781
Test	1	1744	1744.3	17.997	2.77 × 10−05 *
Gender: test	1	104	103.5	1.068	0.302

The asterisk (*) in the table indicates statistical significance at p < 0.005.

).

For female students, the scores increased from an average of 47.5 in the pretest to 52.7 in the posttest, while male students experienced an increase in scores from 48.2 in the pretest to 51.5 in the posttest. These changes were observed within each gender group.

However, when considering the influence of gender on test scores, the ANOVA results indicate that it did not appear to significantly contribute to the observed variations in test scores (F(1, 1.892) = 0.077, p = 0.781).

The ANOVA results demonstrate a significant change in science test scores between the pretest and posttest conditions, with improvements observed for both female and male students. However, gender itself did not emerge as a statistically significant factor affecting these score differences.

These statistics provide a detailed view of the test score distribution within different groups. Notably, the mean scores for both genders tend to be slightly higher under the posttest measurement condition compared to the baseline with corresponding standard deviations indicating variability in scores.

4.2. Assessment of Students’ Competency Level by Subject

4.2.1. Language Competency Level

The results of the language component of the intervention reveal changes in student performance levels from the pretest to the posttest (Figure 7). At the baseline (pretest), the distribution of students was as follows: 27.0% of students were at the Extended level, 22.4% had Achieved the expected competencies, 30.6% were at the Beginning level, and 19.9% were In Progress.

By the endline assessment, the distribution showed a shift towards higher performance levels. The percentage of students at the Extended level increased slightly to 28.1%, while those in the Achieved category saw a more significant rise to 30.1%. Meanwhile, the proportion of students at the Beginning level decreased to 18.4%, and those In Progress increased to 23.5%.

These results suggest an overall improvement in language competencies, with more students advancing to the higher performance levels (Achieved and Extended) by the endline assessment. The decrease in the Beginning level indicates that fewer students remained at the lowest level of performance, while the increase in the In Progress level suggests that some students were still transitioning toward achieving proficiency.

4.2.2. Mathematics Competency Level

As shown in Figure 8, prior to the intervention, 31.3% of students had Achieved the expected competencies, 28.9% were In Progress, 23.9% were at the Beginning level, and 15.9% were at the Extended level.

By the posttest assessment, the distribution of performance levels showed notable changes. The percentage of students at the Extended level increased significantly to 39.3%, reflecting a considerable improvement. However, the percentage of students who Achieved the expected competencies decreased to 19.4%. Meanwhile, the proportion of students In Progress decreased to 18.4%, and those at the Beginning level also saw a slight reduction to 22.9%.

4.2.3. Science Competency Level

The results of the science component of the intervention, as shown in Figure 9, indicate changes in student performance across the four competency levels. At the baseline, the distribution of students was as follows: 22.6% of students were at the Beginning level, 32.6% were In Progress, 24.2% had Achieved the expected competencies, and 20.5% were at the Extended level. By the endline assessment, the distribution shifted, with 18.9% of students remaining at the Beginning level, a decrease from the baseline. The percentage of students In Progress remained constant at 24.2%, while those in the Achieved category increased to 25.8%. Notably, the percentage of students at the Extended level rose significantly to 31.1%. These results suggest a positive shift in student performance, with a notable increase in the proportion of students reaching the higher performance levels (Achieved and Extended) by the endline assessment. The decrease in the Beginning level indicates that fewer students remained at the lowest level of performance, reflecting overall improvement in the science competencies targeted by the intervention.

These results suggest a significant shift towards higher performance levels, particularly with the large increase in students reaching the Extended level by the posttest assessment. The decrease in both the Achieved and In Progress categories may indicate that some students who were previously meeting or nearly meeting the expected standards advanced further to the Extended level, while the slight reduction in the Beginning level suggests an overall improvement in foundational skills.

4.3. Assessment of 21st-Century Skills

As illustrated in Figure 10, the changes in participants’ standardized scores in 21st-century skills indicate that both gender and the educational intervention had a significant impact on student outcomes. The analysis, using both the Wilcoxon rank-sum test and the paired samples t-test, indicated statistically significant improvements in students’ overall scores between the pretest and posttest.

For the Wilcoxon rank-sum test, the results revealed a significant difference between the overall scores at baseline and endline (W = 7829.500, p < 0.001, 95%CI −0.461 to −0.189). This suggests that there was a shift in the distribution of scores between the two time points, indicating that the students’ overall 21st-century skills improved significantly after the intervention.

Similarly, the paired t-test results corroborate this finding (

Table 12. Summary of test statistics, effect sizes, and confidence intervals for 21st-century skills scores.

Test Type	Statistic	Z	Df	p	Effect Size	SE Effect Size	95% CI Lower	95% CI Upper
Student t-test	−3.506		235	<0.001	−0.228	0.078	−0.357	−0.099
Wilcoxon test	7829.500	−4.229		<0.001	−0.332	0.078	−0.461	−0.189

Note: For the Student t-test, effect size is given by Cohen’s d. For the Wilcoxon test, effect size is given by the matched rank biserial correlation.

). The test showed a statistically significant mean difference of −0.641 (t = −3.506, df = 237, p < 0.001, 95%CI −0.357 to −0.099). This indicates a significant increase in the overall score for 21st-century skills between the baseline and endline assessments, further confirming the positive impact of the intervention on students’ skill development.

The ANOVA results further confirm this, revealing significant main effects for both gender and measurement condition on the overall standardized scores. As shown in

Table 13. Analysis of variance (ANOVA) results for the effects of gender and measurement condition on overall standardized scores in 21st-century skills.

Source	Df	Sum Sq	Mean Sq	F-Value	Pr (>F)
Gender	1	35.1	35.09	7.075	0.00808
Measurement condition	1	49.0	48.96	9.874	0.00178
Gender: measurement	1	17.5	17.51	3.532	0.06082
Residuals	472	2340.7	4.96

, there was a significant main effect of gender on the overall scores, F(1, 472) = 7.075, p = 0.008. This result indicates that there were statistically significant differences in the overall performance between male and female students.

Additionally, the analysis showed a significant main effect of measurement condition, F(1, 472) = 9.874, p = 0.002, suggesting that the intervention had a substantial impact on student performance, with overall scores differing significantly between the pretest and posttest conditions.

Furthermore, the interaction between gender and measurement condition approached significance, F(1, 472) = 3.532, p = 0.061. Although this interaction was not statistically significant at the conventional 0.05 level, it indicates a possible trend where the effect of the intervention on overall scores may vary between male and female students. This finding suggests that gender may play a role in how students respond to the intervention, warranting further investigation.

Figure 11 illustrates the changes in mean scores for the five 21st-century skills targeted by the intervention—problem-solving, collaborative work, creativity, communication, and critical thinking—comparing baseline and endline assessments. Problem-solving saw an increase in the mean score from 6.40 at baseline to 6.72 at endline, accompanied by a slight rise in the standard error, indicating increased variability among students. Similarly, the mean score for collaborative work improved from 6.48 to 6.99, with a corresponding increase in standard error from 0.15 to 0.22, suggesting both better average performance and greater variation in outcomes.

In contrast, creativity showed a decrease in the mean score from 6.78 at baseline to 6.13 at endline, while the standard error remained relatively stable, increasing slightly from 0.16 to 0.22. This suggests a decline in overall performance with a modest increase in variability. Communication demonstrated a significant improvement, with the mean score rising from 5.35 to 6.37 and an increase in standard error from 0.19 to 0.21, indicating better average performance and more diverse outcomes among students. Finally, critical thinking experienced a substantial increase in mean score, from 4.57 to 6.57, with the standard error also rising from 0.16 to 0.20, reflecting an overall enhancement in critical thinking skills alongside greater variability. Overall, these changes suggest that except for creativity, students improved in most assessed skills, though with varying degrees of increased variability.

5. Discussion

The findings of this study provide evidence supporting the effectiveness of Project-Based Learning (PBL) in enhancing student competencies across language, mathematics, science, and 21st-century skills. This aligns well with existing research indicating the benefits of PBL on academic performance and skill development across multiple domains [54,55,56]. However, the current study also highlights areas where PBL’s impact varies, offering valuable insights into both its strengths and its limitations.

In the language domain, the intervention led to significant improvements in student scores, with a moderate effect size. Gender differences were noted, with girls generally outperforming boys, echoing findings from previous studies that report similar gender-based trends in language learning outcomes [57,58]. These findings underscore the need to consider gender dynamics within PBL frameworks, as girls and boys may engage with and benefit from PBL differently. Nonetheless, the absence of an interaction between gender and measurement condition suggests that PBL is effective for both genders, positioning it as a potentially equitable teaching approach that fosters improvement across student groups [59].

Similarly, in mathematics, this study demonstrated significant gains from pretest to posttest scores, confirming PBL’s role in bolstering mathematical competencies. These results align with the existing literature showing that PBL can drive meaningful improvements in math through active learning and real-world application [54,55,56]. While gender differences were not significant in this area, sustaining student engagement throughout PBL projects remains an ongoing challenge. Previous research highlights the importance of maintaining student motivation to maximize the benefits of PBL, especially in disciplines that require continuous skill application and engagement [59].

For science, the intervention yielded comparable gains, with statistically significant improvements in scores. Consistent with studies that show PBL’s effectiveness in fostering scientific inquiry and critical thinking, this study found that PBL can engage students in meaningful scientific learning without significant gender disparities in outcomes [55,60]. These results reinforce PBL’s versatility as an instructional method that enhances students’ understanding and engagement in science by promoting inquiry and problem-solving skills.

The assessment of 21st-century skills offered further insights into the intervention’s broader impact. While improvements were observed in problem-solving, collaborative work, communication, and critical thinking, a decline in creativity scores suggests an area for future refinement. This aligns with existing concerns that while PBL encourages critical thinking and collaborative skills, it may require additional support to nurture creativity fully [28,61]. Increased variability in postintervention scores across these skills suggests that while average performance improved, the effects of PBL may differ significantly among student groups, indicating diverse individual responses to the intervention [62].

Overall, these findings highlight both the potential and complexity of PBL as an instructional strategy. The gains observed across most competencies support PBL’s relevance in postpandemic educational recovery, especially in rural contexts. However, the variability in outcomes and specific challenges related to fostering creativity suggest that PBL interventions may need to be adapted or supplemented to address all facets of student development. The observed gender differences also call for tailored approaches that account for the unique ways in which male and female students interact with PBL, indicating a need for further research on how gender influences the effectiveness of such interventions.

6. Limitations

This study employed a pretest–posttest single-group design, which, while valuable for assessing the impact of the educational intervention, comes with certain limitations. One of the primary limitations is the absence of a control group. The use of a control group would allow for a clearer attribution of observed changes in student performance to the intervention itself rather than other external factors. However, in educational research, particularly when working with children, it is not always feasible or ethical to withhold potentially beneficial interventions from a control group. The decision to implement the intervention across the entire group of participants was made to ensure that all students had access to the educational benefits it provided, in line with ethical considerations that prioritize the well-being and development of children.

Another limitation is the lack of random sampling, which affects the generalizability of the findings. The participants in this study were not randomly selected, meaning that the results may not be representative of the broader population. This nonrandomized approach limits the extent to which the findings can be generalized beyond the specific group of students who participated in the intervention. However, despite this limitation, this study provides valuable insights into the potential impact of PBL on student competencies, specifically in math, language, and science, as well as 21st-century skills, offering a foundation for further research in more diverse settings.

While the calibration of the rubric was carried out using inter-rater reliability (IRR) to ensure consistency across different evaluators, the actual data collection during the assessment of 21st-century skills was completed by single observers for each group. This presents a potential limitation, as the use of only one observer introduces the possibility of subjective bias in the evaluation process. Despite the calibration efforts to standardize scoring, the absence of multiple raters during data collection limits the ability to measure IRR during the final assessments. Future studies could benefit from incorporating multiple observers to allow for IRR analysis in the data collection phase, thereby further enhancing the reliability of the evaluations.

Despite these limitations, this study provides valuable insights into the effects of educational intervention on student learning outcomes. However, the limitations related to the study design, sampling, and assessment methods should be considered when interpreting the results. Future studies should address these limitations to provide a more comprehensive and generalizable understanding of the impact of educational interventions, particularly through the inclusion of control groups, random sampling, and more robust assessment methodologies.

7. Conclusions and Recommendations

The results of this study demonstrate the effectiveness of the Project-Based Learning (PBL) intervention in improving student competencies in language, mathematics, and science, as well as enhancing essential 21st-century skills among third graders in rural Colombia. The intervention led to significant gains in standardized test scores and observable improvements in critical areas such as problem-solving, collaboration, communication, and critical thinking. These findings align with the growing body of literature that supports the use of PBL as a powerful instructional strategy that fosters both academic achievement and the development of key skills necessary for success in the 21st century.

The significant improvements observed in student performance, particularly the shift from lower to higher competency levels, indicate that the PBL approach effectively addressed the learning gaps exacerbated by the pandemic. The intervention not only helped students recover lost ground but also positioned them to excel in more advanced cognitive and practical tasks. The positive impact on 21st-century skills further underscores the value of integrating these competencies into the curriculum, ensuring that students are better equipped to navigate future challenges in education and beyond.

However, this study also revealed areas that require further attention. While most students showed improvement in various competencies, the decrease in creativity scores suggests that this area may require additional focus within PBL frameworks. Creativity is a critical component of 21st-century learning, and future iterations of the intervention should explore strategies to better support and enhance creative thinking among students. Additionally, the observed gender differences in overall standardized scores highlight the need for more tailored approaches that address the unique needs of male and female students within PBL environments.

Based on these findings, several recommendations can be made:

Firstly, enhance the focus on creativity. Considering the observed decline in creativity scores, it would be beneficial for future interventions to include more targeted activities and assessments aimed at fostering creative thinking. Creativity is a critical component of 21st-century skills, essential for innovation and problem-solving. To address this, the PBL intervention should be designed to include more activities specifically aimed at fostering creative thinking. For example, introducing open-ended projects where students are encouraged to explore various approaches and solutions can provide a platform for creative expression. These projects could be interdisciplinary, allowing students to combine knowledge from different subject areas to develop innovative outcomes. Additionally, it would be beneficial to integrate teaching strategies that explicitly promote divergent thinking—where students are asked to generate multiple ideas and solutions to a single problem. Classroom activities such as brainstorming sessions, mind mapping, and design challenges could be employed regularly to encourage creativity. Teachers could also provide more opportunities for students to reflect on their creative process, helping them to recognize and refine their creative thinking skills. Moreover, assessments that evaluate creativity should focus not only on the product but also on the creative process itself, rewarding originality, adaptability, and the ability to take risks. Encouraging a classroom environment where experimentation and unconventional ideas are valued will help cultivate a culture of creativity.

Secondly, tailor interventions to address gender differences. This study’s findings revealed significant gender differences in overall standardized scores, suggesting that male and female students may respond differently to PBL. To ensure that all students benefit equally from the intervention, future programs should consider gender-sensitive approaches. This could involve differentiated instruction that caters to the specific learning styles and needs of each gender, as well as providing gender-specific support strategies where necessary. Further research into the influence of gender on learning within PBL contexts is also recommended to better understand these dynamics and to inform the development of more inclusive educational practices.

Thirdly, expand the use of PBL in rural education. The success of this intervention in rural Colombia indicates that PBL is a viable and effective approach for improving educational outcomes in underserved areas. Given the challenges faced by students in rural regions, including limited access to resources and educational support, the expansion of PBL could provide significant benefits. Policymakers and educators should consider implementing PBL on a broader scale across rural schools, ensuring that students in these areas have access to high-quality, engaging, and relevant educational experiences. This expansion should be supported by ongoing training for teachers and administrators to effectively implement and sustain PBL practices.

Fourthly, incorporate more objective assessment methods. To strengthen the evaluation of 21st-century skills, it is recommended that future studies include more objective measures alongside observation rubrics. While rubrics provide valuable insights, their subjective nature can introduce bias, particularly when completed by a single observer. Incorporating performance-based assessments and technology-assisted tools and involving multiple observers can enhance the reliability and validity of the findings. Additionally, conducting inter-rater reliability (IRR) analyses would ensure consistency in the assessment process and contribute to the robustness of the results.

Finally, conduct longitudinal studies to better understand the long-term impact of PBL on student learning and development. While this study provides valuable insights into the immediate effects of the intervention, it is essential to track students over time to assess how PBL influences their academic trajectories, skill development, and future success. Longitudinal studies would provide a more comprehensive understanding of the sustained impact of PBL, allowing educators and policymakers to refine and adapt the approach to maximize its benefits for students in diverse educational contexts.