1. Introduction
Design-Based Research (DBR) has become an extremely popular and relevant methodology in educational contexts that engages in iterative designs to develop knowledge for practices of quality. DBR is praised for its predictive ability focused on designing relevant interventions that are capable of impacting the contexts where the research is being carried out. Being initially developed in the context of architecture and engineering, DBR was transposed into an educational context initially during the 1990s [
1,
2], with a stronger expansion in the beginning of the century [
3,
4], given its emphasis on breaching the gap between theory and practice (in context), so often criticized as a weakness in educational research. DBR looks at the nature of the education’s complexities, aiming for a deeper understanding of the learning-related educational processes (factors and effects), through a diversity of methods and systematic design, and the study of instructional strategies and tools [
4]. Usually, a DBR approach requires both quantitative and qualitative methods, through a combination of theory-based designs with empirical research, through continuous cycles of “design, enactment, analysis, and redesign” [
4] (p. 5), that can be documented [
3]. DBR’s data provide evidences to “understand how, when and why the emergence of educational intervention is in practice” (p. 5), leading to improved outcomes and promoting the implementation of innovations based on evidences, within a close collaboration between researchers, designers, and participants [
4].
Anderson and Shattuck [
5] praised the use of DBR in educational contexts, particularly in K-12 contexts, given its ability to actively engage participants in the field who are often only seen as research subjects. Furthermore, DBR is focused on addressing authentic problems and working based on contextualized interventions involving multiple iterations in order to lead to improved outcomes.
In fact, several studies have emphasized the ability of DBR to support teachers as designers of new learning environments [
6], and using “collaborative interventions to support student learning” [
7] (p. 1). Even more, we have seen the emergence of a scientific journal,
EDeR—Educational Design Research, gathering contributions of DBR studies in a wide range of education topics.
This growing use of DBR in education has called for the need of research efforts that synthesize how this innovative methodology is being used. Anderson and Shattuck [
5] looked at the five most cited articles between 2000 and 2010, trying to analyze how the growing interest in DBR was being translated into effective interventions. The authors concluded that even though most “interventions have resulted in improved outcomes or student attitudes (…) it is unclear if the results achieved are meeting the challenge of promoting widespread adoption of the tested intervention” (p. 24). Zheng [
8] carried out a systematic literature review of DBR studies from 2004 to 2013, in order to clarify how DBR was being utilized. This research looked into 162 published studies and described how most of them are focused on “designing, developing, and re-designing learning environments through interventions” (p. 399), but often fail to describe how the interventions have been revised.
Seven years later, given the continued growth of DBR use in education, we argue for the need to carry out an updated systematic literature review that builds on this previous work in order to analyze how the expanding use of DBR has been implemented in educational contexts. The initial step in the conduction process of the systematic literature, after defining the research goals, is to define the appropriate research questions that will guide the development of the study. Therefore, the research questions guiding this review are:
RQ1. What are the characteristics of the examined studies in terms of year of publication and publication journal?
RQ2. What are the purposes/main goals and theoretical framework reported in the analyzed articles?
RQ3. What are the educational context characteristics (learning domains, instructional methods, research setting) of the empirical studies using DBR?
RQ4. What are the research methodologies used in the studies according to their research design and settings, variables, sample, data sources, data collection, and analysis?
RQ5. What are the DBR approaches used in the studies in terms of DBR kind of interventions, interaction type and duration, measured outcomes, and limitations?
3. Results
The findings are organized based on the search for answers to the research questions set for this review and the data reduction carried out during the analysis process. Thus, we sought to understand in which educational contexts and for what purposes the DBR methodology has been used and what are the main contributions identified in the different studies.
3.1. Publication Characteristics of the Studies
Figure 3 details the number of research papers that used DBR published between 2013 and 2020, on the application of the inclusion and exclusion criteria. Most of the 163 papers were published in 2015 (
n = 25), 2019 (
n = 37), and 2020 (
n = 33). In the remaining years the number of published articles was quite similar.
The analyzed papers present empirical studies developed in an educational context, which were peer-reviewed and published in educational research journals (see
Figure 4). There are several journals with more than two papers published, particularly the
British Journal of Education Technology (
n = 6),
Information and Learning Sciences (
n = 6),
Mathematics Education Research Journal (
n = 4) and,
ZDM Mathematics Education (
n = 4).
3.2. Purpose and Main Goals of the Studies
Considering the Purpose/Main Goal of the analyzed articles, three categories emerged as the most relevant ones (see
Table 1). The category related with the development of Information and Communication Technologies (ICT) competencies, with a total of 55 references, was by far the most representative in our sample. Within this category we find a wide set of studies dealing with such issues as virtual reality environments, mobile learning, educational robotics, online teaching, and digital storytelling.
The focus on curriculum, pedagogy, and assessment interventions also emerged as one of the most cited purposes (n = 34), with topics such as active learning, game-based learning, project-based learning, and pedagogical innovations. In addition, with a large number of references (n = 25), we have the focus on Teacher Education interventions, ranging from pre-service teaching to in-service professional development models, and including specific experiences such as teacher assessment and peer tutoring.
After these three topics, three other content-based purposes emerged: science education (n = 16), mathematics education (n = 7), and language education (n = 7), reflecting the three main scientific areas where DBR has been more used over the last 10 years. Finally, the topics of Inclusion (n = 6), referring to school-based studies supporting the integration and participation of all students and in multicultural contexts, and Partnerships (n = 3) between schools, families, and local communities, emerged as less representative topics. Two studies dealing with neuromyths in education and research innovations were categorized as Other.
3.3. Theoretical Framework
The analysis of the articles allowed us to identify six main topics regarding the theoretical framework of all the research. Most of the theoretical frameworks are grounded in the specific theories of learning of the main subject in which the research was conducted, explaining why 74 of all the articles were contextualized in Mathematics, Algebra, or ICT theories, involving virtual reality environments, mobile learning, etc. This result seems aligned with one of the topics that emerged in the previous category (of purpose), which also identified these three main scientific areas. There was also special attention given to theories of Professional Development (n = 31), Curriculum/pedagogy and Assessment (n = 28), and Conceptual Learning/Development Models (n = 21), such as constructivism, cognitive flexibility, critical thinking, among others. Another two topics also found, but less representative, are Partnerships/Collaborative work (n = 8) between different participants in the educational process (parents, school, and local community), and Scaffolding (n = 4).
3.4. DBR Theoretical Framework
Related to the main authors cited in the analyzed articles to support the definition of the methodological use of DBR, the analysis allowed the identification of a large diversity of DBR approaches proposed by several authors. The most cited work was ‘Design-based Research Collective’ (2003) (n = 23), a paper published by a group of researchers from different universities which presented DBR as an emerging paradigm for educational inquiry, perhaps the first paper discussing the potentialities of DBR to support the research processes in the educational field. The analysis highlighted other work as being the most cited—the paper written and published by Terry Anderson and Julie Shattuck (2012), about the progress of the DBR in educational research between 2002 and 2012; the paper written by Sasha A. Barab and Kurt Squire (2004) where the authors try to establish points of agreement on what constitutes DBR and methods to accomplish it; and the book written by Susan McKenney and Thomas C. Reeves (2012). The importance of the work of these researchers on the use of DBR methodology in education research was evident, having been mentioned in most of the 163 articles analyzed.
3.5. Research Context
To define the context of the design-based research application, in this work, a set of dimensions of analysis was used related with the learning domains, research context and settings, interaction type and frequency, type of data source, main results and outcome, and methodological limitations.
Most of the empirical studies analyzed were carried out in the learning domains of natural science (
n = 61), mixed learning (
n = 37), and engineering and technological science (
n = 34) (see
Figure 5). The learning domains include various subjects presented in the K-12 curricular grades and in the teacher training programs. In addition, the most frequent instructional methods or strategy types reported were the use of integrated teaching methods (
n = 58), authentic tasks (
n = 28), and curriculum unit (
n = 25) (see
Figure 6).
Focusing on the intervention typology, it is possible to identify a variety of approaches in DBR studies that characterize the research activities undertaken with participants. Accordingly, a diversity of interventions is represented in the
Figure 7, such as instructional methods (
n = 46), technological intervention (
n = 45), other models or methods (
n = 33), and integrating teaching methods (
n = 21).
The sample groups reported in the different studies consist mostly of junior and senior high school students (
n = 53), teachers (
n = 38), and mixed participants, including teachers and students (
n = 25) (see
Figure 8). Research activities were conducted in face-to-face classroom (n = 107), mixed research (
n = 19), blended learning (
n = 19), and distance learning (
n = 10) settings (see
Figure 9).
Studies developed on DBR processes typically use mixed approaches towards data collection and analysis, and a diversity of information sources. This review identified that 49% (
n = 80) of the studies used qualitative methods, 46% (
n = 75) qualitative and quantitative (mixed) methods, and 5% (
n = 8) quantitative methods (mostly parts of a wide study) (see
Figure 10). Following the characteristics of DBR studies, most articles (
n = 130) report the use of a diversity of data collection instruments such as questionnaires, interviews, written notes, and observation records, among others (see
Figure 11).
One of the fundamentals of the DBR methodology is its cyclical and iterative nature organized into different timed phases during the conduct of the research process. In each of these phases, the research team should analyze the data and produce knowledge applicable to the following phases. According to
Figure 12, only 60% (
n = 98) of the papers report empirical studies organized in interaction cycles between twice (
n = 32) and more than five times (
n = 23). However, 40% of the articles (
n = 65) do not indicate the number of cycles or iterations, partly since they report only a part of the study developed or under development. Accordingly, 37% do not specify how each cycle was revised (
n = 61), 31% state that there was a revision but do not detail it (
n = 51), and only 25% provide some conclusions on the revision of each cycle (
n = 40) (see
Figure 13).
Most of the research activities reported in the papers are less than 6 months (
n = 44) and between 6 months and 1 year (
n = 28), and about 52 studies do not report the time-schedule (see
Figure 14).
The analysis of the measured outcomes and main results of each research paper are detailed in
Figure 15 and
Figure 16. The measured outcomes are majority integrated (
n = 62) and cognitive outcomes (
n = 38). According to [
5], three major categories are selected as outcomes, cognitive outcomes, attitude, and psychomotor skills. When studies measure multiple types of variables, they are included in the “Integrated” category, and “Others” was considered when studies measured outcomes that did not belong to the previous ones. To analyze the main results pointed out in the studies, we used the seven groups defined by [
5]. Consequently, the reading of the full papers allowed us to identify that 73 of the studies conducted interventions with potential for improved learning, 24 of the interventions have increased the participants’ learning outcomes, 22 improved the attitude and motivation of the participants, and 22 of the studies allowed new understandings about educational phenomena.
Another focus of analysis was methodological limitations and implications to practitioners carried out by the author of each study. It should be noted that most authors have not analyzed the methodological limitations (
n = 103) of their studies or made recommendations (
n = 148) for the use of DBR (see
Figure 17 and
Figure 18). However, it was possible to identify as methodological limitations the specificity of the context (
n = 30), the sample size (
n = 14), the generalizability of the results (
n = 10), and the duration of the study (
n = 5).
Although most of the studies did not mention implications for the practice of using DBR, a small set and articles signaled some recommendations for the organization of cycles for analysis and monitoring methods.
4. Discussion
An education of quality is one of the main topics in international agenda. It demands a change in school organization, curricula, and teaching–learning processes. Systemic perspectives of intervention in schools have been gaining strength, involving all educational contexts, intervenient and outcomes. Our goal is to favor change in educational thinking/theory as well as in practices, with significant gains in students’ behavior and learning. Focused on educational inclusion as the goal, and based on the premise that DBR is a methodology that is becoming more popular in education, especially in K-12 contexts, due to its collaborative work and continuous cycles of iterations, there is an emergent need to update the evidence in this area and level of schooling. The data collection about DBR will allow one to understand the specificities of the educational system and contexts, as well as the processes that are affecting academic success.
The review conducted in this study presented the state of art of DBR use in an educational context since 2013, based on the corpus of 163 papers retrieved from SCOPUS and WoS Databases. Most of the selected papers were published in 2015, 2019, and 2020 in different peer-reviewed indexed journals, in the field of Educational Technologies, Sciences, and Mathematics, namely the
British Journal of Educational Technology,
Information and Learning Sciences,
Mathematics,
Education Research Journal, and
ZDM—Mathematics Education. In fact, the choice of these journals is natural considering that most of the analyzed papers had the purpose of developing research in the areas of ICT, Curriculum, Pedagogy and Assessment, Teacher Education, and Science Education. The Science Education domain was already reported by Zheng [
8] as an important research context where DBR has been used. Although we identified ICT in education as the most important category in the purposes and objectives of the studies, about 50% were developed in science-related learning domains (e.g., natural science, mathematics, physics, chemistry, biology, and environmental science), which is aligned with other studies [
8]. The most frequently reported instructional methods or strategy types were the use of integrated teaching methods, authentic tasks, and curriculum units, which confirms the potential of the DBR methodology to support experiments that draw on diversified teaching techniques, design, and the implementation of authentic tasks with a high practical applicability. It was noted that there was no significant increase in the number of articles published between 2013 and 2020 compared to the systematic review conducted by Zheng [
8]. The author selected 162 papers and, in this study, we analyzed 163. However, more recent studies pointed out that DBR is attracting the attention of researchers around the world, particularly in the last five years [
13].
The analysis of the theoretical framework of each study was organized into two points of view related to the theoretical discussion of the core topics of the research and the DBR approaches and methods. The theoretical frameworks of the analyzed studies are aligned with some specific theories of learning in different scientific areas (e.g., mathematics, ICT, Professional Development, Curriculum, and Pedagogy and Assessment). To support the use of the DBR methodology, most of the studies used reference authors who have been advocating the potential of this approach in educational research processes. The analysis identified important authors who have seen their work in this field widely recognized and who are also highlighted by [
13] in a bibliometric analysis of the use of RBD in science research.
Focusing on the central aspects of DBR research, related to the frequency and duration of the different iterative cycles, and how the analysis of each cycle contributes to the following one, our analysis revealed that, in several of the analyzed studies, the authors did not report information on the duration and number of cycles, nor on how the results of each cycle influence the following cycles, corroborating [
8]. In the articles that report this information, most of the studies were organized in two or three cycles of a short duration, which seems to be aligned with [
5], but differ from Zheng’s systematic review [
8], that pointed out one cycle with an iteration for a year, probably due to the economic and time effort needed.
Connecting the main results and the way they were measured and analyzed, most of the studies presented results that allowed them to potentiate and improve learning, to improve the attitude and motivation of the participants [
5], and to develop new knowledge in the educational phenomenon. Therefore, considering the nature of the DBR studies, most of the authors used qualitative and quantitative data analysis methods which allowed for an integrated analysis of the results. Although Zheng [
8] reported a tendency towards the qualitative data’s collecting method, due to its descriptive and explanatory nature [
14], the mix methods have been pointed out by others studies [
15]. In the process of data collection and analysis, and a variety of instruments such as questionnaires, interviews, field notes, and others were used. This integrated nature of looking at and analyzing the data was also referred to in previous studies as one of the DBR’s potentialities, which enables the use of this information in processes of the improvement and refinement of each research cycle [
5,
8].
The analysis showed that the impact of technology [
5], and/or the need to deepen the knowledge related to it, is almost unanimous in all the studies, corroborating previous evidences [
5,
8]. In different teaching cycles and environments, from the practical application in pre-school education contexts to the study cycle referring to teacher training, its impact is closely related to the promotion of learning and the improvement of students’ performance. In the analysis of 163 articles, there are at least 75 unequivocal references that support this analysis. Campbell [
16] highlights the potentiality of
WhatsApp as a free application that decreases the distance between students and teachers by promoting moments of distance tutoring. Alves and Hostins [
17], meanwhile, emphasize the impact of games in the educational context, and Chin and Tsuei [
18] further add that a digital game-based learning environment promotes the learning of hospitalized children. Whether related to apps, gadgets, games, virtual environments, software, among others, there is a strong trend towards the use of DBR methodology within these research environments.
Additionally, research involving processes characteristic of DBR are used and are revealed to provide significant contributions from the point of view of teacher professional development. These contributions relate to the development of innovative and interactive teaching strategies in mathematics [
19]. Jackson-Barrett et al. [
20] carried out research that uses the principles of DBR in the construction of digital stories in continuing teacher education, considering the fundamental learning environment that this methodology provides. In addition to the examples mentioned, there are at least 47 references in which the authors consider, in some way, that the principles of DBR contribute positively to the advancement, improvement, or development of studies related to teacher professional development.
Within the sample analyzed, the contributions of the DBR methodology as an element that promotes or facilitates students’ learning or competence development are also evident. The DBR methodology provided a systematic framework that enabled the creation of learning environments where students were motivated and interested in history teaching [
21]. Furthermore, design-based research has helped inform about the development of learning guidelines in relation to elementary mathematical writing [
22]. Nevertheless, it is curious to highlight that, in the past [
8], the face-to-face classroom tended to decrease while distant learning settings increased, which was not the case in the studies analyzed. A particular attention should be given to these variables due to COVID-19 experiences.
In addition to the cases that stand out, exemplified above, there is also a set of references that allow one to infer the importance of the DBR methodology. Although brief, it should be noted that the studies analyzed referred to the positive impact of the methodology as it contributes to (a) the relationship with the context or to establishing relationships between the participants in the research, and (b) promote the constant articulation between theory and practice. In addition to these aspects, it is considered that the cycles inherent in the DBR methodology facilitate the understanding that it is necessary to invest in the knowledge and development of the projects studied.
Thus, it is considered that the DBR methodology, or at least some of its principles, can effectively contribute to the enrichment of investigations and investigative practices, and, consequently, to the improvement of knowledge, particularly in Education.
5. Conclusions
This study allowed us to explore how DBR has been used as an established methodological approach in educational research. Focusing on K-12 and teacher education contexts, we have found a total of 162 studies published in Scopus/WoS journals, illustrating the recognition of this methodological approach to bridge the gap between theory and practice, even with the limitation of only including open and in full access articles. Furthermore, and similarly to what was found by other authors [
5,
8], the potential of DBR to foster improved learning for students emerges as one of the most prevalent outcomes of these studies, across a wide variety of learning domains and research settings. Even though there is a clear predominance for using DBR in science-related learning domains, probably due to a longer tradition of more often conducting experimental, or quasi-experimental, research methods, it should also be noted that a wider range of contexts is making use of DBR methods. The potential to promote and facilitate innovations and transformative interventions is often recognized in the sampled studies as one of the main advantages prompting the use of DBR. This is further emphasized by the potential of DBR to facilitate integrated and authentic teaching/learning interventions that are very frequently reported, illustrating its potential to support context-sensitive interventions addressing issues recognized as relevant by the local participants. Technological intervention seems to be the most referenced topic in the DBR studies analyzed, within curriculum/pedagogy and assessment practices context and goals, particularly in natural, social, and technological science. Professional development is a major issue that seems to affect the academic success of students in an integrated perspective, involving cognitive, psychomotor, and attitude skills. All studies comprised teachers and students, as expected, mainly in face-to-face classroom processes. Due to the recent COVID-19 experiences, it particular attention should be given to blended or mixed learning in the future. The instructional methods, alongside technological interventions and integrated teaching methods, seem to be the most mentioned types of intervention methodologies. The qualitative and quantitative approaches emerge as the main methodologies, although a diverse range of procedures for data collection is identified. The theoretical framework tends to be grounded in the specific learning theories of the main subject. Inclusion and partnerships are also two topics addressed, although less frequently. Nevertheless, it should also be noted that most of the included studies do not give a thorough report on their DBR design. Even though supporting references to DBR pioneers are frequent [
4,
5,
14,
22,
23], detailed explanations on DBR design, such as the type and duration of interventions, data collection methods, and end of cycle reports, are often missing. It is also common to find studies reporting only one of the intervention cycles and missing a larger scope of the bigger research design being developed. Many studies also do not report on the obstacles and limitations they have faced, contrary to what is recommended in DBR design [
15], or contribute with recommendations for the improvement of the DBR methodology employed. This may be due to the type of publication selected for this review, as often journal articles do not allow for very detailed explanations given their strict word limits. This must be seen as a problematic issue since the lack of design detail given to the reader, makes it harder to replicate and/or adapt the proposed interventions.
So, one of the recommendations for future research in the DBR field is the need of a more extensive and descriptive approach for the presentation of the design of the intervention planned and implemented, including listing each cycle/iteration and its full documentation (time, commitment, contingencies, activities, data collection, and analysis). The discussion about how results can be emphasized and generalized (since the majority is local), and its impact, is another suggestion for future studies that should be supported by valid and reliable methods and evidences, allowing the replication of research at a larger-scale.
Finally, even though most analyzed studies were carried out in traditional face-to-face environments, the number of studies in hybrid and fully online contexts seems to be growing. Furthermore, given the worldwide pandemic context of the last 2 years, we expect to see a surging number of studies in these contexts, making it a possible research opportunity for the coming years. Even more, given our focus here in K-12 and Teacher Education, we have not included Higher Education contexts in this review, however, this also seems to be an emerging environment where DBR use has been growing and a systematic review can be justified.