Next Article in Journal
Perspectives of Distance Learning Students on How to Transform Their Computing Curriculum: “Is There Anything to Be Decolonised?
Next Article in Special Issue
From Research to Retweets—Exploring the Role of Educational Twitter (X) Communities in Promoting Science Communication and Evidence-Based Teaching
Previous Article in Journal
Development and Evaluation of a Custom GPT for the Assessment of Students’ Designs in a Typography Course
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Research Impact and Sustainability in Education: A Conceptual Literature Review

Research Centre on Didactics and Technology in the Education of Trainers, Department of Education and Psychology, University of Aveiro, 3810-193 Aveiro, Portugal
*
Author to whom correspondence should be addressed.
Educ. Sci. 2024, 14(2), 147; https://doi.org/10.3390/educsci14020147
Submission received: 23 November 2023 / Revised: 18 January 2024 / Accepted: 29 January 2024 / Published: 31 January 2024

Abstract

:
Research sustainability and research impact are closely related concepts, both referring to the importance of extending research projects’ outcomes, changes or benefits beyond their period of implementation. However, the prominence assigned to these two concepts has been provided in an individualized way. This study aims to understand how research sustainability and impact are portrayed and problematized in the literature, in the field of Education, and how they (inter)connect, identifying reference authors and frameworks and describing factors that influence research sustainability and its impact in Education. A conceptual literature review was conducted, and 31 articles were analyzed under the scope of an interpretative paradigm according to the content analysis methodology, using webQDA 3.0 software as support. First, categories were established to identify reference authors and frameworks. Afterward, an inductive codification was carried out, and 15 categories emerged to research sustainability and its impact on Education. This analysis highlights the interconnectedness of both concepts at the conceptualization level and the factors that influence them. The results allow us to identify and discuss action axes, that need more attention in order to assure research sustainability and to evidence research impact in the field of Education.

1. Introduction

Research sustainability and impact are at the epicenter of researchers’ and policy makers’ discussions and agendas and reverberate in various scientific areas and contexts [1,2]. Governments, research funding agencies and researchers themselves are increasingly aware of both their responsibility to ensure sustainability [3] and to evaluate the impact of funded academic research [4]. This concern is reflected, for instance, in the evaluation criteria of research funding programs from several transnational and national agencies, such as the European Research Council from the United Kingdom [5] and the Australian Research Council from Australia [4].
Despite the increasingly acknowledged importance of research sustainability and its impact [4,6], the literature on research projects’ sustainability dates from 2007 to 2008 and onwards [3] and, arguably, only in such a period of time has the challenge of evidencing socioeconomic impact gained particular attention [7]. Likewise, in the area of Social and Human Sciences, the attention given to these concepts has increased. In the Education field, specifically, debates and discussions have focused on some (older and newer) issues such as the difficulty to evidence research impact; the discussion around the means to extend research beyond projects’ implementation period [8]; and educational research projects’ discontinued funding [1]. These debates show a certain (con)fusion between the boundaries of sustainability and impact, which highlights a symbiotic interaction. In spite of this, the prominence assigned to these two concepts in the literature has been individualized, and the interconnectedness between them has been seldom addressed.
Considering this framework, this study aimed to answer the following question: how are research sustainability and impact described and problematized in the field of Education, and in what ways do they (inter)relate? More specifically, this study intended to identify reference authors and studies in the topics of research sustainability and impact, specifically in the area of Education; discuss in what way(s) research sustainability and impact in Education (inter)relate; and describe the factors influencing research sustainability and impact in Education.

2. Materials and Methods

In order to answer this research question and to fulfill the associated research objectives, a conceptual literature review was conducted. In this type of review, the authors focus on concepts and the relationship between them, including relevant theories and empirical research [9]. This review settled on the basic principles of a systematic review in the sense that a guiding question was outlined to drive the process, the methods were clearly described in order to facilitate the replicability of the procedure by others, and specific inclusion criteria were established to define the corpus of analysis [10,11]. However, it was not completely exhaustive, given that the search process in the literature did not reach a saturation point, which means not all knowledge in the area was gathered [12,13]. Based on that and the study goals, we decided to carry out a conceptual literature review, which is focused on the categorization and description of two relevant concepts (research impact and sustainability) and on the establishment of relationships between them [9].

2.1. Corpus Selection Process

Acknowledging the methodological rigor implicit in the literature review process in general [12], it is essential to describe in detail the steps that led to the final corpus. First, it is important to highlight that, due to the first research objective (to identify reference authors and studies), a time span was not established since authors and studies of reference are timeless. Also, given that, as stated before, the literature on research sustainability and on evidencing (socioeconomic) research impact dates back essentially to 2007–2008 [3,7], defining a time interval was not considered to be a relevant criterion. Instead, there could even be limitations due to the difficulties in finding relevant literature for research sustainability, so both searches carried out did not include a time limit.
In addition, we must also clarify that although the focus of this article is the interplay between the concepts of sustainability and research impact, the searches described were conducted individually for each concept. This is because, as previously mentioned, the concepts are addressed in the literature separately, so when searching for both in conjunction, the results returned by the databases were null or insufficient to conduct a literature review.
Given this, in order to identify articles related to the “research impact” in Education, the ERIC database was chosen due to its special focus on this scientific area. In this first search, the keywords of “research impact” and “higher education” were used only in English (which is the database language), between quotation marks, and combined with the Boolean operator “AND”, producing a total of 335 correspondences. It is important to point out that the singular use of the first keyword expression returned 1634 results, making this search too comprehensive, so it became inevitable to group it with the second one—which is relevant because it covers the context where research happens, in general. Additionally, it is important to explain that the “education” keyword was not used because of the focus on the educational area of the ERIC database.
Next, and in order to reduce the sample, the inclusion criteria specified in Table 1 were created and applied to the 335 results.
The first criterion, related to textual typology, intended to exclude long documents, such as books, book chapters, theses, dissertations, or technical reports of the final corpus. The focus of this review was on scientific articles as part of academic journals. Regarding the second criterion, it was established that these articles should be open-access so the authors could have full access to them. On top of that, and in order to ensure the scientific validity of the selected articles, it was stipulated that they should be submitted to a peer review process. Finally, to make the study feasible, it was also established that the articles should be written in one of the languages in which the authors have reading proficiency. Alongside these criteria, “journal articles” and “peer reviewed only” ERIC filters were used, and 229 results were obtained.
Then, titles, keywords and abstracts were read, and those that did not meet the suitability criterion were excluded. This was created in order to lead the final corpus to meet the research question and the research goals. Therefore, the articles were selected according to their existence in the titles, keywords or abstracts of one or both of the following premises: discuss the concepts of research sustainability or research impact in Education and/or address the factors influencing research sustainability and research impact in Education. After excluding the articles that did not fit the study, a sample of 19 articles remained as part of the final corpus.
Out of them, one was not in open access, so 18 were included in the analysis corpus (see Appendix A). To summarize and clarify the overall selection process, Figure 1 provides a flowchart of it.
Concerning the “research sustainability” search, it was also intended to limit it to the ERIC database due to the previously explained reasons. However, because of the difficulty of finding relevant literature in this area, it was also necessary to resort to other databases, namely Scielo, Redalyc, Google Scholar, ISI Web of Knowledge, SCOPUS, Emerald and Science Direct. To widen the search even further, English was not the only language used in the search process, as was the case for “research impact”, as Portuguese and Spanish were also applied, according to each database’s context and scope (e.g., Scielo and Redalyc are databases that include work on Portuguese and Spanish, respectively, and Google Scholar is multilingual). Table 2 summarizes the search codes, the obtained results and the number of articles that were included in the corpus.
The “research sustainability” expression was not used in the above-listed searches because the results obtained centered in a reversal of the word order—“sustainability research”—with the articles focusing on sustainability studies in the most diverse scientific areas (such as agriculture, climate change or sustainability practices within Higher Education Schools). The specificities regarding applied filters in some databases are explained in the footnotes and linked to the respective search code on the table. The titles, abstracts and keywords of the 860 results identified in the previous table were read, and after suitability criteria evaluation and duplicate exclusion, 13 were eligible as part of the corpus (see Appendix B).
Comprising 18 articles related to research impact and 13 linked to research sustainability, the analysis corpus is composed of 31 documents (see Appendix A and Appendix B).

2.2. Data Analysis Procedures

To maintain methodological consistency, it was stipulated that data analysis would be carried out under the perspective of an interpretative paradigm [14,15] and according to the guidelines of content analysis [16,17], using webQDA 3.0 software as support. These decisions were made taking into account both the comprehensive component intrinsic to the interpretative paradigm and the specific characteristics of content analysis. Such a data analysis technique settles on the desire for rigor (also implicit in the previously described literature review process) and on the need to discover (present in the study goals, specifically in the search for interrelations between the concepts of research sustainability and impact) [16].
This need to discover is particularly important in our context, given that the relationships between research sustainability and research impact are an underexplored theme in the Education field. Taking that into account, there is not a pressing need to verify the existing theory. Instead, what is necessary is the amplification of the investigative spectrum, focusing resources and efforts on building new theories interconnecting these two concepts. Hence, the focus of this study relies on the process of constructing theory under a perspective of constant development and update, not intending to provide a finished product.
Regarding the analysis method in itself, the content analysis technique was structured in the following four main stages: analysis organization, coding, categorization, and inference [16,18]. Thus, after the selection of the corpus and the exploratory analysis of the articles (by reading their titles, keywords and abstracts), codes were created in different ways according to the specificities of each of the research goals.
To fulfill the first one and identify reference authors and studies in research sustainability and impact areas, the codes “reference authors” and “reference studies” were created a priori. For the codification and accounting process, both the authors of the corpus and the authors cited in those articles were considered (when and every time they were associated with research sustainability or research impact statements).
In order to identify the factors that may influence and, therefore, enhance research sustainability and impact (individually or together), the inductive coding of the articles was conducted. Hence, articles were read and interpreted in order to develop analysis codes. The initial process allowed the emergence of 22 codes, which, after saturation and relationship exploration, were grouped into 15 categories, as illustrated in the next section.
Both these coding processes resulted in a list with the most representative authors and studies on research impact and sustainability in Education and a table with the most important factors described in the literature. On top of that, such processes allowed a deeper understanding to be inferred of the way in which these two concepts are portrayed in the literature and how they (inter)relate.

3. Results and Discussion

In the following section, the results are presented and discussed for research sustainability; research impact; and the (inter)relation between research sustainability and research impact. The first two subsections address the research goals: (i) identify the reference authors and studies in the topics of research sustainability and impact in the area of Education and (ii) describe factors that influence research sustainability and impact in Education. The last subsection focuses on the way(s) research sustainability and impact in Education (inter)relate, either conceptually or in terms of influencing factors.

3.1. Research Sustainability

As far as research sustainability is concerned, it was possible to verify some unanimity in the acceptance of this concept as the extension of the benefits resulting from a project (whether funded or not) beyond its implementation period (A20, A22, A25, A27, A28, A30, A31). For this general conformity of opinions, the work and definitions provided by the 71 reference authors were contributions, from which we highlight Cecília Guerra, Lee Bone, Mary Ann Scheirer, Mona Shediac-Rizkallah and Robert Savaya as significant authors due to them having the highest number of codifications (four for each of them). In addition, 33 studies were listed as able to constitute relevant bibliography in the area.
Specifically in the area of Education, research sustainability assigns a greater focus to the research process as a whole instead of a mere emphasis on results (A21). With this statement, we mean that all the outcomes (even the non-objective ones) that result from the project during its lifetime (such as the impact on participants, on the form of learning, personal development and created networks) should be considered, and valued, for instance, by being included in the project’s final assessment. In a society eager for visible and/or measurable outcomes, where objectivism and economics take a prominent place [19], it is important to demystify that any research must necessarily produce them. Under the perspective of “not everything that can be counted counts, and not everything that counts can be counted” [20] (p. 43), sustainable research does not need to offer an endless number of tangible outcomes and benefits. Sustainability is also perceived in the positive influences of (educational) research in those who took part or had contact with the study.
Additionally, the importance of leveraging research on previous studies through a strong theoretical foundation (A22) should be pointed out. On top of that, the significance of creating extension possibilities is also worth highlighting, namely regarding knowledge transfer to other scenarios and areas of concern. These possibilities should ideally be based on educational and pedagogical innovations deriving from research projects (A31) or from new initiatives emerging from them (A21).
Considering all of this, research sustainability in Education is a concept that must be looked upon in a wide and open way, not focusing only on a singular person or a given research project. In this sense, the articulation of efforts, the establishment of relationships between the project actors, the involved community and institutions of interest, as well as the dissemination of achievements to other contexts, are of particular relevance.
This “peer support”, in the form of networking with people external to the project and the establishment of formal or informal partnerships, is, indeed, one of the influencing factors of research sustainability portrayed in the literature (A20, A22, A28). This is directly related to the “feeling of belonging” that could be triggered by the inclusion of a non-academic community both in projects’ planning and projects’ development (A20, A25, A27). Apart from allowing a greater focus on existing community problems, the self-motivation and emotional factors that could be activated by this involvement may concur with greater community involvement (A20, A22).
In addition to this, but also in relation to projects’ planning stages, the literature also underlines the importance of including a “program champion” (A28) and a person with specific “sustainability competences” (A26) in the original research team. A program champion is a person who, due to a prominent and privileged position, for example, in research host contexts, may have bigger negotiating and authority capacities to establish the necessary commitments to the achievement of project goals. This may seem unnecessary and, to some extent, obvious, but if a project’s goals are not even fulfilled it will hardly have any sustainability. In turn, a sustainability expert can both promote sustainable practices at every stage of the project and develop sustainability skills in the members of the research team during the project’s lifetime (A26).

3.2. Research Impact

With regard to research impact, despite the discussion around the concept (A11), it is formally defined as any effect on, change or benefit to the economy, society, culture, public policy or services, health, environment or quality of life, beyond academia [21]. To put it simply, it can be seen as the transformation of academic work into knowledge that is useful for society in a holistic way (A7, A8). In the case of research impact, 27 reference authors were singled out for their contribution to the above definitions, from which Alis Oancea, Andrew Gunn, Jonathan Grant, Michael Mintrom and Richard Watermeyer are highlighted (according to the highest number of codifications). Cumulatively, 14 reference studies were identified as relevant bibliographies in the area.
As one can see, mainly since the release of the Research Excellence Framework (REF) by the Higher Education Funding Council for England in 2014, the emphasis has been placed on non-academic contexts (A5, A9, A10, A12), which reflects a change in the paradigm and the evolution of the concept, given that, for years, a more traditional vision persisted (and still does, in some contexts), based on metrics—like the number of publications, citations or article reads. Currently, the literature tackles the impact of public engagement and public knowledge of scientific and academic achievements.
Although REF’s definition of impact is globally the most cited, there is a widespread vision in research in Education (A10, A18) that it needs to be reconceptualized according to the contexts to which it is applied given its generality. Taking this into account, the literature highlights that for academic work to be translated into knowledge useful for society in a successful manner, it is also necessary that bigger importance is given to the research process, to the detriment of the emphasis assigned to results (A14, A15). Due to the difficulty of the immediate measurement of these latter concepts in an educational context, this transformation of academic work into everyday knowledge must happen along the research projects’ lifespan. This could happen, for example, through “science communication” or “knowledge brokerage” initiatives.
Such activities of “academic work translation”, which are signaled as influencing factors regarding research impact, are particularly important. Science communication can be seen as “the process of translating complex science into language and concepts that are engaging and understandable to non-scientific audiences” [22] (p. 182). Knowledge brokerage has a similar direction. It happens through a “knowledge broker”, which is an agent who supports interaction and engagement between the academic and non-academic community with the purpose of encouraging knowledge exchange, emphasizing research utility and reinforcing research impact (A18). Both initiatives focus on making scientific knowledge available and accessible to the public in general and to their target audience, in particular (in our case, the pedagogical community). In their absence, people outside academic contexts have bigger difficulties accessing research outcomes and results.
Such difficulties are, on the one hand, due to the high monetary value of the periodicals where these outcomes and results are generally published, which hinders access to them. On the other hand, when educational actors reach these data, they are usually written in academic jargon, which is adapted to scientific journals’ peer-reviewed process and does not allow an easy adaption to practice (A16). Including these knowledge exchange and science communication strategies officially in research design could make research teams more focused on these strategies, attaching to them an important role along the project rather than taking place sporadically and arbitrarily (A2, A3). In order to facilitate this, the selection of “research methodologies” that are articulated and linked with science dissemination activities is a solution (and a research impact influencing factor) suggested by the literature (A5, A12).
In addition to this topic, the development of these activities ends up resulting in information for educational policies (A1), as well as social welfare, from a mostly socio-economic perspective (A7). In its turn, social welfare manifests itself through resources and/or innovative pedagogical practices and could probably foster higher-quality education.
There are, however, some more factors also related to academic translation activities that can negatively influence their development. Since many researchers accumulate diverse functions in their working contexts, “overlapping positions and lack of time” (A3, A4, A14) is a problem that leads them to, most of the time, not being able to promote dynamics that, in most cases, are secondary compared to their main activity and for which they have a “lack of training” (A14). Thus, they end up not possessing the necessary skills and expertise to carry out knowledge exchange activities.
Finally, it is important to argue, once again, against the immediacy of contemporary society. Mitchell (A11) underlines that “real impacts” are related to long-term effects, changes or benefits and not to those that only last for some time. This positioning is corroborated by Doyle (A10), who claims that although some research may generate immediate impact, others require years or even decades until the value of their produced knowledge becomes evident.

3.3. The (Inter)relation between Research Sustainability and Research Impact

In this subsection, we discuss in what way(s) research sustainability and impact in Education (inter)relate, either conceptually or in terms of influencing factors. Starting with a comparison of the numbers related to the reference authors and studies reported in the previous two subsections, we can see an asymmetry in these codifications’ absolute frequencies; for research sustainability, 71 reference authors and 33 relevant studies were identified in the coding process. Research impact, in its turn, had smaller numbers, with 27 reference authors and 14 relevant studies coded. So, out of a total of 98 authors and 47 studies, 72% of the authors (71 in 98) and 70% of the studies (33 in 47) are related to research sustainability. These data become even more interesting when we acknowledge that the corpus of articles related to this concept (research sustainability) that went to analysis (after being identified in the literature review selection process) had a smaller dimension when compared to that of the research impact (out of a total of 31 articles, 13 were related to research sustainability, and 18 were related to research impact).
Concerning research sustainability, these bigger numbers could mean a broader and more generalized concern with the subject across the globe, implicitly meaning that it is not a concern limited to specific countries and/or researchers. Despite this, these (sparse and dispersed) results may also indicate that there are still not many studies and people carrying out in-depth research on research sustainability.
Regarding research impact, a smaller number of total codifications can indicate a concentration of the works on the same countries and researchers, which evidences the contexts and the people that are more concerned about the (socioeconomic) impact. For example, Richard Watermeyer, the most cited author, is linked to research impact and was counted 20 times (either as an author of the articles in the corpus or as cited in those articles), while the authors identified before as more connected to sustainability (Cecília Guerra, Lee Bone, Mary Ann Scheirer, Mona Shediac-Rizkallah and Robert Savaya) had only four codifications each, as stated previously.
Focusing the attention on how research sustainability and impact are conceptually described and problematized in the field of Education, we conclude that these two concepts (inter)relate and overlap at many points, starting with their definitions, as we have seen in the two subsections above, both pointing to the importance of the research process as a whole, rather than the excessive valuation of results. Moreover, research sustainability and research impact emphasize the relevance of translating academic work into knowledge that is useful for society and the non-measurable impact on participants (such as learnings, personal development and/or created networks) (A14, A15, A21).
Linked to this is the issue of immediacy and the need to deconstruct the idea that research needs to produce an immediate impact since both concepts point to long-lasting effects, changes and/or benefits that should exceed the time limits of the projects’ period of action (A10, A11, A21).
The factors that influence research impact and research sustainability also coincide to a certain extent. As stated in the methodological section, these influencing factors were identified in this article through a coding process, which resulted in 22 codes later grouped into 15 categories. Some of these categories (or factors) were already addressed in the two previous subsections, together with the concept to which they are most closely related. However, to provide a general overview of all of them and to introduce the ones that have a direct connection with both concepts, we present Table 3. In it, factors are organized into the following phases—“pre-project”, “during project” and “post-project”. However, this organization is not exclusive or stationary, meaning that a factor that was included in the research project’s planning or developing phase may keep its influence and repercussions after its terminus.
Addressing the factors that are common to both concepts and, therefore, also help to demonstrate how they are (inter)related, we also take this opportunity to approach and review the factors that have already been introduced in the previous sections. The articles analyzed stress the need to think about ways to extend projects beyond their funding period, as well as to plan knowledge exchange and science communication activities at the very beginning of the project (“research design”). This is indeed a critical stage because it includes reflection on several aspects that may influence either sustainability and/or future impact. In this sense, a few conditions must be considered as follows: the selection of an adequate team for research purposes, with the possible inclusion of a sustainability expert (A26), a knowledge broker (A2, A4, A7) and/or a program champion (A22, A28); the choice of a context where research is more favorable, suitable and necessary (A2, A3); or the selection of research methodologies that are articulated and linked to science dissemination activities (A5, A12).
Concurrently and with direct relation to research planning, our analysis highlights the urgency of including the non-academic community (“community engagement”) both in projects’ planning and development (A2, A3, A5, A9, A14, A20, A25, A26, A27, A28), because it allows for a greater focus on existing problems and triggers emotional factors (like a belonging feeling) that may concur to greater community involvement. As a consequence of that, further interest in keeping and extending projects may increase, as well as existing solutions to materialize this extension—which can be achieved through “institutional and/or organizational support”. In addition, knowledge exchange is equally amplified by this engagement.
Another factor with special emphasis is “funding” (or the lack of it), given that it affects projects’ longevity and, inherently, the planification, materialization and dynamization of impact-promoting activities (A3, A4, A9, A10), especially if it is allocated for short periods of time (A8, A20, A25, A28). This is a factor linked to overlapping positions and a lack of time (A3, A4, A14) since many researchers accumulate diverse functions in their working contexts and end up not being able to promote dynamics which, in most cases, are secondary compared to their main activity and for which they may not possess the necessary skills and expertise (A14).
Finally, “monitorization and assessment” allow collecting forms of evidence about obtained “results” and ensure a continuous review and redefinition of research projects, which contributes to their quality, leading to ongoing improvements that guarantee that they remain viable in the long term (A20, A22). Impact assessment may also be described as a potential learning and quality assurance process and was defended by Sánchez and Mitchell (A23) as an integral and purposeful component of the research’s impact.
However, in relation to this, another point in common between the two concepts is the struggle to evaluate them. In Sánchez-Carracedo, Cabré, Alier, Vidal, López, Martín and García’s work (A24), it is mentioned that, unlike other more objective areas (technical or financial, for instance), where quantitative measurements are used, most sustainability evaluations (in which research sustainability is included) are performed with intangible and proximal criteria. Woolcott, Keast and Pickernell (A18), in their turn, affirm that theoretical and empirical work, which is necessary in assessing and measuring non-academic impact, are in their embryonic stage. In this sense, Abou-Warda (A21) emphasizes that, to reach research sustainability, a new set of innovative skills and abilities is required: a premise that can be extrapolated to research impact as well. Nevertheless, a couple of the analyzed articles call attention to the potential of matrixes as a strategy that may be effective to assess and measure both research impact (A2) and sustainability (A24).
As we can see, apart from the flexibility between phases (“pre-project”, “during the project” and “post-project”), influencing factors also have flexibility between the concepts of research sustainability and impact since six in fifteen are common to both and those that have been specifically allocated to research sustainability or impact are not completely stationary within it either. For example, science communication and knowledge brokerage activities influence (positively or negatively) community involvement and, consequently, the emotional factors that can contribute to the sustainability of the project. This difficulty in separating and establishing the barriers between the two concepts, as we tried to demonstrate with the influencing factors in Table 3, is yet another point that attests to the symbiosis between them.

4. Conclusions

The results of this analysis allow a clearer insight into “research sustainability” and “research impact” concepts in the area of Education, highlighting the relations and connections between them. Therefore, this study provides an articulated vision of the concepts and highlights the influencing factors portrayed in the literature on research sustainability and impact in the area of Education.
First of all, we started by identifying reference authors and studies. In total, 98 authors and 47 studies were listed. From these numbers, 71 authors (72% of the total) and 33 studies (70% of the total) were related to research sustainability, and the remaining were linked to research impact. Such discrepancy between the two fields provides evidence that, in Education, the attention given to research impact is more concentrated on specific contexts and researchers than research sustainability, where the literature is more scattered.
In what relates to the discussion on the (inter)relation between research sustainability and impact in Education, the analysis showed the aspects in which these two concepts overlap, either in terms of conceptualization or concerning the factors that influence them. We begin by highlighting the intersections in the concepts definitions, namely because both stress the importance of emphasizing the research process as a whole rather than assigning excessive valorization to results. On top of that, both seek that the project’s effects, changes or benefits exceed the temporal limits of the action period. Given this, it is necessary to invest in community engagement, science communication and knowledge exchange initiatives and building bridges (even if they are only established by one person—the “knowledge broker”) between society and research. Moreover, it is urgent to create spaces in higher education teachers’ and researchers’ agendas so that impact activities can be included as formal assignments and tasks rather than on an arbitrary and unplanned basis. On top of that, it is important that these individuals are equipped with adequate tools and skills so that these activities can be carried out effectively instead of perpetuating a deficit in knowledge dissemination and promoting a lack of interest from the public in general.
In addition, given the relevance of monitorization and assessment to assure research projects’ quality and dissemination and, consequently, keep their viability in the long term, using one (or several) matrixes is a strategy that may be effective in assessing and measuring both research impact and sustainability. So, creating or adapting a matrix oriented for the Education area and focused on research impact and sustainability is one suggestion for future research.
Moreover, although the articles analyzed in this study stress that the concept of research impact in Education should take into account the non-academic context, the number of references on a scientific impact assessment based on metrics must be underlined. Some authors, like Naidorf [23] or Smith, Crookes and Crookes [24], are very critical regarding this approach since they underline that bibliometric measures do not reflect impact in practice and are a problematic and biased system. Despite bibliometrics’ potential to reflect some scholarly impact, in terms of citation rates, the valorization of quantity over quality “transforms creativity into an accommodative activity, rehashing or repeating ideas in order to line up to the expectations of the watchdogs of intellectual production” [23] (p. 9). In addition, these authors also assert that the “publish or perish” culture, which has been installed in research, makes researchers neglect other activities (related to dissemination and science communication) because they are not valued for academic career progression. Therefore, and according to the prominence assigned to the number of publications, researchers invest a big amount of their time trying to fill their curriculum with papers, which leaves them a small (or inexistent) portion of time to carry out the aforementioned activities, even though its resulting benefits may have greater relevance in the Education context than usually expensive and erudite publications.
To conclude, considering that globally, the result-oriented paradigm, focused on immediate (economic) profit, is currently being rethought, such changes should also happen at the Education and research level. The importance given to the number of publications should be adequate according to each area’s specificities. Research in Education, in particular, seems to need different means of disseminating its work and alternative evaluation tools, different from bibliometric measures, since the target audience of educational research (teachers, educators, parents or students) does not read academic productions. Therefore, a large amount of effort and time is invested in writing those papers, but they end up being mainly directed to other researchers and do not reach their real stakeholders. Such a cycle closes research over itself. Given that such a scenario is considered unwanted because the research’s outcomes are supposed to reach its real audience, educational research would certainly benefit from a decrease in the significance assigned to its number of publications. Impact evaluation (and academic career progression) must consider different criteria defined in accordance with the particularities of an area, and those should always be defined in accordance with research sustainability.

5. Limitations

In this final section, we address the limitations in our study mainly related to the corpus constitution. The non-widespread use of the expression “research sustainability” and the use of other equivalent expressions (such as project planning) may have diminished the corpus of articles since the method of our search was based on keywords. If there are articles addressing this topic using different terminologies from the ones known by the authors, their studies were inadvertently neglected. On top of that, there may be an endless number of articles that approach the (inter)relations of research sustainability and research impact in a language different from the ones the authors have reading proficiency in. This, together with the fact that “sustainability” is a concept with wide use and allocated to global concerns, could explain the difficulty in finding studies in the literature that addressed research sustainability in education.

Author Contributions

Conceptualization, R.T.; methodology, R.T.; software, R.T.; validation, R.T.; formal analysis, R.T.; investigation, R.T.; resources, R.T.; data curation, R.T.; writing—original draft preparation, R.T.; writing—review and editing, A.R.S. and S.P.; visualization, R.T.; supervision, A.R.S. and S.P.; project administration, R.T.; funding acquisition, R.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by FCT—Fundação para a Ciência e a Tecnologia, I.P., grant number 2020.06875.BD.

Data Availability Statement

The data that support the findings of this study are available on the analyzed articles, properly referenced in the appendices. More information regarding coded data can be provided from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A. Research Impact—Corpus

CodeReferenceLanguageKeywords
A1Yates, L. Is Impact a Measure of Quality? Some Reflections on the Research Quality and Impact Assessment Agendas. European Educational Research Journal 2005, 4, 391–403. https://doi.org/10.2304/eerj.2005.4.4.5EnglishNone
A2Scoble, R.; Dickson, K.; Hanney, S.; Rodgers, G.J. Institutional Strategies for Capturing Socio-Economic Impact of Academic Research. Journal of Higher Education Policy and Management 2010, 32, 1–13. https://doi.org/10.1080/1360080X.2010.511122EnglishResearch assessment, research capital, research strategy, socio-economic impact
A3Upton, S. Identifying Effective Drivers for Knowledge Exchange in the United Kingdom. Higher Education Management and Policy 2012, 24, 1–20. https://doi.org/10.1787/hemp-24-5k9bdsv6wms1EnglishNone
A4Knight, C.; Lightowler, C. Sustaining knowledge exchange and research impact in the social sciences and humanities: Investing in knowledge broker roles in UK universities. Evidence and Policy 2013, 9. https://doi.org/10.1332/174426413X662644EnglishHigher education, knowledge brokers, knowledge exchange, research impact
A5Oancea, A. Research Impact and Educational Research. European Educational Research Journal 2013, 12, 242–250. http://dx.doi.org/10.2304/eerj.2013.12.2.242EnglishNone
A6Smith, K.; Crookes, E.; Crookes, P. Measuring research ‘impact’ for academic promotion: issues from the literature, Journal of Higher Education Policy and Management 2013, 35, 410–420. https://doi.org/10.1080/1360080X.2013.812173EnglishBibliometrics, ERA, Excellence in Research for Australia, impact, research quality
A7Naidorf, J. Knowledge Utility: From Social Relevance to Knowledge Mobilization. Education Policy Analysis Archives 2014, 22, 1–31. http://dx.doi.org/10.14507/epaa.v22n89.2014EnglishKnowledge utility, social relevance of research, knowledge mobilization, scientific policy
A8Hazelkorn, E. Making an impact: New directions for arts and humanities research. Arts and Humanities in Higher Education 2015, 14, 25–44. https://doi.org/10.1177/1474022214533891EnglishArts and humanities, economic development, global crisis, impact, Ireland, Netherlands, Norway
A9Gunn, A.; Mintrom, M. Evaluating the non-academic impact of academic research: design considerations. Journal of Higher Education Policy and Management 2017, 39, 20–30. https://doi.org/10.1080/1360080X.2016.1254429EnglishAcademic research, evaluation methods, policy design, research assessment, research funding cycles, research impact, non- academic impact
A10Doyle, J. Reconceptualising research impact: reflections on the real-world impact of research in an Australian context. Higher Education Research & Development 2018, 37, 1366–1379. https://doi.org/10.1080/07294360.2018.1504005EnglishHigher education policy, research impact, Australia, academic identity, qualitative research
A11Mitchell, V. A proposed framework and tool for non-economic research impact measurement. Higher Education Research & Development 2019, 38, 819–832. https://doi.org/10.1080/07294360.2019.1590319EnglishImpact policy, impact measurement, research outcomes, research evaluation
A12O’Connell, C. Examining differentiation in academic responses to research impact policy: mediating factors in the context of educational research. Studies in Higher Education 2019, 44, 1438–1453. https://doi.org/10.1080/03075079.2018.1447556EnglishPolicy analysis, relational perspective, research policy, tertiary education, institutional research
A13Scruggs, R.; McDermott, P.; Qiao, X. A Nationwide Study of Research Publication Impact of Faculty in U.S. Higher Education Doctoral Programs. Innovative Higher Education 2019, 44. https://doi.org/10.1007/s10755-018-9447-xEnglishHigher education, doctoral education, h-index, research impact
A14Wilkinson, C. Evidencing impact: a case study of UK academic perspectives on evidencing research impact. Studies in Higher Education 2019, 44, 72–85. https://doi.org/10.1080/03075079.2017.1339028EnglishImpact, research, engagement, evidence, assessment
A15Gunn, A.; Mintrom, M. Measuring research impact in Australia. Australian Universities’ Review 2020, 60, 9–15EnglishHigher education, research funding, research evaluation, impact and engagement, innovation policy
A16Jerome, L. Making sense of the impact agenda in UK higher education: A case study of Preventing Violent Extremism policy in schools. Journal of Social Science Education 2020, 19, 8–23. https://doi.org/10.4119/jsse-1558EnglishImpact, evaluation, performativity, preventing violent extremism (PVE), Prevent
A17Papatsiba, V.; Cohen, E. Institutional hierarchies and research impact: new academic currencies, capital and position-taking in UK higher education. British Journal of Sociology of Education 2020, 41, 178–196. https://doi.org/10.1080/01425692.2019.1676700EnglishResearch Excellence Framework, research policy, impact, universities, Bourdieu
A18Woolcott, G.; Keast, R.; Pickernell, D. Deep impact: re-conceptualising university research impact using human cultural accumulation theory. Studies in Higher Education 2020, 45, 1197–1216. https://doi.org/10.1080/03075079.2019.1594179EnglishImpact, collaboration, university research, social network analysis, university- community

Appendix B. Research Sustainability—Corpus

CodeReferenceLanguageKeywords
A19Sharp, A. ELT project planning and sustainability. ELT Journal 1998, 52, 140–145. https://doi.org/10.1093/elt/52.2.140EnglishNone
A20Loh, L., Friedman, S. & Burdick, W. Factors Promoting Sustainability of Education Innovations: A Comparison of Faculty Perceptions and Existing Frameworks. Education for Health 2013, 26, 32–38. https://doi.org/10.4103/1357-6283.112798EnglishDeveloping countries, medical education, medical faculty, organizational innovation, program sustainability
A21Abou-Warda, S. Mediation effect of sustainability competencies on the relation between barriers and project sustainability (the case of Egyptian higher education enhancement projects). Sustainability Accounting, Management and Policy Journal 2014, 5, 68–94. https://doi.org/10.1108/SAMPJ-04-2011-0017EnglishNone
A22Guerra, C.; Costa, N. Sustentabilidade da investigação educacional: contributos da literatura sobre o conceito, fatores e ações. Revista Lusófona da Educação 2016, 13–25PortugueseProject manager, sustainability management, sustainability competencies, barriers to sustainability, Egyptian HEEPs, project sustainability
A23Sánchez, L.; Mitchell, R. Conceptualizing impact assessment as a learning process. Environmental Impact Assessment Review 2016, 62, 195–204. https://doi.org/10.1016/j.eiar.2016.06.001EnglishSustentabilidade da investigação, investigação educacional, avaliação do impacte da investigação
A24Sánchez-Carracedo, F.; Cabré, J.; Alier, M.; Vidal, E.; López, D.; Martín, C.; García, J. A Learning Tool to Develop Sustainable Projects. IEE Frontiers in Education Conference 2016, 1–9. https://doi.org/10.1109/FIE.2016.7757468EnglishSustainability, engineering projects, final degree projects
A25Sparks, J.; Rutkowski, D. Exploring project sustainability: using a multiperspectival, multidimensional approach to frame inquiry. Development in Practice 2016, 26, 308–320. http://dx.doi.org/10.1080/09614524.2016.1153041EnglishAid-Aid effectiveness, monitoring and evaluation, development policies, civil society-partnership, participation, social sector-education
A26Aarseth, W.; Ahola, T.; Aaltonen, K.; Okland, A.; Andersen, B. Project sustainability strategies: a systematic literature review. International Journal of Project Management 2017, 35, 1071–1083. https://doi.org/10.1016/j.ijproman.2016.11.006EnglishSustainability, project management, project host, sustainability strategies
A27Aga, D.; Noorderhaven, N.; Vallejo, B. Project beneficiary participation and behavioural intentions promoting project sustainability: The mediating role of psychological ownership. Development Policy Review 2017, 36, 527–546. https://doi.org/10.1111/dpr.12241EnglishCommunity participation, project sustainability, psychological ownership
A28Guerra, C.; Costa, N. Educational innovations in Engineering education: sustainability of funded projects developed in Portuguese higher education institutions. In Proceedings of the 3rd International Conference of the Portuguese Society for Engineering Education, Aveiro, Portugal, 27–29 June 2018. https://doi.org/10.1109/CISPEE.2018.8593497EnglishEducational innovations, engineering education; sustainability of funded research; Portuguese higher education institutions
A29Khalifeh, A.; Farrell, P.; Al-edenat, M. The impact of project sustainability management (PSM) on project success: a systematic literature review. Journal of Management Development 2019, 39, 453–474. https://doi.org/10.1108/JMD-02-2019-0045EnglishProject management, sustainability, project, project success, project sustainability management, triple-bottom line
A30Guerra, C. Sustentabilidade da investigação em educação: da concepção à implementação de um referencial. Revista Práxis Educacional 2021, 17, 1–17. https://doi.org/10.22481/praxisedu.v17i48.8819PortugueseSustentabilidade da investigação; inovações educativas; investigação & desenvolvimento
A31Guerra, C.; Costa, N. Can Pedagogical Innovations Be Sustainable? One Evaluation Outlook for Research Developed in Portuguese Higher Education. Education Sciences 2021, 11, 1–18. https://doi.org/10.3390/educsci11110725EnglishSustainability of research, institutional support, political and research agendas, projects dynamics

References

  1. Guerra, C. Sustentabilidade da investigação em educação: Da concepção à implementação de um referencial. Rev. Práxis Educ. 2021, 17, 196–212. [Google Scholar] [CrossRef]
  2. König, T.; Nowotny, H.; Schuch, K. Impact re-loaded. Fteval J. Res. Technol. Policy Eval. 2019, 48, 8–9. [Google Scholar] [CrossRef]
  3. Aarseth, W.; Ahola, T.; Aaltonen, K.; Okland, A.; Andersen, B. Project sustainability strategies: A systematic literature review. Int. J. Proj. Manag. 2017, 35, 1071–1083. [Google Scholar] [CrossRef]
  4. Oancea, A. Research Impact and Educational Research. Eur. Educ. Res. J. 2013, 12, 242–250. [Google Scholar] [CrossRef]
  5. European Commission. European Research Council Work Programme 2022; European Comission: Brussels, Belgium, 2022; Available online: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2022/wp_horizon-erc-2022_en.pdf (accessed on 6 October 2023).
  6. Aga, D.; Noorderhaven, N.; Vallejo, B. Project beneficiary participation and behavioural intentions promoting project sustainability: The mediating role of psychological ownership. Dev. Policy Rev. 2017, 36, 527–546. [Google Scholar] [CrossRef]
  7. Scoble, R.; Dickson, K.; Hanney, S.; Rodgers, G.J. Institutional Strategies for Capturing Socio-Economic Impact of Academic Research. J. High. Educ. Policy Manag. 2010, 32, 499–510. [Google Scholar] [CrossRef]
  8. Vanholsbeeck, M.; Lendák-Kabók, K. Research impact as a ‘boundary object’ in the social sciences and the humanities. Word Text 2020, 10, 29–52. [Google Scholar]
  9. Frederiksen, L.; Phelps, S.F.; Kimmons, R. Introduction to literature reviews. In Rapid Academic Writing, 2nd ed.; Kimmons, R., West, R.E., Eds.; EdTech Books: Utah, UT, USA, 2018; Available online: https://edtechbooks.org/rapidwriting/lit_rev_intro (accessed on 19 October 2023).
  10. Amundsen, C.; Wilson, M. Are We Asking the Right Questions?: A Conceptual Review of the Educational Development Literature in Higher Education. Rev. Educ. Res. 2012, 82, 90–126. [Google Scholar] [CrossRef]
  11. Grant, M.J.; Booth, A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Inf. Libr. J. 2009, 26, 89–168. [Google Scholar] [CrossRef] [PubMed]
  12. Cardoso, T.; Alarcão, I.; Celorico, J.A. Revisão da Literatura e Sistematização do Conhecimento; Porto Editora: Porto, Portugal, 2010. [Google Scholar]
  13. Newman, M.; Gough, D. Systematic Reviews in Educational Research: Methodology, Perspectives and Application. In Systematic Reviews in Educational Research: Methodology, Perspectives and Application, 1st ed.; Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K., Eds.; Springer VS: Wiesbaden, Germany, 2020; pp. 3–22. [Google Scholar]
  14. Bogdan, R.C.; Biklen, S.K. Qualitative Research for Education: An Introduction to Theories and Methods, 5th ed.; Pearson A & B: Boston, MA, USA, 2007. [Google Scholar]
  15. Coutinho, C.P. Metodologia de Investigação em Ciências Sociais e Humanas: Teoria e Prática, 2nd ed.; Almedina: Coimbra, Portugal, 2014. [Google Scholar]
  16. Bardin, L. Análise de Conteúdo, 4th ed.; Edições 70; Lda: Lisboa, Portugal, 2009. [Google Scholar]
  17. Bryman, A. Social Research Methods, 4th ed.; Oxford University Press: Oxford, UK, 2012. [Google Scholar]
  18. Krippendorff, K. Content Analysis: An Introduction to Its Methodology, 2nd ed.; Sage Publications, Inc.: Thousand Oaks, CA, USA, 2004. [Google Scholar]
  19. Muller, J.Z. The Tyranny of Metrics, 1st ed.; Princeton University Press: Princeton, NJ, USA, 2018. [Google Scholar]
  20. Cameron, W. Informal Sociology: A Casual Introduction to Sociological Thinking, 6th ed.; Random House: New York, NY, USA, 1963. [Google Scholar]
  21. REF. Research Excellence Framework 2014: The Results; Research Excellence Framework: Bristol, UK, 2014; Available online: https://www.bristol.ac.uk/research/environment/assessment/ref2014/ (accessed on 6 October 2023).
  22. Mercer-Mapstone, L.; Kuchel, L. Core Skills for Effective Science Communication: A Teaching Resource for Undergraduate Science Education. Int. J. Sci. Educ. Part B 2015, 7, 181–201. [Google Scholar] [CrossRef]
  23. Naidorf, J. Knowledge Utility: From Social Relevance to Knowledge Mobilization. Educ. Policy Anal. Arch. 2014, 22, 1–31. [Google Scholar] [CrossRef]
  24. Smith, K.; Crookes, E.; Crookes, P. Measuring research ‘impact’ for academic promotion: Issues from the literature. J. High. Educ. Policy Manag. 2013, 35, 410–420. [Google Scholar] [CrossRef]
Figure 1. Research impact analysis of the corpus’ constitution process flowchart.
Figure 1. Research impact analysis of the corpus’ constitution process flowchart.
Education 14 00147 g001
Table 1. Inclusion criteria.
Table 1. Inclusion criteria.
CriteriaDescriptors
  • Textual typology
The document is a scientific article.
b.
Open access
The article is in open access.
c.
Scientific validity
The article was submitted to peer review.
d.
Writing language
The article is written in one of the following languages, according to the authors’ linguistic competences: Portuguese, English or Spanish.
e.
Suitability
The article meets the research question and objectives for the proposed literature review.
Table 2. Summary of the different searches carried out for “research sustainability”.
Table 2. Summary of the different searches carried out for “research sustainability”.
DatabaseSearch CodeResultsIncluded
ERIC“project sustainability” 1731
Scielosustentabilidade da investigação 2850
Redalyc“sostenibilidad de la investigación”771
Google Scholar“sustentabilidade da investigação”484
Scopus“project sustainability” AND education777
ISI“project sustainability” AND education1774
Emerald“project sustainability” AND education1952
Science Direct“project sustainability” 31282
1 Peer review filter was applied. 2 Quotation marks were not used because, with them, the search was null. 3 Title, abstract and keywords search was carried out.
Table 3. Factors influencing research sustainability, impact or both.
Table 3. Factors influencing research sustainability, impact or both.
SustainabilityCommon ZoneImpact
Pre-project Research design Pre-project
During the project Institutional and organizational supportScience communicationDuring the project
Sustainability competencesCommunity engagementLack of training
Program championsFundingKnowledge brokerage
Feeling of belongingMonitorization and assessmentResearch methodology
Overlapping positions and lack of time
Post-projectPeer supportResults Post-project
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Torres, R.; Simões, A.R.; Pinto, S. Research Impact and Sustainability in Education: A Conceptual Literature Review. Educ. Sci. 2024, 14, 147. https://doi.org/10.3390/educsci14020147

AMA Style

Torres R, Simões AR, Pinto S. Research Impact and Sustainability in Education: A Conceptual Literature Review. Education Sciences. 2024; 14(2):147. https://doi.org/10.3390/educsci14020147

Chicago/Turabian Style

Torres, Ricardo, Ana Raquel Simões, and Susana Pinto. 2024. "Research Impact and Sustainability in Education: A Conceptual Literature Review" Education Sciences 14, no. 2: 147. https://doi.org/10.3390/educsci14020147

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop