A Framework for Assessing the Climate Impacts of Research and Innovation Projects and Programmes

Abstract

Public spending on research and innovation (R&I) to tackle grand societal challenges, such as climate change, is increasing. Consequently, research funding organisations face an ever-growing demand to demonstrate the social return of their investments. However, tools and frameworks that facilitate the description and assessment of the climate impacts of R&I activities are largely lacking. The present paper addresses this gap by conducting a comparative case analysis of corporate R&I projects co-funded by a thematically open R&I funding programme with the aim of identifying the key impact pathways. Data for the cases were collected through document analysis and video interviews. The results show that assessments of the climate impacts of R&I need to take into account the expected change in technology maturity and the impacts that are caused up- and downstream in a company’s value chain. The results serve as a basis for the development of a framework for the ex-ante assessment of climate impacts of public R&I programmes. A series of workshops with research funders, companies, and evaluation experts helped refine and validate the framework and ensure its real-world applicability. The framework provides proposal writers and evaluators with a pragmatic and easy-to-use orientation tool for describing and assessing the climate impacts of a planned R&I activity during the proposal evaluation stage. It therefore supports a more systematic and systemic assessment of R&I impacts that can help funding organisations better address the challenges of climate change.

1. Introduction

Research and innovation (R&I) are considered key for tackling grand societal challenges such as climate change mitigation and adaptation or the overexploitation of natural resources [1,2,3,4]. Ever larger amounts of financial resources are invested by both governments and companies into the development of technical and societal solutions that, for example, reduce energy consumption in industry and households, decarbonise transport systems, improve the circularity of material flows, and foster sustainable consumer behaviour [4,5,6]. While R&I input can easily be quantified in monetary terms, attempts to measure the extent to which funded R&I activities actually contribute to tackling societal challenges such as climate change represent an ongoing struggle [5,7,8,9]. Governments and research funding organisations consequently face an ever-increasing demand to demonstrate the so-called “social return” of their public investments into R&I [10,11,12].

To address this demand, public R&I programmes are increasingly subject to extensive monitoring and evaluation schemes, including ex-ante, interim (mid-term), and ex-post approaches [10,13,14]. Since the realisation of expected impacts largely depends on the initial design of an R&I project [15], the ex-ante assessment of research proposals can be considered to be the most powerful factor in ensuring that the R&I funding provided supports the funding programme’s objectives [16,17]. At the same time, ex-ante assessment approaches are also the most challenging, especially when R&I activities are per se already associated with a rather high degree of uncertainty regarding their eventual outcomes [10,13,14,18]. Much of the research evaluation literature thus focuses on ex-post approaches, while the development of methods that support the ex-ante assessment of the societal impacts of R&I has received considerably less attention [10,19]. Especially in the area of climate change, there is currently no assessment framework that would provide proposal writers and evaluators with a pragmatic and easy-to-use orientation tool for (better) describing and assessing the climate impacts of a planned R&I activity (see [9]).

The present study addresses this gap by developing a framework that supports research funding organisations in the ex-ante assessment of their to-be-funded R&I projects. To this end, this study explores the expected impact pathways of R&I projects through a comparative case study approach. The selected cases cover corporate R&I projects from the Austrian manufacturing sector that have been co-funded by the Austrian Research Promotion Agency’s (FFG) General Programme (“Basisprogramm”). For the empirical investigation of these cases, FFG has granted the authors access to the respective research proposals from the companies involved (after obtaining their consent to share this information). The analysis of these cases focuses on the climate impacts of the funded R&I projects, thereby addressing an issue that is considered the single most important existential threat to human civilisation [20,21] and that is therefore increasingly central to European R&I programmes [3,22,23,24].

The remainder of this paper is structured as follows. First, the challenges of assessing the societal impacts of R&I—especially regarding climate change—are reviewed. Second, the method and the selected cases are described. Section 3 presents the results of the cross-case synthesis, while Section 4 introduces the proposed framework and discusses its limitations. Section 5 draws conclusions regarding the applicability of the developed framework for research funding organisations and companies.

1.1. Assessing the Societal Impacts of Research and Innovation Is Both Relevant and Challenging

Research funding organisations play a particularly important and powerful role in society by investing large amounts of public money into research and innovation (R&I) activities and thereby guiding both basic research and corporate innovation processes towards areas of public interest [16,17]. For example, the European Union’s (EU) research framework programme alone will spend more than EUR 95 billion in the period from 2021 to 2027 on fostering R&I activities across Europe and beyond [25]. Public research funding attracts additional R&I spending, especially from companies [26,27,28]. In 2021, total spending on R&I in the EU amounted to about EUR 330 billion, equalling 2.3% of the EU’s gross domestic product [29]. In countries with a strong high-tech industrial sector, such as Sweden, Belgium, or Austria, R&I expenditure accounts for more than 3% of GDP [29]. On average, for every euro spent by governments on R&I across Europe, the business sector spends almost two euros [27,29].

Due to their key role in setting the R&I agenda, research funding organisations have increasingly been scrutinised with regards to the (societal) objectives that their programmes are addressing and the (societal) impact that the R&I projects they fund are achieving [3,5,10,11,12]. Increased awareness of the importance of R&I for addressing grand societal challenges such as climate change has led to a shift in both the thematic orientation of R&I programmes and the relevance of impact to the evaluation of research proposals [2,12,16,22,30]. While historically public R&I programmes (and their impact assessments) mainly focused on contributing to competitiveness, job creation, facilitating knowledge transfer, and improving the mobility of and cooperation between researchers [1,7,31,32], many R&I funding schemes now follow a challenge-driven structure, for example, by dedicating funding streams to climate-related research [22,23,24]. One of the most prominent examples in this regard is Horizon Europe, the EU’s current research framework programme, which is spending more than 42% of its budget for the 2023 to 2024 period—around EUR 5.7 billion out of a total budget of EUR 13.5 billion for these two years—on R&I addressing climate change mitigation and adaptation [24]. Moreover, while the decision on whether a proposed R&I activity should receive funding has historically been almost entirely based on scientific excellence, the (societal) impact of research is nowadays considered an equally important evaluation criterion in many funding programmes, including Horizon Europe [7,14,16].

The literature on assessing the impacts of R&I has a long history, going back to the 1950s [32,33]. For example, in 1958, researchers were already attempting to assess the rate of return on investments in R&I, noting that “we have some idea of how much we have spent but very little of what we got in return” [34]. While initially, impact assessments mainly focused on economic returns from R&I spending, their scope broadened over time to also take into account other (societal) benefits relating to health, the environment, or quality of life [1,7,8,11,31]. At the same time, this broadening of the scope made the assessments themselves more challenging, partly due to different definitions and understandings of the term “impact” itself [7,11,35,36].

The assessment of research impacts becomes even more complex when considering climate change mitigation, where the focus is on reducing greenhouse gas (GHG) emissions (measured in carbon dioxide equivalents (CO₂ eq.)). R&I activities can contribute to this objective indirectly by reducing energy consumption or switching to less carbon-intensive fuels (for example, switching from fossil fuels to renewables), and directly by reducing process-related emissions [37]. While direct emission reductions usually occur at a company’s site, reductions in energy consumption might also materialise up- or downstream in the value chain, for example, as a result of switching to less carbon-intensive raw materials in the supply chain or when a manufactured product requires less energy in its use-phase [38]. In addition, R&I activities do not necessarily directly lead to actual changes in a company’s production processes or its manufactured products; they might only provide the necessary knowledge for doing so at a later stage (for example, in the form of prototypes) [6,39].

1.2. Assessment of the Climate Impacts of R&I Should Consider Technology Maturity and Value Chains

The review above suggests that assessment of the climate impacts of R&I represents a specific, multidimensional challenge. On the one hand, assessments of the magnitude of an expected climate impact—and which part of this impact can be attributed to the provided (public) funding—need to take into account the maturity of the technology developed, as the expected climate impact(s) will only materialise once the technology is applied [22]. On the other hand, the assessment also needs to consider at which stage of the value chain the impact will materialise, since the more downstream an impact is expected to occur, the more its realisation will depend on factors outside a company’s sphere of influence [40]. The development of an assessment framework for the climate impacts of R&I consequently needs to take into account both dimensions: the logic of innovation (technological development) and the logic of corporate (climate) impacts along the whole value chain.

The technology readiness level (TRL) scale can be considered as the standard approach for assessing technology maturity in contemporary European R&I funding programmes [41]. While the TRL scale was initially developed by NASA for space technologies, with a respective international standard—the ISO 16290:2013 standard for space systems—adopted in 2013 [42], it is now used more broadly in a variety of sectors such as nanotechnology, biotechnology, advanced materials, and advanced manufacturing [41,43]. In its current context, the TRL scale can be understood as an indicator of how far a technology is from being applied in its intended operational environment, or, in a looser interpretation, how far a product is from marketability [41]. Although the TRL scale has received some criticism, for example, regarding its focus on single technologies, its assumption of a linear innovation process, and the lack of attention it pays to setbacks in technology maturity [43], it is now a standard element in European R&I funding programmes such as Horizon Europe (and its predecessor, Horizon 2020). As shown in

Table 1. Technology readiness levels (TRLs) used in European funding programmes [41,44].

TRL	Definition
TRL 1	Basic principles observed
TRL 2	Technology concept formulated
TRL 3	Experimental proof of concept
TRL 4	Technology validated in lab
TRL 5	Technology validated in relevant environment (industrially relevant environment in the case of key enabling technologies)
TRL 6	Technology demonstrated in relevant environment (industrially relevant environment in the case of key enabling technologies)
TRL 7	System prototype demonstration in operational environment
TRL 8	System complete and qualified
TRL 9	Actual system proven in operational environment (competitive manufacturing in the case of key enabling technologies or in space)

, the TRL scale ranges from a basic understanding of principles (TRL 1) to the proven functionality of a system in its operational environment (TRL 9) [41,44].

Regarding the assessment of (corporate) climate impacts, the GHG protocol developed by the World Resources Institute and the World Business Council for Sustainable Development in 2004 [37] can be considered as the standard approach [15,45]. The rationale of the GHG protocol can be traced back to approaches to impact-oriented corporate sustainability which acknowledge that all companies have impacts on the environment and society through the processing of inputs and their transformation into marketable products and services. These are brought to market, bought, used, and eventually disposed of, and each step in this value chain uses resources, generates waste, and affects stakeholders outside the company [46]. The GHG protocol takes up this logic by defining three “scopes” that distinguish between direct and indirect emissions. Scope 1 refers to direct emissions at a company’s sites, scope 2 refers to indirect emissions from purchased electricity, and scope 3 refers to indirect emissions caused up- and downstream in a company’s value chain [37,38]. While scope 1 and 2 emissions refer to activities directly under a company’s sphere of control and are therefore quite easy to assess, scope 3 emissions also include those caused by activities not (fully) controlled by a company, such as the aftermarket processing and use of their products and their end-of-life treatment [37,38,47]. The assessment of scope 3 emissions is therefore quite challenging, most importantly due to lack of data and the difficulty of attributing emissions occurring downstream in the value chain [15,40,48,49,50].

A central novel contribution of the present paper lies in the combination of the assessment of technology maturity (TRLs) with the assessment of corporate climate impacts along value chains (the GHG protocol), as shown in Figure 1. Since both TRLs and the GHG protocol put particular emphasis on supporting decision making during (future) project implementation [38,42,47], the present study focuses on the ex-ante assessment of R&I activities that usually takes place at the proposal evaluation stage. The focus is thus on supporting both proposal writers and evaluators in their assessment of climate impacts of (corporate) R&I projects, thereby addressing the day-to-day management of R&I programmes. Since proposal writing and evaluation are already quite challenging and resource-intensive processes [5], a central objective of the present study is to provide a pragmatic and easy-to-use orientation tool for assessing climate impacts at this stage, rather than to develop a comprehensive quantitative approach that would make both processes even more complex.

2. Materials and Methods

The present study applied an exploratory case study approach using a multiple-case design [51,52]. Case study research is a prominent approach in qualitative social sciences that allows an in-depth investigation of a phenomenon in its real-life context, drawing from multiple sources of evidence [51,52,53]. It is the preferred approach in situations where the research (a) mainly attempts to answer “how” or “why” questions, (b) has little or no control over behavioural events, and (c) focuses on a contemporary (as opposed to an entirely historical) phenomenon [51]. Case study research is considered to be particularly well suited to R&I impact assessments due to its ability to capture the context-specific and variable nature of impacts [31,54,55]. The multiple-case design applied in this study is a mix of single and embedded units of analysis [51]. The cases under scrutiny are individual R&I projects carried out by companies, with some companies running two projects (and thus cases) in parallel. A multiple-case design offers analytical benefits (such as more robust results) over single-case settings, including allowing for a cross-case synthesis [51].

The funding programme chosen for conducting the analysis was the General Programme (in German “Basisprogramm”) of the Austrian Research Promotion Agency (FFG). There were two reasons for this. On the one hand, FFG granted the authors access to research proposals submitted for funding by companies and thereby to documents and data that are otherwise kept strictly confidential. On the other hand, FFG’s General Programme can be considered as an archetypical example of a European funding programme aimed at supporting the innovativeness and competitiveness of companies. Funding agencies with similar programmes include Bpifrance (France), CDTI (Spain), VINNOVA (Sweden), and Business Finland, as well as—at EU level—the European Innovation Council (EIC) and the European Institute of Innovation and Technology (EIT) (see [56]). FFG’s General Programme is open to all branches of industry and research topics, and companies and projects of all sizes are eligible. It aims to strengthen the competitiveness of companies by co-funding the development of new products, processes, and services, providing up to 50% of the total cost of each project. Submitted proposals are evaluated in terms of their technical and economic aspects, including the degree of innovation involved, the technological challenge, the potential for commercialisation, and the economic performance of the applicant (see [57]).

The selection of companies and cases for the present study was carried out by the authors in co-ordination with FFG. The selection strategy focused on identifying companies that (a) have considerable (expected) climate impacts due to their energy- or carbon-intensive processes, (b) are technology/market leaders in their respective sectors, and (c) operate in sectors (for example, steel production) with particular relevance for the Austrian economy and for climate change mitigation. Additionally, within the chosen companies, case selection focused on large-scale R&I projects that could be expected to lead to reduced climate impacts, either as primary or secondary objectives. The applied purposive sampling strategy can therefore be considered a combination of ‘critical case sampling’ and ‘politically important sampling’ [52]. It resulted in the selection of 13 cases (R&I projects) carried out in eight companies (see

Table 2. Companies and R&I projects selected for this study. The specification of the sector refers to the scope of the selected R&I projects and does not necessarily cover the full spectrum of a company’s activities.

Company	Sector	Project (=Case)
Agrana	Food products	EMiCo (energy minimization and colour reduction) Protein4Decarbon (protein upgrade for decarbonisation)
AMAG	Metals (aluminium)	OExA (Operational Excellence Automotive)
Engel	Machinery	EVOLVE (modules for machinery for fully automated production of thermoplastic fibre composites) FULIO (future portfolio)
Lenzing	Textiles	New Technologies for Sustainable Growth Upscale to Circularity
Magna Powertrain	Automotive	etelligentDrive system
Magna Steyr	Automotive	ProSIT (new processes in function development, simulation, and testing) NeMo (e-vehicle concepts for new mobility)
Montanwerke Brixlegg	Metals (copper)	Valuable Metals Pellets (agglomerated residues from metal recycling for the smelting metallurgical recovery of valuable metals)
Voestalpine	Metals (steel)	greentecFLAT (greentec steel for flat products) HiPer (highly permeable electrical steel for high-torque drives)

).

The analysis of these cases relied on two main sources. Firstly, FFG provided the authors with access to the project proposals for the selected cases (after consent was obtained from the respective companies to share this information). These proposals followed a common structure defined by FFG for applications for funding from its General Programme, containing inter alia descriptions of the company, the proposed project, and the resources required to carry out the project, as well as a market analysis with plans for exploitation and dissemination. The description of the project included information about its objectives, how the proposed project goes beyond the current state of art, the pros and cons of the (technical) solution for the company and its clients, and the project’s impacts on the environment and on resource and energy use, as well as a description of the (technical) risks and mitigation options. In cases where funding from FFG had been requested (and granted) for several consecutive years (up to three), more than one proposal was available for a project. Where necessary, additional contextual information about the company was obtained online, for example, from the company’s website.

Video interviews between the study authors and company representatives—including both project managers and technical experts—constituted the second main data source. These calls lasted between one and two hours and focused on obtaining additional information that was not described (or was only described to a limited extent) in the respective proposals, as well as on confirming the authors’ understanding of the project’s objectives and its direct and indirect climate impacts. The interviews were semi-structured in order to allow for an in-depth discussion of specific issues relevant to the case in question. The interviews were conducted by all four authors, with two conducting the interview and the other two taking notes [53]. Eight interviews were conducted (one per company), involving between one and four company representatives and covering all cases of the respective company. The additional data collected through the interviews included the strategic relevance of climate, sustainability, and responsibility issues for the company, the expected improvement in the technology readiness level (TRL) (if this information was lacking from the proposal), the extent to which the developed technology could be replicated and upscaled both within the company and in other companies (for example, in the same industrial sector), and which further climate benefits could be gained from this.

In order to gain a deeper understanding of the observed phenomena and to increase the generalisability of results, within-case analyses were conducted and followed by a qualitative cross-case synthesis [51,58]. First, the 13 cases were examined separately, processing the aforementioned sources and summarising the available information according to the assessment logic described in Section 1.2 [59,60]. To enhance internal validity, all four authors analysed the data, first individually and then together, following the principle of investigator triangulation [61]. Based on the previous steps, the authors identified similar themes, cross-case patterns, within-dimension similarities and differences, and resulting categories. This involved identifying relevant dimensions, grouping the cases by internal similarity and external heterogeneity, and characterising the constructed types as combinations of their attributes [62].

The cross-case synthesis [51] focused on comparing the cases with regards to (a) how the technical solutions developed contribute to reducing climate impacts, (b) at which stage of the company’s value chain these impacts will materialise, and (c) which components of these impacts can be attributed to the funding received via FFG’s General Programme. A central objective of the cross-case synthesis was to refine the two-dimensional logic of technology maturity and value chain impacts (see Figure 1) with additional elements extracted from the cases. The development of the framework involved multiple rounds of feedback, discussion, and validation. Draft versions of the framework were discussed in workshops with thematic experts from FFG, with representatives from the case companies, and with external experts. The experts from FFG provided feedback from the perspective of proposal evaluators and programme managers (i.e., those responsible for managing the public funding of the selected projects), while the company representatives added the perspective of those applying for funding (i.e., proposal writers). The involvement of additional external experts served as a litmus test of the framework, ensuring its validity and completeness as well as discussion of its limitations, for example, with regards to applying it to other types of R&I funding programmes.

An important limitation of the present study concerns the disclosure of information from the cases. Since the analysed cases refer to ongoing corporate R&I projects of strategic (competitive) relevance, the participating companies have a strong interest in the confidentiality of the information and data related to their cases. Apart from aggregated or anonymised data, the authors can therefore only disclose certain (technical) details of the individual cases. Since the purpose of this paper is to develop a framework for the assessment of the climate impacts of corporate R&I activities, the information that the companies agreed to disclose is mainly used to illustrate the relevance of the framework’s individual elements in Section 3.2, while the overview of the case studies in Section 3.1 is necessarily limited to aggregated and anonymised information.

3. Results

3.1. Overview of Case Studies

Overall, this study covered 13 R&I projects carried out in eight companies. Eight of these projects had been going on for a year, while the remaining five had received follow-up funding for a second or a third year. The total funding provided by FFG for the analysed cases amounted to EUR 18.3 million, with the amount of funding provided for the individual cases ranging from EUR 200,000 to EUR 3.3 million; the majority of cases received funding of less than EUR 1 million. Since the funding provided in the FFG’s General Programme covered only around a quarter of the total project costs, the 13 analysed project cases accounted for about EUR 73 million of spending on R&I activities in the eight companies.

Table 3. Ranking of project objectives of the analysed R&I project cases. The importance was rated on a scale from 1 (very high importance) to 5 (very low importance). Objectives related to reducing climate impacts are highlighted in bold.

Project Objectives	Mean Score	1 (Very High)	2 (High)	3 (Medium)	4 (Low)	5 (Very Low)
Achieve or improve technological leadership vis-á-vis competitors	1.46	10	0	3	0	0
Develop or enable new products	1.77	7	4	1	0	1
Increase flexibility	1.77	6	5	1	1	0
File patents	2.08	5	4	2	2	0
Reduce energy consumption & process-related GHG emissions	2.15	5	2	5	1	0
Decrease production costs	2.31	2	7	2	2	0
Reduce indirect climate impacts	2.38	2	7	1	3	0
Switch to different and/or cheaper raw materials	2.62	3	3	4	2	1
Improve re-use of waste or intermediary products	2.77	4	2	2	3	2
Ensure access to raw materials and improve resilience	2.85	2	4	2	4	1

illustrates how the interview partners for each case rated the importance of different project objectives. In line with the overall orientation of FFG’s General Programme, which funds the development of new products, processes, and services, objectives related to technological leadership, product development, and flexibility were rated as most important across the cases. Objectives related to reducing climate impacts (highlighted in bold in Table 3) were considered less important by the companies in the analysed cases, with a slightly higher emphasis on direct (on-site) impacts than on impacts up- or downstream in the value chain. Objectives related to ensuring the supply of raw materials were considered least important.

Figure 2 shows the expected change in the technology readiness levels (TRLs) of the technical solutions developed in the analysed cases. While cases which started at low TRLs (referring to the situation at the beginning of the project) expected a considerable improvement in technological readiness by the project’s end, projects in other cases had already started at a rather advanced level and therefore expected only small changes. On average, across all cases for which data on TRLs were available, the respondents expected an increase from TRL 4 (technology validated in lab) or 5 (technology validated in a relevant environment) to TRL 7 (system prototype demonstration in an operational environment) or 8 (system complete and qualified) [44]. It should be noted that the TRLs expected by the companies at the end of their project might take into account exploitation activities that go beyond the scope of the co-funding received, since FFG’s General Programme focuses on R&I activities only in terms of the lead-up to prototype development (i.e., it does not provide funding related to launching products or services on the market).

3.2. Towards the Formulation of a Framework

This section presents the different elements that the assessment framework needs to consider based on the results from the cross-case synthesis. It is structured according to the main steps in the impact pathway, such as the progression from research to results, from results to on-site impacts, etc. The combination of these individual elements into an overall framework is described in Section 4 below.

3.2.1. From Research to Results

The funding granted by public research funding organisations such as FFG to companies usually only covers a certain part of the companies’ costs (in the case of FFG’s General Programme, only up to 50% of the costs are covered). While there is no difference in the scope of the funding compared with the overall project (since the funding is not provided solely for a certain (small) part of the project, but the whole project is co-funded to a certain degree), the assessment framework needs to distinguish between the funding and the overall project so that the (climate) impacts achieved can be allocated to those who financed the research (for example, pro-rata according to the budget share).

This issue also touches upon the question of the “additionality” of the funding provided [26], which companies need to address in their proposals by describing if and how the R&I activity would be carried out if FFG did not provide any funding. In all of the analysed cases, the companies involved stated that the R&I activity would still be carried out if FFG funding was not received; however, the project’s scope would be reduced, and it might start later and take longer to complete if no funding was granted.

Moreover, it is the nature of R&I projects to face uncertainties and risks, especially when starting at rather low TRLs. For the assessment framework, it is therefore important to differentiate between the project (which has a certain scope) and its results (which might differ from the original scope). For example, achievement of the intended project results might depend on external framework conditions that the company can only influence to a limited extent, as illustrated by the case of Lenzing summarised in

Table 4. Case example: Lenzing “Upscale to Circularity”.

Lenzing’s “Upscale to Circularity” project investigates the production of pulp from post-consumer waste textiles (in co-operation with the Swedish company Södra Cell). The current production volume (in 2022) is 2000 tonnes per year, which Lenzing aims to increase to 25,000 tonnes by 2025, eventually reaching 100,000 tonnes per year at a later stage. Contamination can hamper the process of producing the recycling pulp, so the collected textiles need to be sorted. Because automatic sorting processes are still being developed, the textiles have so far been sorted manually, a process which is prone to error. Additionally, increasing the production capacity to 25,000 tonnes per year and above requires the establishment of new logistical processes and supply chains.

. A quantification of such risks is hardly possible and was thus lacking in the analysed cases.

3.2.2. From Results to On-Site Impacts

Across the analysed cases, the main results of the funded R&I projects took the form of improved know-how, enabling the company in question to improve its processes and/or equipment. Depending on the nature of these processes and equipment, such improvements may require additional investment on the part of the company. This is also illustrated by the expected changes in TRLs shown in Figure 2, which shows that the funded R&I projects usually do not result in systems which are proven in an operational environment (TRL 9). While the dissemination and exploitation strategies for project results were generally well described in the proposals, the required investment for the actual application of the results was usually lacking in the analysed cases. For the assessment framework, it therefore made sense to differentiate between project outputs (results) and their application in terms of improved processes and equipment in order to better understand the investment needs of the companies in terms of applying the results. For example, in one of the analysed cases, the funding provided by FFG amounted to only one-thousandth of the investment that would still be required to apply the project results in the form of new processes and equipment.

The application of a company’s project results in the form of improved processes and/or equipment is at first usually limited to a single production site. Across the analysed cases, the main on-site climate impacts generally materialised in the form of reduced energy consumption due to new or more efficient production processes. Additional impacts included reductions in emissions of GHGs such as methane or a switch to alternative—usually renewable—energy sources. The use of carbon capture and utilisation (CCU) technologies was only mentioned in one of the cases (see the example of Lenzing in

Table 5. Case examples: Agrana’s “EMiCo (Energy Minimization & Color Reduction)”, Lenzing’s “New Technologies for Sustainable Growth”, and Magna Steyr’s “ProSIT (New Processes in Function Development, Simulation + Testing)” projects.

Agrana’s “EMiCo (Energy Minimization & Color Reduction)” project aims to improve the energy efficiency of a sugar factory’s production of white sugar, especially in terms of heating. This could lead to a reduction in GHG emissions of 18.4 kg CO2 eq. per tonne of sugar produced. Given that Agrana produces a total of 455,000 tonnes of sugar in Austria annually [63], this would result in a total reduction in GHG emissions of around 8000 tonnes CO2 eq. per year.

Lenzing’s “New Technologies for Sustainable Growth” project focuses on improving existing and newly developing pulp and fibre production technologies. The project aims to reduce energy consumption and CO2 emissions by improving energy efficiency, optimising the evaporation process, developing new washing concepts, applying membrane technology, and using carbon capture and utilisation (CCU) processes. Lenzing expects that such new technologies would lead to a reduction in CO2 emissions of more than 10%, compared with the status quo.

Magna Steyr’s “ProSIT (New Processes in Function Development, Simulation + Testing)” project seeks to develop innovative simulation models and methods for the design and production of electric and hybrid vehicles. A comprehensive quantification of the climate impacts is difficult, as the research mainly focuses on developing methods and algorithms, as well as adapting and validating software (for example, in terms of computing efforts). However, since the simulation also covers the production process, some impacts—such as a 10% reduction in energy consumption during the coating process due to improved adjustment of the drying nozzles—can be quantified.

). The direct climate impacts realised at a site thus consist of reductions in both direct (scope 1) GHG emissions, such as those resulting from fuel combustion or chemicals production, and indirect (scope 2) emissions resulting from the generation of the electricity purchased by the company [37]. Data on the magnitude of the expected reduction in energy consumption and/or GHG emissions were usually available from the companies; they were either included in the project proposals or provided during the interviews. Table 5 illustrates different examples of direct on-site climate impacts in the Agrana, Lenzing, and Magna cases.

The realisation of climate impacts can also depend on (external) influencing factors. Throughout the analysed cases, companies saw their competitors as the main external factor influencing their R&I activities. In contrast, the conditions imposed by the regulatory framework and commodity prices were only mentioned as important influencing factors by two companies (see

Table 6. Case examples: Montanwerke Brixlegg “Valuable Metals Pellets” and Magna Steyr “NeMo (E-Vehicle Concepts for New Mobility)”.

Montanwerke Brixlegg consider research projects like “Valuable Metals Pellets” increasingly relevant as they address the natural gas price hike, for example, by supporting the switch from fossil fuels to renewable energies.

Magna Steyr’s “NeMo (E-Vehicle Concepts for New Mobility)” project explicitly considers the existing regulations regarding the operation of autonomous vehicles in public transport as a potential development risk. For example, self-driving vehicles can currently only be operated with special permits and are not allowed to drive faster than 20 km/h.

). It is worth noting that the interviews with the companies were conducted in April 2022, when the rise in commodity prices that took place throughout 2022—especially for energy carriers—had only just begun.

3.2.3. From the Company to Dissemination in the Sector

The transformation towards a climate-neutral economy, as called for in EU strategy documents such as the European Green Deal [64] and the EU Climate Law [65], requires the widespread dissemination of low-carbon production methods. However, throughout the analysed cases, the dissemination (replication) of the improved processes and equipment at other sites belonging to the same company or even across the sector (for example, through license agreements with other companies) was usually not foreseen. Companies explicitly considered replicating improved technologies and processes at other sites as belonging to them in only two cases (see

Table 7. Case examples: Agrana “EMiCo (Energy Minimization & Color Reduction)” and Montanwerke Brixlegg “Valuable Metals Pellets”.

Both Agrana’s “EMiCo (Energy Minimization & Color Reduction)” project and Montanwekre Brixlegg’s “Valuable Metals Pellets” project aim to develop technologies and processes that could, at least in part, also be applied at other sites belonging to these companies (including those external to Austria; for example, in Slovakia).

), and one other company had plans to license the technology it developed to other companies. The cross-case synthesis thus revealed that significant direct climate impacts from R&I projects can only be expected at large-scale plants on individual sites, while a widespread scaling up of processes or technologies with only small- to medium-sized impacts is not taking place. For the assessment framework, it therefore makes sense to determine whether the (direct) climate impacts resulting from new or improved processes and equipment are expected to occur at a single site, at several sites of the company, or across several companies in the sector.

3.2.4. From Company to Upstream (Supply Chain) Impacts

Corporate R&I activities do not necessarily (only) result in direct climate impacts at a company’s site but can (also) lead to indirect impacts upstream in the company’s supply chain, thereby affecting its scope 3 GHG emissions. For example, switching from primary to secondary raw materials (for example, recycled metals or scrap) can lead to considerable emission reductions at the supply stage, as the case of AMAG illustrates (see

Table 8. Case example: AMAG “OExA (Operational Excellence Automotive)”.

AMAG’s “OExA (Operational Excellence Automotive)” project is intended to expand the use of scrap for the production of specific aluminium rolling products. Increasing the share of scrap in the production process from 50% to 80% would lead to a reduction in CO2 emissions of 91,000 tonnes per year. However, a central challenge is the security of supplies of scrap, as a large fraction is exported to countries outside of Europe, such as China. These exports lead to increases in scrap prices in Europe, contributing to the economic unviability of this resource for AMAG. The company has thus called for regulatory limits on the export of scrap outside Europe.

). Throughout the analysed cases, the companies usually had a good overview of the amount and composition of the raw materials that they purchased from their suppliers, making it feasible to approximate the magnitude of the expected climate impacts occurring upstream. However, much like direct impacts, the actual realisation of upstream indirect impacts may depend on external influencing factors such as the availability and quality of secondary raw materials.

3.2.5. From Company to Downstream (Value Chain) Impacts

Indirect climate impacts can also occur downstream in the value chain, when clients or consumers use the products manufactured by the companies or when these products are disposed of or recycled at their end-of-life stage. In the GHG protocol, the resulting GHG emissions are included under scope 3, which covers climate impacts both up- and downstream in a company’s value chain [38].

In most of the analysed cases, the first stage in the companies’ downstream value chain were the client companies (B2B) which further process or use the products manufactured by the companies. Apart from the automotive value chain, which was very prominent among the selected cases, this study also included a manufacturer of machines (Engel) which are sold to a broad variety of clients. Even though such B2B relationships usually involve close co-operation between the supplier and client companies, data that would allow an approximation of the climate impacts occurring at this stage of a company’s value chain were mostly lacking in the analysed cases. The interviews with the companies revealed that climate impacts can typically be expected from weight reductions (for example, lighter components for cars) or decreased process heat (for example, due to more efficient machines), resulting in reduced energy consumption by the companies’ clients. Although quantification of the resulting scope 3 emission reductions is not feasible, the example provided by Engel (see

Table 9. Case examples: Engel “EVOLVE (modules for machinery for fully automated production of thermoplastic fibre composites)” and “FULIO (Future Portfolio)”.

Both Engel’s “EVOLVE (modules for machinery for fully automated production of thermoplastic fibre composites)” and “FULIO (Future Portfolio)” projects aim to improve the efficiency of the machinery that the company produces. Quantification of these projects’ climate impact is challenging, since up to 90% of the impact will materialise among Engel’s clients, for example, in the form of reduced energy consumption. Detailed data on the operation of the machinery produced by Engel (such as duration and intensity of use) are only available for a small proportion of Engel’s clients, as such data usually constitute protected intellectual property (production know-how) belonging to the client.

) shows that indirect climate impacts at this stage can be substantial. External factors, such as clients’ energy costs, that might influence the climate impacts which can be achieved at this stage were rarely addressed in the analysed cases.

The cross-case synthesis revealed that by far the largest climate impacts can be expected at the consumption or product use stage. This was especially true for automotive and food industry companies. The climate impact would mainly result from changing entire systems, for example, switching mobility systems from combustion engines to electric vehicles, or switching food systems from animal-based protein to plant-based protein. However, even though macro-level data at the systems level are usually available, approximating the climate impact resulting from the cases is hardly possible due to the number of external influencing factors involved. For example, in the case of Agrana (see

Table 10. Case example: Agrana “Protein4Decarbon”.

Agrana’s “Protein4Decarbon” project investigates the extraction and further use of plant protein from by-products of starch production. It aims to produce 6000 tonnes of protein isolate per year for use in food products. A one-to-one substitution of animal protein (beef and pork) would result in a reduction in GHG emissions of around 50,000 tonnes CO2 eq. per year. However, the realisation of this reduction potential depends on changes in consumption patterns and dietary behaviour. Additionally, a widespread application of the developed technology would lead to shortages of the fodder needed to raise livestock, which would have further knock-on effects (including on the climate, when farmers switch to other carbon-intensive fodder sources). Agrana consequently highlighted that reducing climate impacts within individual companies or sectors might lead to unexpected negative side effects in other areas.

), the climate impact would be considerable if consumers switched from animal-based to plant-based protein, while it would be negligible if plant-based products were consumed in addition to animal-based products (instead of replacing them). Similarly, the climate impact of a more efficient powertrain for electric vehicles would be considerable if such improved vehicles replaced conventional combustion engine cars, while it would be negligible if it only replaced existing electric vehicles fuelled by renewable energy sources. Although the analysed cases usually included estimations of the market share that might be achieved with a new product, it was difficult for the companies to provide reliable data that could be used for an approximation of the indirect climate impacts at this stage, especially when the product is sold globally. Descriptions of climate impacts in the analysed cases usually focused on the most optimistic scenarios, with companies tending to overestimate their own role in a systemic change. This is because the analysed cases only focused on developing or improving new or existing products, which is within the respective companies’ sphere of competence and influence, while they did not address the issue of how to trigger and manage the change required at the systems level.

Indirect climate impacts in a company’s value chain might also occur at the end-of-life stage, for example, when products are reused or recycled and thus serve as secondary raw materials (thereby replacing the extraction and production of primary raw materials). However, in contrast to consumption and product use as described above, none of the analysed cases addressed the indirect climate impacts that might be caused at the end-of-life stage. The interviews with the companies indicated that while such impacts are not considered irrelevant, they tend to be beyond the scope of R&I activities, especially when the products developed are intended for use in fixed assets such as buildings and other infrastructure. Nevertheless, the results of the case analysis suggest that it makes sense for the purpose of developing an assessment framework to consider the end-of-life stage as a third part of a company’s value chain with potential indirect climate impacts (in addition to B2B and the consumption/product use stages).

3.3. Overview of Main Impact Pathways across the Cases

Table 11. Overview of cases and their most relevant aspects in terms of impact pathways. Climate impacts are highlighted in different shades of green, planned activities in yellow, and risks in red.

Case	From Research to Results	From Results to On-Site Impacts	From Company to Dissemination in the Sector	From Company to Upstream (Supply Chain) Impacts	From Company to Downstream (Value Chain) Impacts
Agrana: EMiCo		Main impact	Planned
Agrana: Protein4Decarbon		Low impact			Main impact
AMAG: OExA				Main impact
Engel: EVOLVE					Main impact
Engel: FULIO					Main impact
Lenzing: New Technologies for Sustainable Growth		Main impact
Lenzing: Upscale to Circularity	Lacking logistics and supply chains as risk for upscaling				Main impact
Magna Powertrain: etelligent drive					Main impact
Magna Steyr: ProSIT		Low impact			Main impact
Magna Steyr: NeMo		Regulation as development risk			Main impact
Montanwerke Brixlegg: Valuable Metals Pellets		Important impact	Planned	Main impact
Voestalpine: greentecFLAT		Main impact		Important impact
Voestalpine: HiPer					Main impact

complements the presentation of cases in the previous sections with an overview of the main impact pathways and the related challenges. It shows that in the majority of cases, the most important impacts of the R&I activities were expected to occur downstream in the companies’ value chains. At the same time, it is usually challenging to assess these impacts, either due to a lack of the data (for example, regarding the use of a product) needed for precise measurement or because assumptions need to be made regarding the achievable market share (and the degree of substitution for existing, more carbon-intensive products).

4. Discussion

4.1. A Framework for Assessing the Climate Impacts of Research and Innovation Programmes

The cross-case synthesis reviewed in the previous section shows that R&I projects’ pathways towards climate impacts are complex and affected (to varying degrees) by external factors. The path from an R&I activity to its eventual climate impact comprises multiple intermediate steps, each involving different challenges in terms of the assessment of the impact and its allocation to the R&I project under scrutiny [18]. Moreover, impacts can unfold at different stages of a company’s value chain, both up- and downstream [37,38,46]. As shown in the previous section, the direct climate impacts at a company’s site and the indirect impacts caused upstream at the supply stage are usually the easiest to quantify. In contrast, the further downstream an impact is expected in the value chain, the more challenging is its approximation [15,40]. At the same time, indirect climate impacts—especially downstream impacts during consumption and product use—appear to be of much higher magnitude and significance than the direct impacts occurring at a company’s site, which are only relevant for large-scale plants [45].

Due to these challenges, the purpose of the assessment framework developed in this study goes beyond a mere quantification of the climate impacts in terms of scope 1, 2, and 3 emissions reductions. Instead, its aim is to enable companies to qualitatively describe the most relevant climate impact pathway(s) of their planned R&I activity, even if not all parts of this narrative can be fully substantiated with quantitative data. The framework thus facilitates a systematic description of the different steps that lead from a (funded) R&I project to a direct and/or indirect climate impact, including the risks, challenges, and assumptions that underlie this narrative. As such, it also enables research funders and their evaluators to identify the main climate benefits of R&I project proposals submitted for funding and to compare different proposals according to a similar schema.

Integrating the elements described in Section 3.2 into the two-dimensional logic of technology maturity and value chain impacts (see Figure 1) resulted in the framework proposed in Figure 3. In the vertical dimension, the bottom part of the framework refers to the technological development of the R&I activity—that is, the funded project, its results, and how these are incorporated in new or improved processes and equipment—thereby addressing the logic of technology maturity (i.e., TRL). The two levels above this (coloured in blue and grey in Figure 3) represent the company’s value chain (both up- and downstream) and the external contextual factors that influence whether and how climate impacts might occur at these stages. The uppermost level in Figure 3 (coloured in green) refers to the nature and magnitude of the (direct and indirect) climate impacts caused by the R&I activity.

The red arrows shown in Figure 3 that connect the different elements of the proposed framework represent possible impact pathways and influencing factors. They are numbered in order to facilitate a qualitative (and quantitative, if possible) description of how the impact(s) unfold across the different steps of the framework, including descriptions of the risks, challenges and assumptions association with each step. Drawing from recently developed R&I impact assessment frameworks (see [54]), the idea behind this approach is that companies should identify the arrows that represent the main climate impact expected to result from an R&I activity when describing this impact in their project proposals, rather than trying to cover all possible pathways (irrespective of whether they are significant or not). Figure 4 shows an example of how the framework can be used to describe the pathway towards an indirect climate impact that is expected to unfold at the consumption and product use stage of a company’s value chain. In its project proposal, the company would thus only focus on the arrows numbered 1, 2, 3, 4, 11, and 12.

Some of the framework’s (theoretical) impact pathways deserve a closer examination with respect to the results of the cross-case synthesis. Firstly, as mentioned in Section 3.2.2 and illustrated in Figure 2, funded R&I projects usually result in technologies at TRL 7 (demonstration of a system prototype in an operational environment) or 8 (complete and qualified system). When applying the proposed framework at the proposal submission stage, (public) research funders should therefore request that applicants describe (and if possible, quantify) the investment necessary to eventually employ the project results in operational systems (i.e., processes and equipment; see arrow number 2 in the framework) and thus realise the expected climate impact. In the case of FFG’s General Programme, companies would thus need to describe activities beyond those for which they receive co-funding, which only covers R&I activities up the level of prototype development.

Another important finding from the cross-case synthesis is the limited, and in most cases even lacking, replication and dissemination of the developed processes and equipment, which represent an inherent conflict in most publicly funded R&I programmes. On the one hand, the (societal) return on the (public) investment—and consequently the climate impact of R&I—would be greatest if developed (new or improved) technologies were disseminated as widely as possible [66,67]. On the other hand, since corporate R&I activities are usually undertaken with the aim of increasing a firm’s competitive advantage [68,69], the companies involved have a strategic interest in protecting and limiting the dissemination of the developed technology as much as possible [66,70]. This is also reflected in the prioritisation of project objectives by the companies involved in this study, as shown in Table 3, with achieving or improving a technological advantage over competitors being the by far most important objective of participation in FFG’s General Programme. As such, throughout the analysed cases, the most important direct climate impacts were usually expected to occur at single sites belonging to the companies involved, suggesting a huge but untapped potential for multiplying these impacts at other company sites or across the entire sector.

As a consequence of the limited exploitation of the direct impacts of R&I activities, the indirect climate impacts expected up- or downstream in the value chain were of much higher relevance in most of the analysed cases (also see [15,40,45,48,49]). This appears to be in contradiction with the ranking of project objectives by the companies shown in Table 3, with the reduction in direct climate impacts being considered of higher importance by the involved companies than the reduction in indirect impacts. This discrepancy might be related to the fact that direct impacts are usually within a company’s sphere of control, while the realisation of indirect impacts is much more dependent on external contextual factors. As shown in this study, the multitude of external influencing factors also hinders or even prevents the approximation of indirect impacts downstream in a company’s value chain.

Another finding emerged from the interview with Agrana (see Table 10). While the R&I activities that it undertakes as part of FFG’s General Programme help the company to achieve its objective of climate neutrality, they might also lead to undesired negative impacts for other actors, especially when the results of the company’s R&I project are applied at a larger scale (for example, across the sector). This illustrates that focusing on reducing climate impacts within individual companies or sectors may have detrimental effects in other sectors or at the level of a whole system; this reveals a lack of an overarching systems-level perspective [45]. To avoid such negative effects, R&I policy making—like the more general level of sustainable development policy making—would need to take into account the interlinkages (synergies and trade-offs) between different impact areas and policy objectives [71,72].

4.2. Quantification of Climate Impacts in Terms of GHG Emission Reductions

Analysis of the cases included in this study revealed that a quantification of their climate impacts—with regards to scope 1 and 2 as well as scope 3 emissions—is theoretically feasible but methodologically quite challenging for two reasons. Firstly, the R&I projects analysed in this study do not necessarily lead to immediate reductions in energy consumption or GHG emissions. Instead, as stated in Section 3.2.2, they usually produce the knowledge necessary for realising such impacts later on, when processes or equipment are eventually adapted after the end of the funded R&I project. However, such changes might require considerable (additional) investments by the companies involved, and the realisation of the expected climate impacts (as described in the project proposals) thus depends on whether these investments actually happen. For example, Voestalpine currently runs several parallel R&I projects (with public co-funding) which aim to support a switch in steel production from the coal-based blast furnace route to a green electricity-based electric steel route [73]. While these projects therefore lay the groundwork for a significant decrease in the company’s GHG emissions, which would reduce Austria’s total GHG emissions by almost 5% [73], this decrease will only materialise after a considerable investment by Voestalpine in replacing its current equipment. The first major challenge thus lies in the question of how much of the expected climate impact can be attributed to the R&I project (given that the total amount of the investment needed to realise these impacts may not yet be known) and—as exemplified in the case of Voestalpine—how to avoid attributing the same impact to several different R&I activities (which would lead to double counting and thus an overestimation of the total impact) [15,49].

Secondly, a central challenge of quantifying the climate impacts of the analysed cases lies in the fact that such impacts might only occur at the use stage of the products (rather than at the company itself). This is especially relevant for manufacturers of machines (such as Engel) and other consumer products (such as cars in the case of Magna or protein in the case of Agrana). While the efficiency improvement in a single unit of the manufactured product is usually known (for example, a 30% reduction in energy consumption compared with the status quo), the eventual climate impact depends on the market share that the company’s product is able to achieve in the future. This requires knowledge (or assumptions) about (a) the number of units that will be sold and (b) how many (and which) products currently in use will be replaced by the newly developed product. For example, in the case of Agrana (see Table 10), the realisation of the expected climate impact depends on consumers switching from animal-based protein sources to plant-based protein sources. In Engel’s case (see Table 9), the actual reduction in energy consumption also depends on their machines’ use patterns at the client companies: how long the machines run for each day and which products are manufactured using them. Since these data might reveal information about a company’s competitive advantage, they are usually kept strictly confidential by Engel’s clients and are therefore not available for use in quantification of the climate impacts of funded R&I projects at Engel. Additionally, the methodological challenge of double counting climate impacts also applies here; for example, both Engel and its client companies might claim the reduction in energy consumption for themselves. Further research thus seems necessary to overcome these methodological challenges in order to accurately quantify the climate impacts that can (potentially) be achieved by R&I activities.

4.3. Implications and Further Extensions

The objective of this study was to develop a framework that facilitates the assessment of climate-related impacts arising from R&I projects funded under FFG’s General Programme. Nevertheless, throughout the discussions with the companies as well as with the thematic experts from FFG and other external experts in the course of the study, participants repeatedly referred to the question of the extent to which the developed framework would be applicable to other areas and programmes.

With regards to thematic orientation, the proposed framework is not limited to climate change per se but can be considered sufficiently general in nature to also be applicable to other impact areas such as resource use, biodiversity, or sustainable development in general. Indeed, some of the cases considered for inclusion in this study focused (as primary or secondary objectives) on biodiversity conservation efforts rather than reducing climate impacts, and the logic of the framework’s impact pathways seems equally applicable to this area. Similarly, the interviews with the companies suggest that the proposed framework can also be applied to the assessment of impacts on resource use. Several cases, such as those of Lenzing (see Table 4) and AMAG (see Table 8), revealed that changes in their suppliers (for example, as a consequence of switching from primary to secondary raw materials) would have significant impacts on material flows (which would then cause the expected upstream climate impacts). This not only suggests that material flows might be as important as climate impacts (and that the two topics are closely interlinked) but also demonstrates the applicability of the proposed framework to this area.

Additionally, while the framework was developed on the basis of a thematically open (bottom-up) funding programme, it appears to be equally applicable to a thematic programme focusing specifically on, for example, climate mitigation technologies. However, in such a case, it seems reasonable to expect proposal writers to address—and as far as possible, quantify—the full spectrum of climate impacts expected to result from an R&I activity (i.e., all of the impact pathways shown in Figure 3) rather than only focus on (qualitatively) describing the main impact pathway, as depicted in the example in Figure 4.

There is, however, a caveat for the use of the proposed framework for other areas than climate change: the issue of quantifying impacts. The analysed cases show that even in the area of climate impacts, where it is feasible to measure and aggregate all effects into one single unit (GHG emission reduction measured in CO₂ equivalent), a quantitative approximation of the impacts resulting from R&I projects is quite challenging (see [48]). The situation is much more complex in other areas such as resource use, where the measurement and aggregation of impacts using one single unit is hardly possible. For the assessment of impacts on resource use, the measurement challenges relate to both the quantity (for example, traded goods only vs. the amount of extraction required at the source) and the quality (for example, hazardous vs. non-hazardous substances) of material flows. In such cases, use of the proposed framework appears to be limited to a qualitative description of the direct and indirect impacts.

The challenges related to the quantification of impacts were also reflected in the workshops with the participating companies and other experts. While company representatives considered the proposed framework as helpful in that it permitted a more accurate description of the expected climate impacts of their R&I activities, they also raised concerns about quantifying (and aggregating) these impacts, due to the multitude of external influencing factors involved. A central concern was that funders such as FFG might use these data to calculate the climate efficiency of the funding they provide—that is, how many tonnes of GHG emissions a project is expected to avoid per euro of funding provided. The company representatives warned that using this number as an (additional) selection criterion for proposal evaluation might dilute the funding programme’s current focus on supporting technological excellence and competitiveness. The main value added of the framework can therefore be seen as its facilitation of the qualitative description of the main impact(s) to be expected from an R&I activity, while quantification of these impacts appears challenging across different sustainability areas.

In contrast to its applicability to different impact areas, the transferability of the proposed framework to other types of funding programmes appears more limited. In accordance with the nature of FFG’s General Programme, the proposed framework has a strong focus on technological developments (i.e., improvement in TRLs) that result in new or improved processes and equipment within the participating companies. Other funding programmes might have different orientations, for example, funding basic (rather than applied) research involving other actors such as universities, private research organisations, or civil society (representatives), or focusing on social (rather than technological) innovations. In such cases, the outputs of the funded R&I activities can also take on other forms, such as academic journal papers, policy recommendations, or the establishment of new (social) networks. These outputs do not directly lead to changes in the organisations involved but require uptake and implementation by other actors (that are usually external to the funding itself). For example, a research project that studies citizens’ consumption patterns might result in policy recommendations which are eventually taken up in new government regulations on waste management and recycling. Describing the impact pathway of such a project would consequently require modifying the impact assessment logic, as well as the terminology used in the proposed framework. Additionally, the development of the framework was based on cases of large—sometimes multinational—corporations. Applying it to funding programmes targeting small- and medium-sized enterprises (SMEs) might therefore require similar adaptations. For example, the question of applying the technology developed by a company at different sites belonging to that company is obviously not applicable to SMEs.

The framework’s objective of providing a pragmatic and easy-to-use orientation tool for proposal writers and evaluators also comes with certain limitations. For example, the logics of both technology maturity and value chains used in the proposed framework is necessarily simplistic. A full appreciation of the streams of the literature discussing innovation systems (as opposed to innovation chains) and value webs (instead of value chains) would have considerably increased the complexity of the framework (see [4,10]), thus reducing its usefulness and making its uptake by research funders less likely. Additionally, while circularity is of little relevance for energy carriers [74], this topic would need to be addressed more prominently when applying the framework to other areas, such as resource use. Furthermore, since the proposed application of the framework does not require a formalised quantification of climate impacts, companies may still be tempted to exaggerate the potential contribution of their R&I activity to climate change mitigation in their funding applications. Finally, the very nature of R&I means that not all (technological) objectives of a project will necessarily be achieved in the long term, as the FFG’s annual impact monitoring reports show [75]. As such, the use of the framework does not mean that the expected climate impacts described by companies in their project proposals can or will be fully realised.

A further extension of the framework along the points raised above might also serve to broaden its scope and open up the possibility to apply it to the (continuous and ex-post) assessment of climate (and other) impacts of whole R&I programmes rather than focusing on supporting ex-ante R&I proposal selection. The presented framework can be seen as a first step on the route towards a more comprehensive assessment of the societal impacts of R&I programmes, and future studies could seek to incorporate additional elements such as life cycle analysis, risk assessment, and stakeholder engagement. This would also involve considering broader environmental impacts beyond GHG emissions. Moreover, the framework would need to be integrated into existing R&I governance mechanisms to ensure continuous monitoring and feedback loops in support of R&I programme design.

5. Conclusions

The purpose of this paper was to develop a framework that facilitates—in a pragmatic and easy-to-use manner—the ex-ante assessment of the climate impacts of (publicly funded) R&I projects, using FFG’s General Programme as an example. To do so, the authors conducted a cross-case analysis of 13 R&I projects undertaken by eight companies. By combining the logics of technology readiness levels (TRLs) and carbon footprinting (scope 1, 2, and 3 GHG emissions), the authors extracted the individual steps and potential pathways describing how an R&I activity leads to a climate impact in a company’s value chain. Additionally, the interviews with representatives of the participating companies revealed the methodological challenges involved in applying the proposed framework to the evaluation of R&I programmes, especially with regards to quantifying the contribution of the provided funding to these impacts.

The framework developed in this study can be directly applied by research funding organisations such as the Austrian Research Promotion Agency (FFG) for proposal evaluation in their technology-oriented R&I programmes. Validation workshops carried out with the participating companies, the thematic experts from FFG, and external experts confirmed that the proposed framework is both reasonable and practicable for the ex-ante assessment of expected climate impacts. Although the quantification of impacts remains challenging, the framework can help companies describe the climate impacts of their R&I activities in a more focused and structured way. This, in turn, will help evaluators better assess the worth and merit of a submitted proposal in terms of its societal impacts. Such a systematic and systemic assessment of the impacts of R&I supports funding organisations by helping them better address the challenges of climate change.

In addition to supporting proposal evaluation in R&I programmes, the framework can be used by companies for their own internal climate impact assessments. For example, its participation in this study triggered one of the companies to revisit the presentation and assessment of its climate impacts (and more generally its sustainability impacts), with the aim of improving the ex-ante impact assessment of its R&I activities. The framework additionally helps companies to shed light on the contextual factors that influence their climate impacts and targets. For example, another participating company emphasised that its achievement of its own climate objectives is dependent on the availability of sufficient green (renewable) energy, including through the public power grid.

The findings of this study also contribute to improving theory and practice in the R&I programme evaluation community. Future studies could focus on testing the applicability of the proposed framework in different thematic areas or funding schemes with a view to improving the universality of the framework’s scope. The authors also invite research funding organisations to apply the framework in their technology-oriented funding programmes in order to gain practical experience of how the framework supports and can potentially change funding decisions. Follow-up studies can then investigate the extent to which the climate impacts described in project proposals actually materialise in practice and assess the most important external factors influencing these impacts. Additionally, with regards to programme design, such research could focus on the question of whether undesired system-level effects have the potential to offset the climate impacts achieved by individual projects.

The framework presented in this article can be considered a first step in making the climate impacts of R&I more explicit and thereby supporting research funding organisations in highlighting their relevance to addressing the world’s current grand societal challenges.