Facilitating Change for Climate-Smart Agriculture through Science-Policy Engagement

Abstract

Climate change impacts on agriculture have become evident, and threaten the achievement of global food security. On the other hand, the agricultural sector itself is a cause of climate change, and if actions are not taken, the sector might impede the achievement of global climate goals. Science-policy engagement efforts are crucial to ensure that scientific findings from agricultural research for development inform actions of governments, private sector, non-governmental organizations (NGOs) and international development partners, accelerating progress toward global goals. However, knowledge gaps on what works limit progress. In this paper, we analyzed 34 case studies of science-policy engagement efforts, drawn from six years of agricultural research for development efforts around climate-smart agriculture by the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS). Based on lessons derived from these case studies, we critically assessed and refined the program theory of the CCAFS program, leading to a revised and improved program theory for science-policy engagement for agriculture research for development under climate change. This program theory offers a pragmatic pathway to enhance credibility, salience and legitimacy of research, which relies on engagement (participatory and demand-driven research processes), evidence (building scientific credibility while adopting an opportunistic and flexible approach) and outreach (effective communication and capacity building).

1. Introduction

The agricultural sector is at the intersection of three major challenges. Firstly, all aspects of food security (availability, access, utilization and price stability) are affected by climate change [1] and adaptation efforts are needed to achieve food security and secure rural livelihoods. Secondly, even as the increasing (largely negative) climate impacts are being felt across crop, livestock and fisheries systems [1], agricultural systems needs to produce 60% more food by 2050 compared to levels in 2005/07 [2]. The sector, which is a major employer and pathway out of poverty [3], will need to sustain an increasing number of smallholder farms, expected to rise to about 750 million by 2030 [4]. Thirdly, agriculture (and the broader food system) is in itself a major driver of climate change, contributing globally 19–29% of anthropogenic greenhouse gas (GHG) emissions [5]. To achieve the global goal of limiting temperature rise to 2 °C, which was adopted as part of the Paris Climate Agreement in 2015 [6], the sector will need to reduce emissions to the extent of ~1 GtCO2e/year by 2030 [7], as current technologies and practices can only deliver 21 to 40% of needed mitigation [7]. In addition to climate change, agriculture is also a major driver for exceeding planetary boundaries for biosphere integrity, biogeochemical flows, land system change and freshwater use [8]. The concept of climate-smart agriculture (CSA) responds to these triple challenges, by sustainably increasing productivity and enhancing achievement of food security goals, enhancing resilience, and reducing greenhouse gas emissions where possible [9]. These outcomes are addressed in a context-specific manner as their relevance will vary in different contexts [9]. Given the far reaching changes which are needed within the sector, more ambitious policy options are required [7,10]. The gap between research and implementation [11,12] will need to be bridged, and agricultural research for development (AR4D) will need to transform to enable achievement of development outcomes [13] in a rapid and effective manner, where innovations emerging from research inform policies and implementation efforts. In this context, the CGIAR (formerly the Consultative Group on International Agricultural Research), the world’s largest and most experienced research network for AR4D [14,15,16] has a major role to play to ensure that its approach to AR4D responds to the challenges and opportunities posed by climate change. The climate change program of the CGIAR, the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS), was developed to respond to these challenges and opportunities in a concerted manner [15], and invested USD 414 million from 2011–2016 on AR4D in relation to CSA. CCAFS adopted a theory of change approach to achieve development outcomes [13] and science-policy engagement efforts are at the heart of this approach [17]. In this paper, we evaluated the CCAFS program theory on science-policy engagement in relation to actual case studies of science-policy engagement leading to development outcomes, and based on the results, we propose a revised improved program theory to enhance credibility, salience and legitimacy [18]. This offers empirical insights to researchers and practitioners of science-policy engagement, and a pragmatic approach for AR4D research efforts to be more outcome-oriented in the context of climate change. Our findings also have practical implications, with CCAFS initiating a second phase, involving a proposed research investment of USD 388 million from 2017–2022 [19], to enable the program to effectively achieve its goals of helping 11 million farm households adopt CSA, assisting 9 million people out of poverty, improving the food and nutritional security of 5.5 million people, and reducing agriculture-related greenhouse gas emissions by 0.16 Gt CO2-e/year [19].

The interface between science and policy has emerged as an important aspect of research efforts in the context of global environmental change [18,20,21,22,23,24,25], and can offer valuable insights to improve science-policy engagement in the context of CSA. Science-policy engagement enriches decision-making through exchanges, co-evolution and joint construction of knowledge by interactions between researchers and policy actors [26]. Enriching decision-making involves the use of scientific knowledge in policy processes to understand the problem setting, to explore, design and implement policy responses, to inform policy evaluations, and to apply knowledge systematically in dialogue between different stakeholders [27]. Several studies have identified “best” practices for science-policy engagement, including clear and strategic communications, targeting, ensuring accessibility to research findings, timing, developing a policy acumen, participation in policy processes, knowledge brokering, and joint knowledge production [28,29,30,31,32]. However, operationalizing the best practices identified and generating outcomes is not simple [33], various challenges have been identified including trade-offs in terms of time and resources, maintaining quality, oversimplification, maintaining continuity of engagement efforts, institutional and organizational challenges, and achieving coherence between demand and supply of knowledge [34,35,36,37]. In relation to climate change, the fragmented nature of scientific knowledge of its causes, mechanisms, effects, response strategies and time horizons involved [38] and the shift in governance, from a state centric approach to an approach which also focuses on non-state actors [39,40], make science-policy engagement complex. Notable among studies on science-policy engagement is Cash et al. (2003), who set the overarching aim of science-policy engagement efforts as to maximize credibility, salience and legitimacy [18]. Credibility refers to the perceived adequacy of the knowledge produced and salience to its perceived relevance. Legitimacy refers to the extent to which knowledge production has been respectful of the divergent values and beliefs of stakeholders, unbiased in its conduct and fair in its treatment of opposing views and interests [18,41].

In this context, further areas for research on science-policy engagement emerge. Van Enst et al. (2014), have identified priorities for a research agenda in science-policy interface, which includes the identification of design principles for engagement efforts in specific contexts [36]. Sutherland et al. (2012), have also proposed elements of a research agenda for science-policy research, which includes understanding the role of scientific evidence in policy-making [42]. These priorities match those identified at the convergence of climate change, agriculture and food security. Steenwerth et al. (2014) have identified research on institutional and policy aspects, specifically the role of science-policy partnerships and science-based actions to be a key theme for research in this area [43]. However, systematic empirical assessments, to gain insights into mechanisms of co-evolution of policy-demand and scientific capability to realize development outcomes [44] are limited, and we aim to contribute to addressing this gap by offering a sectoral perspective from AR4D in relation to CSA.

We address the question: in the context of achieving climate change and food security goals, what are the success factors for science-policy engagement in AR4D in relation to CSA, to inform policies and realize development outcomes? The empirical insights which we present will be from developing country contexts, where such empirical insights into science-policy interfaces are limited [14]. We examined the program theory of the CCAFS, a key player in science-policy interface related to CSA, which invested USD 414 million from 2011–2016 on AR4D in relation to CSA, we then evaluated how the program theory has performed using case studies, inspired by literature on program theory evaluation and reconstruction [45,46,47], to propose a revised and improved program theory. The CCAFS program theory is in line with what many believe to be good practices for science-policy engagement efforts and is consistent with the literature [13,14,18,48,49,50,51]. Therefore, any insights into how the program theory works in practice, and revisions which are needed, will not only enable better design of science-policy engagement efforts, but also make an important contribution to the literature on science-policy engagement, where much of the focus has been on conceptual insights, and systematic empirical insights into what works in practice are only emerging.

2. Contextualization of Program Theory

In response to the climate change challenges faced by the agricultural sector, CCAFS adopted a theory of change approach which is grounded in the achievement of development outcomes through the provision of incentives, greater flexibility, encouraging learning and improving effectiveness of its researchers [13]. As part of the program’s approach to facilitating policy change, the program leadership, Vermeulen and Campbell (2015), drew on the literature and the program’s experiences to put forward a program theory [52] consisting of ten principles. As is highlighted in scholarly literature on the evaluation of program theories, a program theory is not a scientific theory but instead a depiction of the logic used by the program management. By evaluating this program theory, the plausibility, soundness and feasibility of the theory can be assessed, and the results can improve science-policy engagement efforts and contribute to the emerging empirical literature on science-policy engagement. Although these principles were formulated in the context of the CCAFS program, they are consistent with present day insights into conditions for successful science-policy interactions, as the added linkages to the wider literature including path breaking papers may signal (

Table 1. Principles underlying the CCAFS program theory explained.

1. Navigate toward specific points of leverage	Points of leverage are areas where a small intervention can lead to large changes [48]. Weak leverage points have limited ability to drive change [50], therefore it is essential to identify leverage points which are tangible and have the ability to drive change. In the context of complexity associated with confronting wicked problems such as climate change, this principle proposes that science-policy engagement efforts should navigate toward points of leverage, which are likely to lead to change [52].
2. Allocate resources in three thirds	This principle proposes that effective AR4D programs should invest a third of resources on research, a third on engaging with next users and a third on improving the capacity of next users for uptake of research [52]. This principle is derived from lessons learned from life cycle assessment studies [49]. This does not mean strict allocation of financial resources in thirds, but adopting an approach which puts emphasis on partnerships and capacity building, in addition to generating sound science [13].
3. Join in external processes	This principle proposes that rather than creating new processes and events, science-policy engagement efforts should join existing processes of next users wherever possible [52]. This includes boundary spanning work [53] between researchers and user groups, to define products and to foster dialogue.
4. Use research products to build scientific credibility	Enhancing credibility, i.e., scientific adequacy of technical information, is key to successful science-policy engagement [18]. Cash et al. (2003) found that in addition to credibility, salience and legitimacy are important factors, in order to respond to the needs of next users, and to ensure that the process is fair and respectful of stakeholders [18]. This principle proposes that researchers should use a strategy based on high impact publications, research and open access policies, to enhance their scientific credibility and thus support science-policy engagement processes [18].
5. Sustain co-learning throughout policy engagement and implementation	Co-learning processes facilitate knowledge exchange, coproduction and learning in the science-policy engagement process [48,50]. This principle proposes that through co-learning processes research products should be tailored and translated to suit needs of next users.
6. Tackle power and influence	Power relations, including the status of individuals involved in the engagement process may affect the outcomes of the process [41,54]. This is especially true in the case of the agricultural sector, where knowledge is highly politicized [18] and researchers need to navigate power relations. Also, in the context of power and influence, the United Nations Environment Program has called for gender equality in all science-policy activities, to avoid aggravating existing inequalities [22]. This principle proposes that researchers should be mindful of gender and other power differences [52].
7. Invest in and monitor capacity enhancement	Strengthening the capacity of farmers and agricultural sector actors such as extension services is a priority to enable farming communities to cope with climate change impacts [9]. Capacity enhancement efforts can both help next users better articulate demand, and to effectively translate knowledge into actions at the field level [14]. In this context, AR4D has a role to play, and the principle proposes that research efforts should focus on enhancing the capacity of next users and research partners and measuring progress [52].
8. Mainstream higher-level goals	AR4D efforts integrate research activities and outputs with an impact pathway leading to development outcomes, and international development partners pursue this pathway to realize impacts for higher-level goals such as improved livelihoods and food security [13]. This principle proposes mainstreaming higher-level goals of poverty reduction, gender equity, social inclusion, environmental sustainability and improved nutrition in policy engagement efforts, to help focus on development outcomes [52].
9. Create mechanisms for internal learning	Mechanisms for internal learning, such as a theory of change approach, can help balance research efforts with the priorities of next users [13]. This principle proposes that researchers should include processes to review the theory of change, re-align the strategy for impact, and seize emerging opportunities in order to be successful [52].
10. Communicate strategically and actively	Effective communication between researchers and next users is a key boundary management function [18], and the emphasis of communication efforts has shifted from generic approaches to targeted ones which facilitate knowledge brokering [51]. This principle proposes that research efforts should develop communications strategies to link closely with the impact pathways identified.

). The principles provide researchers part of the program with a pragmatic approach to realize development outcomes, and emphasize on the production of demand-driven research, effective engagement in policy processes, building scientific credibility, strategic communications and capacity building. In addition, the principles take cognizance of the political nature of policy processes and call for tackling power and influence, navigating toward leverage points, and for mainstreaming higher-level goals such as food security and poverty alleviation in science-policy engagement efforts. The principles also call for a different approach to resource allocation, investing a third of resources for engagement and communications respectively. Lesson learning is also considered to be important, and co-learning with policy-makers and internal learning by researchers, are also highlighted.

3. Methods

Over the period 2011-16, CCAFS, completed its first six-year phase, which involved a cumulative research investment of USD 414 million. In this period, the program worked in over 20 countries, at the local, subnational and national levels, and complemented in-country actions with efforts at the regional and global levels. The focus of the program was on context-specific actions, thus consistent with the interpretation of CSA [9]. Over this period, 210 case studies of science-policy engagement leading to outcomes were reported as part of program-wide reporting. Outcomes are changes in behavior, relationships, activities, or actions of non-research partners with whom a program works [55], while outcomes are important milestones in the pathway to impact, they are not measures of actual impact which are further downstream and long term in nature [56,57]. CCAFS interprets outcomes as use of research by non-research partners to develop new, or change, policies and practices. The outcome case studies reported were evaluated by the program’s management unit and independent experts representing user groups (farmers and development practitioners), and the latter experts’ scores were accorded higher weight (66%) to avoid any form of bias from internal reviewers. The case studies were evaluated across three criteria: significance, evidence availability and clarity. Significance, the criterion accorded the highest weight focused on how significant or transformative the impacts arising from the outcome are likely to be and how widely these are likely to be felt [58]. Evidence availability was the criterion accorded the second highest weight and focused on how good the evidence was for a research-attributed outcome [59]. The third criterion was clarity, which focused on how clearly the narrative [59] describes the outcome and associated activities, using qualitative and quantitative information. Based on the scoring across these three criteria, 41 case studies were rated highly. In addition to the information submitted as part of program-wide reporting, we endeavored to collect additional qualitative and quantitative data relating to these 41 case studies, through semi-structured interviews and a survey [60] structured around the program theory, together with open-ended questions to bring additional insights that may not fall under the proposed principles (e.g., what were the three most important success factors which helped achieve this outcome?). We received responses in relation to 34 case studies and these form the basis of this paper (

Table A1. List of case studies analyzed.

Case No.	Title and Description
1	National Climate Change Adaptation Strategy and Second National Communications to the United Nations Framework Convention on Climate Change (UNFCCC) (Sri Lanka) Engagement with Sri Lankan Government agencies to support the development of the National Climate Change Adaptation Strategy and the Second National Communications to UNFCCC.
2	Agriculture gets recognized in the UNFCCC Durban Agreement (Global) Engagement in UNFCCC processes to facilitate agriculture getting into the Durban Agreement.
3	Low-cost “greenhouses” for horticulture to adapt to climate change and reduce expansion into carbon-rich grasslands (Peru-subnational) Work with NGOs and subnational Government agencies to develop and scale out low-cost greenhouses as an adaptation strategy.
4	Climate-smart banana-coffee intercropping systems supported through policy (Uganda, Rwanda, Burundi) Science-policy engagement efforts in Uganda, Rwanda and Burundi to stimulate the adoption of coffee-banana intercropping systems.
5	African group of negotiators plays major role in agricultural negotiations in the eighteenth session of the Conference of the Parties (COP18) to the UNFCCC (Regional—Africa) Efforts to build capacity of the African Group of Negotiators (AGN) led to African countries making joint submissions to the UNFCCC on agriculture.
6	Findings from Commission on sustainable agriculture and climate change penetrate diverse policy forums (Global) Findings from the Commission on Sustainable Agriculture informed Mexico’s climate change law, Kenya’s agriculture act and recommendations on climate change and food security of the Committee on World Food Security.
7	10-year USD 50 million program focused on crop wild relative collection and pre-breeding for climate change adaptation established (Global) Informing a 10-year USD 50 million program focused on crop wild relative collection and pre-breeding for climate change adaptation.
8	Regional scenarios to guide policies, investments and institutional change (Regional) Use of participatory regional scenarios by policy-makers and investors in different regions.
9	Use of climate and weather data by numerous agencies and farmers (Regional-Africa) Use of research monographs on African agriculture and climate Change in West, East and Southern Africa to assist policy-makers, researchers, and NGOs.
10	Improved rainfall thresholds for index insurance (India-subnational) Supporting the efforts of the Agriculture Insurance Company of India to develop improved index-based insurance schemes for various crops that led to protection of more than 50,000 rain-fed farmers from the vagaries of rainfall in one crop season alone.
11	Linking herders to carbon markets (China-subnational) Methodologies for accounting and monitoring grassland carbon sequestration approved by the Chinese Government for domestic carbon trading markets, and by the Verified Carbon Standard for global use.
12	Beyond the climate science: CCAFS Climate data applied by thousands of non-research users around the world (Global) The CCAFS Climate portal used by NGOs, foundations, non-research international/national organizations, donors and governmental institutions to support planning and implementation efforts.
13	The Intergovernmental Panel on Climate Change (IPCC) adopts new methodology for wetlands greenhouse gas inventories (Global) Inputs into the IPCC Wetlands Supplement, which is now mandatory for all countries preparing national GHG inventories.
14	Climate change adaptation strategy adopted by Ethiopian government (Ethiopia) The Ethiopian government’s Climate Change Adaptation Strategy is informed by research outputs.
15	National adaptation policy adopted in Nicaragua and resulting investments in coffee and cocoa sector (Nicaragua) Informing the national adaptation policy in Nicaragua, which leveraged a large-scale International Fund for Agricultural Development (IFAD) investment to support implementation of the policy.
16	CCAFS informs large-scale global and national investments in food security and climate change (Global) Drawing on multiple analyses, informed the allocation of over half a billion USD of international public finance (grants and loans) to food security under climate change, via close collaboration with the agencies.
17	Cambodian climate change priorities action plan for agriculture (Cambodia) The Cambodian Climate Change Priorities Action Plan for Agriculture (USD 147 million) developed in an intensive collaboration with CCAFS over 9 months.
18	Scaling Climate-Smart Villages (India-subnational) Climate-smart villages (CSVs) scaled up by the Indian state of Maharashtra, and considered by Ministry of Panchayati Raj (local level development) in local development plans.
19	FEDEAAROZ # incorporates climate information in farm extension systems (Colombia-subnational) Research findings prompted Colombia’s rice producers’ federation (FEDEAAROZ) to incorporate climate information in farm extension systems. A decision not to plant in Cordoba—informed by seasonal forecasts and big data—prevented 1800 ha of rice crop loss (saving USD 3.5 m in input costs).
20	Inputs into the IPCC fifth assessment report (Global) Inputs into the chapter on food production and food security and summary for policy-makers, has far reaching influence on policy-makers globally, providing the evidence base for informed decision-making.
21	Shamba Shape Up * and increasing use of CSA information (Regional-East Africa) Informing content of popular TV reality show which presents scientific findings to smallholders, with average viewership of 9 million a month.
22	The International Model for Policy Analysis of Agricultural Commodities and Trade (IMPACT) used in Organisation for Economic Co-operation and Development (OECD) global and regional policy analysis (Global) Continued collaboration with OECD improves capacity to estimate and analyze climate change impacts.
23	CIAT/CCAFS science contributes to programming and implementation of about 75 million USD IFAD financing for farmers’ resilience (Uganda, Comoros, Liberia) Informing programming and implementation of about USD 75 million IFAD financing for farmers’ resilience.
24	Climate-Smart Villages scaled out in Haryana (India-subnational) In India, the State Government of Haryana launched a program to pilot 500 climate-smart villages in the rice-wheat systems districts of the state.
25	Scenario-guided policy development in eight countries (Honduras, Cambodia, Bangladesh, Tanzania, Uganda, Burkina Faso, Colombia and Ghana) Support to formulate a range of agriculture, climate and development policies and plans, in Honduras, Cambodia, Bangladesh, Tanzania, Uganda, Burkina Faso, Colombia and Ghana.
26	The impact of climate information services in Senegal (Senegal) Seasonal forecasts transmitted nationwide through 82 rural community radio stations and SMS, potentially reaching 7.4 million rural people across Senegal.
27	Agriculture is not excluded from the post-2015 UNFCCC agreement in Paris (Global) Work with policy and research partners toward ensuring that agriculture was not excluded from the post-2015 UNFCCC agreement announced in Paris in December 2015.
28	Scaling climate-smart dairy practices (Kenya) CCAFS research was used for the dissemination of climate-smart feeding and husbandry practices among 600,000 farmers who are members of six producers’ organizations.
29	Scientifically-designed index insurance protects a million Maharashtra farmers from increasing extreme rainfall events (India-subnational) Development of new region and crop specific rainfall triggers applied to provide rainfall risk cover to crops of almost one million farmers.
30	CSA Profiles in Kenya drove national/county plans, informed USD 250 million World Bank investment (Kenya) Informed the development of the USD 250 million Kenya Climate-Smart Agriculture Project.
31	330,000 farmers in Honduras and Colombia use tailored seasonal forecasts and recommendations to adapt to climate (Honduras, Colombia) Ministries of Agriculture of Honduras and Colombia are reaching-up to 330,000 farmers through nine Local Technical Agro-climatic Committees (LTACs). LTACs provide recommendations generated through local-scientific knowledge exchange using agro-climatic information to support decision-making.
32	Adoption of digital system for emergency response data collection and decision-making (Costa Rica) Support in the adoption of a data collection and analysis system to document USD 57.6 million damage of Hurricane Otto. The new system reduced response time and allowed more in-depth data analysis.
33	The Climate-Smart Village (CSV) approach inspired a World Bank-funded CSA project (Niger) The learning agenda capitalized from AR4D in Kampa Zarma CSV served to inform the design of a USD 111 million World Bank-funded project on climate-smart agriculture in Niger.
34	Scaling of Climate-Smart Villages across 38 districts of Bihar (India-subnational) CSA practices have been mainstreamed in the Government of Bihar’s investment and agricultural development plan targeting CSVs to be implemented across all 38 districts.

^# FEDEAAROZ is Colombia’s rice producers’ federation. * Shamba Shape Up is a popular television show in East Africa, which promotes agricultural practices to enable farmers to makeover their farms.

). CCAFS is a reflexive program, and the authors of this paper include the Program Director, Global Policy Engagement Manager, researchers, and science-policy interface experts from outside the program, which ensures unique insights of reflexive practitioners as well as external insights from science-policy interface experts outside the program.

The case studies [59,61] are narratives of science-policy engagement efforts undertaken by researchers part of the program, and outline the activities conducted, related research outputs, partners, next users and evidence to attribute the outcome to research efforts. The case studies include engagement with national and subnational governments, regional and international processes, development banks, investors and non-governmental organizations, who are next users of the research as opposed to the final beneficiaries (e.g., smallholder farmers and rural communities) who benefit from the impact generated by the research. The semi-structured interviews and survey [60] was conducted with 23 researchers who led science-policy engagement efforts in the case studies.. The program theory which formed the basis for the interviews and survey, revolves around ten principles for effective AR4D programs, identified by the program based on its experiences and from the literature, which is consistent with the different terms for program theories identified by Hoogerwerf (1990) [45].

Semi-structured interviews provided insights into processes of science-policy engagement adopted in the case studies, while the survey responses included ranking of the importance of the proposed principles on a five-point scale (5 = very important, 4 = important, 3 = fairly important, 2 = slightly important, 1 = not important), and provide quantitative data for statistical analysis. We analyzed the data to identify patterns of similarities across survey results and correlation among the proposed principles. In addition, we also endeavored to understand the challenges and failures faced by researchers, through an open-ended question on challenges and failures encountered in the engagement process. The results were used to critically assess the program theory [46]. Although the responses are self-reported opinions of the respondents, and this may be considered a limitation, in order to reconcile the supply and demand for knowledge [62], it is important to understand supply side perspectives and experiences to design research and engagement efforts which can deliver outcomes more effectively.

4. Results

Results from the interviews are examined below in relation to the ten principles outlined within the program theory [52]. This helps to ascertain the relevance of the principles and the context. This is followed by summaries of explanatory factors, challenges observed, and quantitative analysis based on the survey.

4.1. Navigate Toward Specific Points of Leverage

Due to the complexity involved in engaging at the intersection of climate change, agriculture and food security, navigating toward leverage points in science-policy engagement efforts was found to be an effective approach by many of the respondents. For example, researchers at the International Institute of Tropical Agriculture (IITA) engaged in East African policy processes to scale up climate-smart banana-coffee intercropping systems discovered that “The higher up you go in terms of the innovation the more you have to understand complexity and the more important it becomes to understand your leverage points. When you come with superb genetic material, sooner or later people will discover it and take it up and the more you go up in that scale, from plant, to plot, to farm, to community, to landscape to national level, the more the complexity and the leverage points become more important” (Case 4). Even in cases where this principle was not explicitly highlighted as an important factor, the role of leverage points in science-policy engagement efforts came out implicitly, as illustrated by this quote, “we tried to listen as much as possible to our next users to understand what they were really needing according to their specific characteristics and basically responding to their demands. Even though this topic is very complex we tried to focus on what was really the need of our next users and that helped us a lot to focus on our research agenda” (Case 31). The need to navigate complexity comes out quite strongly in most cases, although in some contexts this is less relevant, for example when research efforts are directed by a policy process, as illustrated in Case 1, “addressing complexity was not that important as this was directed research, we focused on water resources, agriculture and food security as leverage points as that is the focus of the International Water Management Institute”. Leverage points identified in the case studies included navigating toward higher-level goals such as food security, engaging through multi-stakeholder processes, and linkages with present day issues and concerns of policy-makers. These varied depending on the scale of efforts, for example while at the community level, the leverage point could be nutrition of the community, at the global level, it could be global food security. Leverage points also appeared to differ based on the stage within the policy cycle, for example whether at the decision or implementation stage of the policy process.

4.2. Allocate Resources in Three Thirds

In majority of the case studies, the respondents found an approach to research which integrates communications and engagement, desirable as these help build relationships with users. For example, while engaging with the African Group of Negotiators on agriculture and climate change issues, it was important to allocate resources for research, engagement and communications, as recognized by one of the respondents, “all the three issues identified were fairly critical. This happened at the time when we were building relationships with our partners, at the time CCAFS had just started. So it was very important to invest in different avenues so we were able to establish a working relationship with the different partners.” (Case 5). In several cases, the principle did not seem to be integrated at the planning phase. However, when analyzing how things happened and how resources were allocated, respondents realized that the distribution of resources had reflected this principle, as illustrated by this quote, “we did not explicitly follow this principle, but our basic approach (find out what next users’ needs and priorities are), research in relation to these priorities, and communicate with users to move towards uptake, was in effect what we did” (Case 28). While the overall approach of investing in research, engagement and communications resonated with most of the respondents, the need to adapt this to the context of each research project was highlighted by many respondents, for example, in some contexts research projects are designed solely with the aim of putting research into use as explained by one respondent, “In this case, the research had been done beforehand, and we were not doing a research activity. Resources allocated were: 50% to relationship building, 0% to research and 50% to improve the capacity to enhance uptake of science.” (Case 16). On the other hand, some respondents recognized that even if they considered the three components equally important, more resources were generally allocated to research. Others explained that at the beginning of their project, research was considered more important, and only after learning and adapting the other two components, the resources were expended at the same level.

4.3. Join in External Processes

Taking cognizance of the fact that engaging in existing processes can be more efficient than “re-inventing the wheel”, majority of the respondents were in agreement with this principle, as one respondent (Case 10) noted, “it is important to join them on processes dealing with current issues, which are already in place…” (Case 10). Building on existing processes allows to embed new interventions in the larger institutional fabric to capture more benefits. For example, in the case of informing policies and investments through participatory future scenarios, the respondent noted, “we are basically inserting ourselves into policy processes and that is the whole project” (Case 8). However, in instances where an existing process may not be present, it becomes essential to create a new process, but it is important to engage the relevant stakeholders in these new processes to ensure sufficient buy-in, as illustrated by this quote, “these were new things that were happening, so it is not that there was a process that was going on, so again with them, we started a new way of using climate information to make decisions on the ground” (Case 31).

4.4. Use Research Products to Build Scientific Credibility

While the role of scientific credibility in science-policy engagement was reaffirmed by most of the respondents, as argued by case 10—”We really need to strengthen science and build credibility. The credibility comes from past research, and takes some time to build, it relates to both the credibility of the individual as a scientist and also that of the institution”, the case studies show that in addition to credibility of research products, the credibility of research institutes, researchers themselves, and processes were important factors. The inclusiveness and participation of next users in research processes also played a role in research uptake. In case studies where this principle was not found important, we found that this was due to context-specific factors, for example where the expectations of stakeholders were not scientific outputs but business models, as illustrated in this quote, “stakeholders were most interested in business models that suit their working context. This is not an area where scientific credibility matters, but our past experience in developing successful business models served the same purpose—establishing credibility” (Case 28).

4.5. Sustain Co-Learning Throughout Policy Engagement and Implementation

Co-learning in policy engagement and implementation appeared to be more relevant in cases where participatory processes with next users were involved, for example, participatory scenario development or co-development of products, here co-learning becomes an important strategy to put research into use, as illustrated by this quote, “If you work in isolation and then come up with some output and then share the output with the next user, it is very difficult to convince them. But if you keep the stakeholders or the next users in the loop from the beginning it is really helpful to communicate your work with them” (Case 24). In these cases, respondents felt that achieving policy influence requires more than a linear approach because policy processes are complex, influenced by several divergent and sometimes competing claims and factors. Therefore, respondents recommended setting aside time for trust building among the stakeholders involved to lay the ground for consensus building and to create a window of opportunity for policy change. However, in cases where decision-making processes are closed and occur in short time frames, there are fewer opportunities for co-learning, as illustrated by one of the respondents, “over the short term there is not really this two-way interaction between the information going out on the television and people trying it out in the field” (Case 21).

4.6. Tackle Power and Influence

Tackling power and influence was a key aspect of many of the case studies, as shown in the Colombia case (19)—“Our strategy was first to learn who were the powerful players”. Having tacit knowledge about the different stakeholders, including influence, motivation and limitations of the key actors as well as information and financial flows within the organizations was an important factor to tackle power differences. Strategies for tackling power and influence were many, including stakeholder mapping, working with champions in target institutions, and combining bottom-up and top-down approaches. Identifying leverage points and using tailored approaches for the different stakeholders was also highlighted as good practices. As a respondent noted, “Power was very important. There are umpteen numbers of players in the insurance industry, therefore a stakeholder mapping was used to identify the key persons that we need to link with. This was why Maharashtra was selected, as they were Government and industry officials who could make a difference” (Case 29). At the community level, empowerment, capacity building of farmers, and communities to deal with local power imbalances was an effective way observed in two case studies.

While gender considerations were not observed in majority of the case studies, others recognized addressing gender inequalities to be very important while tackling power and influence, as one respondent explained, “as we worked on the process we realized that addressing gender issues in the dairy value chain would be essential, not only in achieving the overall effectiveness but also equitable outcomes as well, so we undertook some extra work together with private sector companies to understand how they can incorporate gender issues into their marketing and service delivery activities” (Case 28).

4.7. Invest in, and Monitor, Capacity Enhancement

Investing in capacity enhancement was found to be important in majority of the cases: “It [capacity enhancement] is very important. [...] There’s no point in talking about a particular practice if you don’t follow up when people need particular information. I think enhancing capacity was a key point in this” (Case 21). Case studies included activities to strengthen the capacity of research partners as well as those of next users, although capacity enhancement of next users was considered more important than of research partners. For next users, focus of capacity enhancement was rather directed to interpreting scientific information, while efforts to enhance capacities to conduct research were undertaken with research partners. Government organizations, non-governmental organizations (NGOs), national research institutions, farmer organizations, and private sector actors were the key beneficiaries of capacity enhancement efforts. Some case studies highlighted that capacity enhancement is not necessarily an essential component of all research for development strategies, for example when part of an international process such as the Intergovernmental Panel on Climate Change, and in another instance, when there was reverse capacity enhancement efforts, with researchers’ capacity being enhanced as a result of engaging with next users. In all cases, capacity enhancement efforts were monitored through the planning and reporting system used by CCAFS.

4.8. Mainstream Higher-Level Goals

This principle did not resonate with many of the respondents, especially when these goals were perceived to be too broad and unhelpful in achieving outcomes, for example in the case of scaling up index-based insurance in India, the respondent noted that, “The next users are more interested in day to day work and insurance related terms. They wanted to improve the scheme, which may implicitly take into account these goals” (Case 29). However, in the cases were this principle was found relevant, the three major high-level goals commonly seen were: ensuring food security, poverty reduction, and environmental sustainability. Multi-agency collaboration, alignment with government and donor priorities were among the key factors revealed by the respondents as motivations in setting high-level goals.

4.9. Create Mechanisms for Internal Learning

While in most of cases, internal learning processes helped in realigning strategic approaches and improving implementation tactics, researchers also benefited from these processes to respond to emerging opportunities. For example, in the case of scaling out climate-smart villages in Haryana, India, the respondent noted, “When we work in the diversity of partnerships, diversity of people, diversity of geographies, all those things, then we see what is important where. The same approach may not be appropriate for achieving those outcomes, so I think targeting the stakeholders is important for that outcome” (Case 24). While few case studies had explicit Theories of Change (ToC), many of the case studies had implicit ToCs, which were perceived to be more flexible and allowed researchers to be opportunistic, and save time spent in writing detailed ToCs.

4.10. Communicate Strategically and Actively

There appeared to be broad agreement on the importance of strategic communications efforts. In the only case where communications was found to be unimportant, the focus was on informing governments and international organizations in policy or investment decisions, and therefore closed processes. The approach to communications varied across case studies even when they agreed on the importance of communications, few case studies had a formal communications strategy, while others responded to demand, but agreed that if they had had a communications plan, impact would have been higher. While a formal communications strategy was absent in many cases, components of a communications strategy such as videos, mass media, social media, blogs, direct communications etc., were included by most of the case studies. Often, communications with stakeholders was done through events or other forms of direct communications. Many also used print and non-print formal and social media outlets to communicate results and increase impact. Internal communications, understood as communicating with stakeholders at all levels, e.g., farmers, government representatives, etc. is often crucial for success, and will, despite a lack of a communications strategy, often be part of the project. External communications is more varied but is seen as important for scaling up and generating wider uptake.

4.11. Explanatory Factors for Successful Science-Policy Engagement

In addition to questions related to the ten principles discussed earlier on in this section, we also asked respondents for their views on what they considered as the most important factors in their science-policy engagement efforts. This helped identify additional explanatory factors for success as well as getting further nuances on the principles. We found that, engagement of key stakeholders (including research partners and next users) was a common success factor. Designing research efforts such that they address the priorities of next users was another common success factor, together with an opportunistic and flexible approach to respond to changes in demands. Opportunism refers to the ability of projects to identify emerging needs and to engage in local, national or international processes that will facilitate the uptake of the research products or add value to them. Flexibility to readapt the project objectives to deliver solutions for meeting the identified needs and align the project to relevant processes was recognized as critical for the success of the projects. Scientific credibility, including the credibility of scientists themselves as well as the credibility of their organizations was another important success factor. Strategic communications to share research findings was identified by some of the cases as important, and the need for a wide range of communication tools targeting different audiences was recognized by most of the case studies.

4.12. Challenges in Science-Policy Engagement Efforts

We endeavored to understand the challenges and failures which researchers face while engaging at the science-policy interface. Replicability of successes was often seen as a challenge, wherein the success in one region/context cannot be replicated within another region/context, differences in policy priorities were mentioned as one of the main reasons for lack of replicability. Lack of follow up is another challenge, which leads to efforts not realizing full potential in terms of results. Another challenge that emerged was the lack of ex-post impact assessments which mean that full sense of how partners use science is often unavailable. Some of the respondents mentioned how efforts may have failed due to external factors beyond their control, such as rapid turnover of local, national and regional government staff, policy priorities etc. In addition to challenges, some of the projects referred to ‘serendipity’ as a key factor which enabled them in their efforts, i.e., being at the right place at the right time.

4.13. Quantitative Analysis and Contextualization of Principles

Across the ten principles which we assessed [52], building scientific credibility and strategic communication of research results stood out as having the highest importance on average (average of 4.5 on the 1–5 scale). Mainstreaming higher-level goals was ranked as having the lowest importance on average (average 3.3), with 32% of case studies reporting this as having little to no importance (score of 1 or 2). However, we found differences in the rankings across scales (Figure 1), for example at the local and national scale, the principle of understanding power differences was rated as significantly more important than in the case studies at the global scale (p < 0.001 and p < 0.01; Tukey’s Honest Significant Difference). Similarly, there were also scale differences as to how other principles were rated. For example, case studies at the global and local scale rated scientific credibility as significantly more important than those at the national/regional scales. Some of the principles were correlated—particularly navigating complexity and understanding power differences (r = 0.48), and co-learning and capacity building (r = 0.45)—showing these principles are explaining much of the same variance.

The analysis of the case studies reveal that on an average 11% of the case studies find one or more principle either slightly important or not important, indicating different behaviors have played roles in facilitating outcomes. Among all the principles “Communications” appears to be the most consistently important across all the outcomes. The principles mainstreaming higher-level goals (32%), sustained co-learning (21%) and internal learning (21%) are the ones with highest percentage of projects ranking them as slightly or not important. Consequently, there is no strict rule on applicability and importance of these principles highlighting the contextualization of them depending on the nature of the case study, type of the stakeholders and most importantly, how efforts are operationalized in achieving successful outcomes.

5. Discussion

We used the ten principles for effective AR4D proposed by Vermeulen and Campbell (2015), the program theory of the CCAFS program, as the basis for our analysis, while rooting these principles in relevant existing literature. In general, there is overlap between the principles, which became evident in the case studies. For instance, while the fifth principle proposes sustaining co-learning, this is also reflected in capacity enhancement efforts (principle 7). Similarly, while principle ten proposes a strategic communications approach, communications activities are taking place in relation to many of the other principles. A demand-driven research approach will incorporate multiple principles including those relating to co-learning (principle 5), internal learning (principle 9) and capacity enhancement (principle 7). Consequently, there is no strict boundary between these principles, and what is important is how these are operationalized in the context of each case study. However, upon empirically scrutinizing the principles, their relevance for enhancing credibility, salience and legitimacy [18] in AR4D was confirmed, and the results offer an opportunity to nuance the principles, synthesize some of them and to specify how they work in the context of enhancing credibility, salience and legitimacy. To do this, the principle which calls for allocation of resources in thirds (principle 2) provides a useful framing, as it calls for redistribution of research effort into three broad areas: (1) engagement with partners and stakeholders; (2) developing evidence; and (3) outreach through communication and capacity building. The three-thirds principle draws on lessons in life cycle analysis [49], and offers an approach to operationalize the concepts of credibility, salience and legitimacy identified by Cash et al. (2003) for AR4D under climate change, drawing upon empirical insights from the case studies analyzed [18].

Table 2. Comparison of the three-thirds principle against the key components of knowledge systems for sustainable development [18].

	Credibility	Salience	Legitimacy
Evidence	- Generation of scientifically adequate technical evidence and arguments. - Building credibility of researchers through high quality publications. - Building credibility through complementary methods (knowledge products, participation in processes etc.).	- Flexible and opportunistic approach to ensure results are tailored to the needs of next users.	- Unbiased and rigorous research outputs.
Outreach	- Communicating research results actively to build credibility among decision makers.	- Enhancing capacity of decision makers for uptake of research results. - Communicating research in formats that can be understood and used by decision makers.	- Two-way communications to incorporate diverse views.
Engagement	- Building credibility of researchers through active participation in scientific and policy processes.	- Demand-driven research which address the knowledge needs of decision makers.	- Participatory approaches to enhance legitimacy by taking into account divergent values and beliefs.

compares the three components of knowledge systems for sustainable development [18] against the three-thirds principle, which identifies opportunities for actions to improve science-policy engagement.

To enrich this framing with empirical insights drawn from this study, we elaborate the three areas further as explained below.

5.1. Engagement

While allocating a third of research effort to engagement efforts, two key components emerge for effective operationalization, notably: (i) participatory approach and (ii) targeted and demand-driven approach.

5.1.1. Participatory Approach

Participatory approaches were observed in several case studies, e.g., through interaction with government officials and researchers, through joint scenarios development processes, coproduction of knowledge products etc. Coproduction of knowledge with next users has been identified as an effective strategy for science-policy engagement [18], and Hegger et al. (2012) has proposed a set of success conditions for joint knowledge production including the creation of a protected space for knowledge development and allocation of appropriate resources [41,63]. The case studies show that relying on participatory approaches can enable researchers to effectively engage with next users for open dialogue, mutual learning, and consensus decisions. However, effective science-policy engagement strategies rely on developing an understanding of the expectations and interests of the stakeholders, and protected spaces for joint knowledge production and appropriate resources to support such spaces should be a priority. Coproduction of knowledge can also stimulate ownership, and make room for tension to emerge (e.g., certain issues discussed can be controversial or provocative, there may be unexpected dynamics or competition between participants).

5.1.2. Targeted and Demand-Driven Approach

Reconciling the supply for knowledge to meet the demand emerging from stakeholders is key [62]. To operationalize this, the case studies showed a targeted and demand-driven approach by researchers to be crucial. Almost all the case studies have taken a demand-driven approach, which helps ensure that knowledge produced is salient to next users. The trend toward demand-driven research has focused attention on the inclusion of users (e.g., farmers, policy-makers) in research planning. Theoretically, this should enhance ownership and increase the applicability of research. However, in practice, several tensions emerge regarding the operationalization of such ‘user-driven research planning systems’ (e.g., information asymmetries between the actor groups which can influence their capacity to successfully act in the research planning system etc.). A demand-driven approach can help ensure that research is salient (i.e., relevant to the needs of next users) [18].

5.2. Evidence

Effective science-policy outcomes are underpinned by credible evidence, and key components for generating credible evidence include (i) scientific credibility, and (ii) opportunism and flexibility.

5.2.1. Scientific Credibility

As observed in the results, a vast majority of the respondents found this to be an important factor in science-policy engagement efforts, and this was highlighted in detail in several case studies. Scientific credibility enabled researchers to get involved in processes led by private sector and Government agencies, and thus inform their decisions. For scientific information to be accepted by end-users (e.g., policy-makers), it must be credible, and it is important for researchers to think about how accurate and credible the information being produced is, to be useful. While Cash et al. (2003) interprets credibility as the adequacy of the technical evidence and arguments [18], we found that beyond the technical evidence and arguments, the credibility of the individuals and institutions also play an important role, i.e., in addition to the credibility of knowledge produced, the credibility of knowledge producers and knowledge producing institutions are also important factors in science-policy engagement. As Heink et al. (2015) points out, it is necessary to specify the concept of credibility for specific contexts [64], and we find that this needs to be broadened to cover the credibility of institutions and individuals.

5.2.2. Opportunism and Flexibility

Researchers require the flexibility to pursue opportunities as part of their science-policy engagement efforts. Many of the case studies’ success depended on responding to opportunities as and when they appeared, for example by responding to an opportunity rather than as part of a planned impact pathway. Therefore, a key aspect of science-policy engagement efforts is to maintain flexibility and to be opportunistic, which enable the production of knowledge salient to user needs. Dealing with policy partners involves different timeframes to research project timelines and researchers need to be flexible to these [65]. Therefore, it is essential for researchers to seize opportunities when they arise. Participatory approaches can be useful for researchers to engage with next users and adopt approaches which adapt to their needs [66]. In the context of reconciling the supply of and demand for science, Sarewitz and Pielke (2007) have highlighted the concept of missed opportunities, wherein opportunities to connect science with policy are missed when research agendas may not meet next user demands or when social or institutional constraints prevent information use [62], and a flexible and opportunistic approach can be a solution in some of these contexts as observed from the case studies.

5.3. Outreach

Reaching out to next users, including through strategic communications as well as through capacity building efforts is the third area to focus on, and it includes (i) communications and (ii) capacity building.

5.3.1. Communications

Most of the case studies noted communications to be of importance in science-policy engagement efforts, and communications activities included formal as well as informal approaches. This takes cognizance of the growing transition of traditional science communications to knowledge brokering characterized by targeted approaches to inform different stakeholder groups [51]. Communication between researchers and policy-makers is not easy, and difficulties associated have been highlighted by various authors (e.g., Guston 2001, McNie 2007, Holmes and Clark 2008) [21,67,68]. Cash et al. (2003) has also noted the importance of active, iterative and inclusive communications with decision makers [18]. Researchers and research managers need to take cognizance of the importance of communications and incorporate it as part of their research strategy. Effective communications underpins most of the case studies which we analyzed. Communicating (complex) research findings to non-scientists is challenging, but necessary, because next users need to understand, accept, and use the research outcomes/findings. Despite the importance of communicating effectively with next users, we must be aware that communication is not simply a one-way transmission of information from the scientist/research to intended audience; but rather, it is an iterative, engaged process in which both the science and the stakeholders benefit from exchanges of information.

5.3.2. Capacity Building

In AR4D, capacity building is crucial to articulate the demands of next users and to convey scientific findings to non-specialists [14,69]. In the case studies which we examined, this role of capacity building was evident. This crucial role of capacity building is taken for granted in other sectors [14], but in the rural development context, it is an area which needs explicit focus. The case studies show that capacity building should be part of the overall outreach efforts, complemented with the communications strategy for results. Capacity building efforts can also enable active engagement of stakeholders and provide greater legitimacy for knowledge generated.

A Revised and Improved Program Theory

Based on the framing provided by the three thirds principle, and the components identified within the three focal areas, we propose a revised and improved program theory, which can be applied by the CCAFS program and more broadly for AR4D under climate change. This program theory for effective science-policy engagement is captured in Figure 2, and proposes allocating research effort in thirds. The first third of research effort should focus on engaging early and throughout with key partners and stakeholders within the impact pathway. Researchers need to join in key processes, actively participate and ultimately identify ways to navigate the stakeholder networks and institutional dynamics. These engagements should help make the research demand driven and ideally the research products are co-designed with next users. The second third of effort should go toward developing scientifically credible evidence. The research to develop evidence should allow for opportunism and flexibility to take advantage of quickly emerging needs along the impact pathway. This has traditionally been where the bulk of research efforts have focused, but it needs to be balanced with engagement and outreach efforts. The final third of the research effort should go toward outreach. This includes communicating research outcomes in formats that can be understood and used by the next users, and capacity building to enable use of research outputs and to ensure sustainability of the outcomes generated.

Encountering challenges and difficulties is bound to happen in every research project, and the importance of learning from these challenges is crucial for success. However, we have not included a separate component for lessons learned, as it should be part of any monitoring, evaluation and learning process. From the analysis, it was clear that most researchers are aware of how important planning is, but many also referred to the element of serendipity in reaching outcomes. The element of serendipity cannot be planned or measured but will be an intrinsic element of success. Social, cultural and historical contexts also affect the success of science-policy engagement efforts, and it is not possible to have a uniform approach across contexts, and science-policy engagement should be tailored to the context.

6. Concluding Remarks

In many areas of research, there is a major gap between science and action, or, more narrowly, between science and policy; in health, public policy, conservation and agriculture—a phenomenon long recognized but entrenched (e.g., [11,21,70,71]. In the context of sustainable development, enhancing credibility, legitimacy and salience through science-policy engagement has been identified as a priority to overcome this gap [18]. To meet this priority and generate tangible outcomes, within the agriculture sector, we found that an approach which relies on engaging stakeholders to demand and co-develop knowledge (i.e., engagement), generation of scientifically credible evidence in an opportunistic and flexible manner to be salient (i.e., evidence), and communicated in appropriate formats together with capacity building efforts to raise capacity for implementation (i.e., outreach) can accelerate progress toward global goals for climate action and food security. However, while such an approach may offer a promising new way to achieve development outcomes, the large-scale adoption of such an approach will depend on the existence of incentives for researchers. Current systems for measuring scientific quality limits researchers’ engagement in processes that generate societal impact [72]. New ways of measuring scientific performance, including measuring actual societal outcomes, as practiced by the program from which this data set has been drawn could help make a shift. Such a shift is inevitable if we are to overcome mega-challenges of achieving food security, adapting to climate change and mitigating emissions from agriculture. Data from such outcomes can also be used to learn lessons, as lessons learning is crucial, as we attempt to improve in the face of these mega-challenges.