Objectives for Stakeholder Engagement in Global Environmental Assessments

Abstract

Global environmental assessments (GEAs) are among the most large-scale, formalized processes for synthesizing knowledge at the science–policy–society interface. The successful engagement of diverse stakeholders in GEAs is often described as a crucial mechanism for increasing their legitimacy, salience and credibility. However, the diversity of perspectives on the more precise objectives for stakeholder engagement remains largely unclear. The aims of this study are to categorize and characterize the diversity of perspectives on objectives for stakeholder engagement in GEAs; to explore differences in perspectives within and between different stakeholder groups and categories; and to test whether the more practical prioritization and selection of objectives in GEAs can be linked to deliberative policy learning as a higher-level rationale for stakeholder engagement. For these purposes, we conduct a grounded theory analysis and a keyword analysis of interview material and official GEA documents relating to two GEAs: UN Environment’s Fifth Global Environment Outlook and the Working Group III contribution to the Intergovernmental Panel on Climate Change Fifth Assessment Report. Based on the analysis, we identify six categories of objectives and present as hypotheses promising ways forward for prioritizing and characterizing objectives for stakeholder engagement in GEAs, as well as potential reasons for the differences between perspectives on objectives. This study draws attention to the need for future GEA processes to have more explicit discussions on the objectives for stakeholder engagement, as well as the importance of moving towards increasingly deliberative and inclusive assessment processes more broadly.

1. Introduction

Navigating the interface between scientific expertise, policy-making and society more broadly is a crucial concern for environmental governance. Doing so can present significant challenges in terms of legitimacy, credibility and policy-relevance, in particular when addressing wicked sustainability problems, which are transboundary, long-term and uncertain in nature, and where facts and values are inherently intermingled [1,2,3,4]. Perhaps the most prominent, elaborate and legitimate mechanism currently in place for navigating this interface is the collaborative synthesis of policy-relevant knowledge in global environmental assessments (GEAs).

GEAs are large-scale, formalized processes through which scientific and other types of knowledge is assessed and synthesized in order to inform policy-making processes at multiple scales. The major goals of GEA processes are to frame and analyze the drivers, impacts and potential solution pathways associated with global environmental problems in a legitimate, credible and salient manner, and to communicate these findings to their target audiences [1,2,5]. More than 140 GEAs have been initiated to date, covering topics such as climate change, agricultural production, ozone depletion and biodiversity loss [6]. GEAs are often mandated by intergovernmental or international bodies [1,3,7]. When well-designed, these highly interdisciplinary and collaborative processes have been shown to exert an influence on policy discourses, lead to shifts in thinking in science, and have contributed to developments in international environmental governance [1,7,8,9]. Among many other elements, GEA processes require the involvement of a large number of stakeholders who participate in a many different roles, and who often bring into the process a variety of different perspectives on the issues being assessed [10].

While there is no unified definition of the term stakeholder, this study employs the term very broadly, considering any individual involved in a GEA process in some capacity as a stakeholder, as well as non-involved target audience members (similar to other studies, for example [10,11,12]). In this sense, the term stakeholder can refer to government representatives, expert scientists from diverse disciplinary and geographic backgrounds, as well as representatives of international organizations, non-governmental organizations, business and industry, among others. Stakeholders have been engaged in GEAs dating back to their origins in the 1970s [6]. Since then, stakeholder engagement has been increasingly recognized as an integral component to the success of GEAs, as evidenced for example in the case of the Millennium Ecosystem Assessment in the early 2000s [5,9] and more recently in the Intergovernmental Platform on Biodiversity and Ecosystem Services (IPBES) [13,14,15].

Stakeholder engagement activities in GEAs have the potential to significantly strengthen environmental governance, for example by building bridges between different scientific disciplines and approaches, between political and administrative barriers, scales and boundaries, as well as between conflicting interests and value beliefs. However, recent research on GEAs has found that, in order to continue playing a valuable role in informing decision-making at multiple scales, their approaches to stakeholder engagement must be systematically adapted to the evolving environmental governance landscape [3,6,10,16]. This evolving landscape has been characterized by: (1) the proliferation of agreed environmental goals, such as the Paris Agreement and the Sustainable Development Goals, without clearly suitable policies for their achievement; (2) a related shift towards analyzing solutions, and in particular policies, in environmental assessments; and, in turn, (3) the emergence of newly active groups of stakeholders (including, for example, the direct engagement of indigenous and local knowledge holders through multiscale processes or the increasing prominence of sub-national scale environmental advocacy groups). This emergence of new actor groups has occurred in part because focusing on solutions in a legitimate manner forces assessments to explicitly address multiple conflicting interests, knowledge and value systems, geographic contexts, socioeconomic contexts and policy interdependencies, as well as issues related to uneven institutional capacity [1,6,16,17].

As the stakes and stakeholder groups relevant to solution-oriented GEA processes have diversified, so too have the perspectives on the objectives for stakeholder engagement. A first step towards adapting the approaches to stakeholder engagement in GEAs to the evolving environmental governance landscape is to establish a better understanding of the current, often underestimated diversity of objectives. Objectives can include broader or more specific rationales for engagement, are sometimes only implicit, and can be understood from different perspectives and suit different purposes and motivations (societal, group or individual). Those who mandate and produce GEA processes often represent the official, sanctioned objectives for stakeholder engagement, though they may have their own personal objectives as well. Authors and other actors who take part in GEA processes also have in mind certain objectives for their own engagement, which may or may not align with the official objectives. Less often addressed are the objectives for stakeholder engagement held by target audiences, who have comparatively little influence in a GEA process but nonetheless have expectations for engagement which can influence their use of GEA outputs and outcomes. Improving our understanding of the objectives for stakeholder engagement in GEAs is fundamentally important to facilitate systematic evaluation and improvement of engagement activities along the lines proposed by Garard and Kowarsch [10]. Characterizing the diversity can serve to challenge the predominant tendency to focus only on very few objectives, encouraging more nuanced deliberations over the relative merits of different options. Understanding the diversity of objectives can also help provide better incentives for effective engagement by more clearly and explicitly framing and communicating engagement objectives [10], as well as helping to identify divergent viewpoints (and potential conflicts) early on regarding these objectives.

The importance of understanding the diversity of objectives for engagement has been reflected for example in the literature on co-production [18] and in theories of co-creation or co-design [19], and the topic of stakeholder engagement at the global scale has been addressed in the literature as well [2,5,10,17,20,21,22]. However, the peer-reviewed literature and official documents guiding GEA practice usually analyze or propose only a limited number of objectives (if at all) in isolation, failing to do justice to the increasing diversity of perspectives on objectives and how these relate to one another [10,23]. Thus, there is a gap in the literature and GEA documents regarding an explicit and comprehensive analysis of the diversity of more precise objectives for stakeholder engagement in GEAs and the relationships between different perspectives and objectives.

In light of this, the more specific aims of this particular study are to characterize and categorize the diversity of perspectives on objectives for stakeholder engagement in GEAs, to explore differences in perspectives within and between different stakeholder groups and categories, and finally to test whether different objectives in GEAs can contribute to deliberative policy learning as a normatively significant, higher-level rationale for stakeholder engagement. Based on these analyses, we aim to suggest as hypotheses practical ways in which stakeholder engagement in GEAs can be strengthened.

The GEAs selected and the four stakeholder engagement activities analyzed are described in Section 2.1. The research methodology is presented in Section 2.2. The results are presented in Section 3, including the categories and characteristics of objectives from multiple perspectives (Section 3.1) and the results of a keyword analysis linking categories of objectives to the higher-level goal of fostering deliberative policy learning (Section 3.2). We then discuss the practical implications of the analysis and present as hypotheses promising ways forward for stakeholder engagement in GEAs, focusing on the selection of objectives (Section 4.1), the use of indicators (Section 4.2) and addressing the trade-offs between objectives (Section 4.3).

2. Materials and Methods

2.1. Cases: GEAs and Methods of Stakeholder Engagement

The first step in the study was selecting cases in which to analyze the objectives for stakeholder engagement in GEAs, which was done based on four criteria. The first criterion was that the GEA process be recurring to maximize the relevance of our findings for future iterations of the assessments. The second criterion was that the GEA strongly emphasize the exploration of solutions, important in light of the evolving governance landscape described in the Introduction. The third criterion was that the GEA process aims to address multiple aspects of environmental problems and solution options, requiring the engagement of a broader diversity of stakeholders as opposed to processes examining a more limited number of perspectives. Finally, the fourth criterion was that information on stakeholder engagement in the selected GEAs be relatively accessible to the researchers.

These criteria led to the selection of two GEAs. The first GEA is UN Environment’s Fifth Global Environment Outlook (GEO-5), published in 2012. GEO-5 placed considerable focus on exploring concrete solutions, for example conducting an appraisal of specific policies and policy clusters at the regional scale in Chapters 9–14 and analyzing tools and strategies relevant at the global scale in Chapter 17. GEO-5 also covered a very broad diversity of topics from multiple perspectives. The second GEA selected is the Working Group III contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC WGIII AR5). The IPCC in general is very comprehensive in terms of covering issues related to climate change from as many perspectives as possible, and WGIII focuses on potential solutions for mitigating climate change. The collaboration with UN Environment staff and IPCC WG III practitioners (see Acknowledgements) provided us with excellent access to both cases.

Different methods for engaging with stakeholders take place at different points during a GEA process and target different objectives. This study focuses on four prominent methods employed in GEAs to engage with stakeholders which span the entire GEA process and cover all major stages at which interactions occur: (1) the scoping meeting at the outset of the GEA to determine its outline and guiding questions; (2) the regional consultations which take place partway through a GEA and contribute to content development; (3) the review process which takes place towards the end of the process and opens up draft text to comments from a very wide diversity of actors; and (4) the Summary for Policy Makers (SPM) negotiation meeting, where the contents of the summary document are discussed and negotiated line-by-line by government representatives from around the world (

Table 1. Descriptions of four methods for stakeholder engagement in GEAs [24,25].

Method of Engagement	GEAs	Goals	Participating Groups	Additional Information
Scoping meeting	GEO-5, IPCC WGIII AR5	Determine general scope and guiding questions, as well as precise chapter outlines for the assessment.	Government representatives, assessment producers, research or academic institutions, civil society organizations, business organizations, intergovernmental organizations, and others.	3-day (GEO-5) or 5-day (IPCC WGIII AR5) meeting before assessment begins.
Regional consultations	GEO-5	Determine 3–5 priority issues by region, associated multilateral environmental agreements, and discuss promising solution options.	Government representatives, assessment producers, research or academic institutions, civil society organizations, business organizations, intergovernmental organizations, and others.	2 days of meetings in each of 6 UN world regions.
Review process	GEO-5, IPCC WGIII AR5	Rigorously review contents of assessment report prior to publication (in a process similar to peer-review).	Government representatives, assessment producers, research or academic institutions, civil society organizations, business organizations, intergovernmental organizations, and others.	Two rounds of expert and governmental review, requirement to respond to all comments.
SPM negotiation	GEO-5, IPCC WGIII AR5	Governments negotiate line-by-line the content of SPM draft (requiring consensus on final SPM) with input from authors.	Government representatives, assessment producers, selected authors, and others (as observers only).	3-day (GEO-5) or 5-day (IPCC WGIII AR5) meeting at the end of the assessment process.

Source: Own elaboration.

).

2.2. Research Approach and Methods

2.2.1. Data Collection and Grounded Theory Analysis

Grounded theory was chosen to structure the study for two main reasons. Firstly, this methodology provided a robust structure for the research design, where concurrent data collection and analysis allowed for improvement of the interview protocol over time. Secondly, grounded theory is an appropriate method for analyzing material where the goal of the exercise is not only to summarize the findings and develop categories, but also to understand the relationship between different perspectives within and between categories [26].

Collecting data on the different perspectives on objectives for stakeholder engagement in GEAs is difficult, since it is dispersed across many different sources which are not always easily accessible. First, to capture the official objectives from the perspective of the organizations producing the two selected GEAs, GEA documents were analyzed. After reviewing a number of official documents, including official mandates, meeting reports, fact sheets, and procedural guidelines, we determined that two documents in particular contained all of the objectives for stakeholder engagement in these two GEAs which could be found elsewhere. The first document is the Statement on the Objectives, Scope and Process of the fifth Global Environmental Outlook by the Global Intergovernmental and Multi-stakeholder Consultation for GEO-5 [24], which was proposed by UN Environment actors and agreed on by a wide diversity of stakeholders at the first GEO-5 consultation in March 2010. The second document is the Appendix A to the Principles Governing IPCC Work: Procedures for the Preparation, Review, Acceptance, Adoption, Approval and Publication of IPCC Reports for IPCC WGIII AR5 [25], which was agreed on by the Intergovernmental Panel members (government representatives) and most recently amended in October 2013.

To capture the perspectives of as many stakeholders as possible and go beyond the official objectives, 99 semi-structured interviews were conducted with authors and other scientific experts, with government representatives involved in one of the two GEAs, as well as with non-involved target audience members (

Table 2. Information on interviews and interviewees, organized into three different stakeholder groups.

Group of Interviewees	Description (and Examples)	Number of Interviews	Rate of Response
Authors and other involved scientific experts	Authors, producers, expert members of oversight bodies and panels	76	31.5%
GEO-5	GEO-5 High Level Panel	68	29.9%
IPCC WGIII AR5	Technical Support Unit	8	66.6%
Involved government representatives	Members of national delegations officially representing their country	13	19.4%
GEO-5	Member of Council of Permanent Representatives	5	15.6%
IPCC WGIII AR5	IPCC national focal point	8	22.9%
Target audience (not involved)	Civil society organizations, intergovernmental organizations, etc.	10	12.7%
Total		99	25.6%

Source: Own elaboration.

). Priority was given to individuals who were either involved with GEAs or whose work aligned closely with them, because relatively little is commonly known about stakeholder engagement in GEAs outside of these circles, let alone on the topic of objectives. Speaking to individuals with some experience is thus a reasonable starting point for the given research aims. Within the three broad groups of interviewees, concerted efforts were made to achieve a balance amongst interviewees along multiple criteria; however, this did not end up being the case due to a lower response rate amongst some groups. The reasons for this lower response rate are now known. It is interesting to note, though, that the lack of perfect balance amongst interviewees mirrors, to some extent, the imbalance which has been observed in practice and written about elsewhere, for example regarding diversity in geographic origins [27,28] and disciplinary background [29,30]. The relatively higher number of authors and other scientific experts involved in GEO-5 who were interviewed comes as a result of the theoretical sampling employed as part of the grounded theory approach. Through these interviews, the first conducted, it became more apparent the types of additional perspectives missing from the analysis. Thus, subsequent interview invitations were more targeted, and the additional perspectives were collected from a relatively smaller number of interviews.

All interviews were conducted in English between July 2013 and April 2015 via Skype or telephone, lasting on average 55 min. Prior to each interview, written informed consent was obtained. eighty-eight interviewees gave explicit permission for the interviews to be recorded and transcribed, and for quotes to be shared as long as no identifying information was conveyed. Interviewees who did not want to be recorded gave permission for extensive interview notes to be prepared, which were subsequently analyzed in lieu of a transcript. Due to the sensitive nature of the issues discussed, the full transcripts of interviews are not available to the public, but a list of all interviewees who consented to having their names released as well as a longer portion of transcript for all quotes employed will be made available upon reasonable request. Each interviewee was asked questions along five themes: stakeholder engagement objectives and methods; GEA impact and influence; treatment of divergent viewpoints and conflict; policy analysis methodology; and a historical overview of GEAs. More precise follow-up questions were asked in cases where the interviewee showed interest or had direct experience. On the theme of stakeholder engagement, the theme most relevant to this study, these follow-up questions included queries about personal objectives, institutional objectives, and perceptions of other actors’ objectives, as well as what aspects of different methods of stakeholder engagement were deemed to be most successful and why. The specific wording of the questions was tailored to the different groups of interviewees in order to promote more open discussion.

Concurrently with data collection, the interview material along with official GEA documents containing the official objectives for the engagement methods were iteratively coded using grounded theory analysis, following the guidance of Anselm Strauss and Juliet Corbin [26], in Max QDA. The material was coded in four iterations, first identifying segments of text related to stakeholder engagement activities at all, then identifying any mention of objectives, including personal, institutional and official objectives for engagement (either held by those involved, or claimed by observers), and different interpretations of official objectives or the objectives of other stakeholders. In the third stage of coding, the objectives were grouped into broad categories. In the fourth stage, the categories were refined through constant comparison of different perspectives on objectives, both within and between interviewee groups and categories. All categories and category names were derived exclusively from the interview material and official GEA documents analyzed, and remain grounded in the data. The results of the iterative coding analysis were visualized using the Code Relations Browser in Max QDA, a tool which displays the intersections between selected coded segments (in this case, depicting when descriptions of the different categories of objectives overlapped with descriptions of the methods of engagement).

For each of the six categories of objectives distilled from this analysis, a brief review of selected peer-reviewed literature was conducted in order to highlight additional relevant insights, drawing on the limited knowledge specifically on stakeholder engagement in GEAs, as well as on environmental management literature more broadly. This served to complement and enrich the empirical material on objectives, to contextualize the findings within a broader environmental governance framework and to strengthen the understanding of different perspectives.

2.2.2. Keyword Analysis

Next, to explore the potential contribution of different categories of objectives to the realization of deliberative policy learning, which has been identified in the literature as plausible normative rationale for engagement in GEAs [1,31], an exploratory keyword analysis was conducted. A list of keywords was developed associated with four major building blocks for deliberative policy learning based on the literature (see

Table 3. Description of four major building blocks of deliberative policy learning and root keywords.

Building Block	Description	Normative Rationale (e.g.)	Root Keywords	References
Representation	To the extent possible, represent the perspectives of diverse individuals who might be affected by process or might affect its outcomes. Ensure that representation is fair and balanced.	It is normatively right and procedurally fair to include the perspectives of those affected. It can improve the quality of decisions to involve perspectives of those who can affect the process or its outcomes.	represent*, divers*, inclu*, behalf, perspective, view*, belief, vision, account*	[1,21,32,33,34]
Empowerment	Ensure all actors are able to engage on even footing, and different perspectives are given equal weight. Break down participation barriers in particular for marginalized groups.	It is normatively right and procedurally fair to strive to overcome power imbalances, and to broaden the knowledge base contributing to the GEA. Empowerment can support trust and mutual respect between actor groups.	power, equal*, marginaliz*, strong, weak, peer, even, level	[1,21,31,35]
Building capacity	Enable effective and meaningful participation and inter-disciplinary collaboration (between different experts and non-experts alike). Consider differential capacity needs (and capacity-building mechanisms) for different actors.	It is important to strengthen the quality of debates and the ability of different actors to contribute together in the context of an inter-disciplinary and multi-scale process. This can foster mutual understanding between groups.	capacity, learn*, understand, grow*, know*, comprehen*, ability, competenc*	[1,21,36,37]
Spaces for deliberation	Provide physical space for true face-to-face (or, online) deliberation and trust-building (or, online space). True deliberation can include, e.g., backing up opinions and perspectives with rational justifications based on underlying worldviews.	It is important to build trust, increase willingness to work collaboratively, and foster learning, which can be to which deliberation can contribute. This in turn can improve understanding of different perspectives on potential solution options.	space, deliberat*, convene, bring, dialogue, place, discuss*, face*, person, location	[1,31,38,39,40]

Source: Own elaboration.

). In Max QDA, the coded material from interviews and GEA documents associated with a category of objectives was selected one category at a time. Within the text selected for each category, a keyword search was performed using the root words in Table 3. The text surrounding each statement containing both a reference to a category of objectives and a keyword was then carefully studied. This was done in order to confirm that the keyword was indeed being used in a manner associated with deliberative policy learning, and also to ensure that the statement could be interpreted to imply that the respective objective could contribute positively towards realizing the building block. For each category of objectives, the number of times a category of objectives was mentioned in conjunction with a keyword associated with deliberative policy learning was averaged in order to take into account the number of times objectives from this category were mentioned overall. This step avoids frequency bias, where the most often-mentioned category of objectives would automatically be the one that seems to contribute most strongly towards the building blocks of deliberative policy learning. Interviewees were not explicitly asked about linkages between objectives and deliberative policy learning to avoid distorting their responses and to get more spontaneous, unbiased viewpoints.

3. Results: Existing Objectives for Stakeholder Engagement

3.1. Categories of Objectives

Six categories of objectives were distilled from the Grounded Theory Analysis. These categories comprise all objectives mentioned in the interviews and in the selected GEA documents. The objectives mentioned in interviews represent the perspectives of different stakeholders, while the GEA documents represent the official objectives. Given their diverse backgrounds, the interviewees described particular objectives in very different ways. For example, different interviewees discussed their own personal objectives or motivations for engaging with a GEA process, their personal interpretations of the official objectives or the objectives of others, or the perspective of the institution, organization or government they were representing. Moreover, in both interviews and GEA documents, objectives were discussed somewhere along a spectrum between being targeted to a specific meeting and being overarching objectives transcending the stakeholder engagement activities to confer benefits on the GEA process as a whole or even on society more broadly. This often coincided with the extent to which an objective was described in a pragmatic sense, using practical and precise wording and reflecting directly on outcomes, as opposed to a more conceptual or theoretical description. These characteristics are used to describe the Results below and to highlight the different viewpoints within categories of objectives.

The category of objectives mentioned most often was the importance of providing a source of information (discussed by 86 interviewees and found in both GEA documents, for a total of 88 mentions). The second most often mentioned category was fostering a dialogue (76). These are followed in frequency by the category of objectives to improve communication and understanding (71), and the category to create a sense of ownership (67) respectively. Lastly, the categories of objectives to exert control over the process (52) and to facilitate learning (47) were mentioned the least often by interviewees and in GEA documents. These six categories are presented in the following subsections (Section 3.1.1, Section 3.1.2, Section 3.1.3, Section 3.1.4, Section 3.1.5 and Section 3.1.6, respectively), each of which includes specific examples to highlight the diversity of perspectives on objectives which underlie the categories. For each subsection, a brief overview of selected literature is included in order to bring in pertinent ideas which complement or go beyond the empirical material.

The extent to which each category of objectives was mentioned in interviews or in GEA documents in conjunction with a specific method of stakeholder engagement is shown in Figure 1, where the four lines represent the four methods detailed in Table 1. Figure 1 shows the absolute number of sources (interviews or GEA documents) which mentioned a method and category of objectives together at least once.

3.1.1. Source of Information

The most prominently mentioned category of objectives in interviews and in GEA documents alike was that stakeholders should be engaged with in order to provide a source of information to the assessment. There were two overarching ways of describing this category of objectives in interviews, which were mentioned evenly across all groups and which were sometimes both mentioned by one interviewee. The first and most commonly mentioned standpoint is pragmatic in the sense that it refers to actually writing the report, emphasizing primarily the role of stakeholders as authors and other scientific experts. One government representative involved in the IPCC stressed the need to include information from “a mix of disciplines involving social science, natural science, economics, technology, and others” to make the findings more policy-relevant. A scientific expert who participated in GEO-5 stated that “you just can’t do the assessments without access to stakeholders’ information”. One target audience member not involved in either assessment also took this perspective, explaining that you need authors with “different science and research background to present for understanding, for discussion, different interpretations of science findings”.

The other standpoint mentioned prominently within this category of objectives is that GEAs should aim to diversify the sources of information to the process beyond scientific information alone. Two interviewees involved in GEO-5 scientific expert panels described the need to engage stakeholders from different “cultural backgrounds”, and representatives of “private sector, civil society, [and] holders of traditional knowledge” for example. A government representative involved in IPCC WGIII AR5 stressed that assessments should explore a “plurality of perspectives of the problems”. A target audience member eloquently described the need to move “beyond a view that the only kind of expertise which is valid to bring in to assessments is from formal science and accredited peer-review publication towards opening up to a wider range of stakeholders and a wider range of forms of knowledge […] to contribute to the kinds of solutions that might emerge”. One interviewee with experience as an author in both GEO and IPCC assessments explained the need for diversifying sources of information “as a validation of whether solutions are actually feasible”, important in light of the shift towards solution-orientation. However, one interviewee involved in IPCC WGIII AR5 warned that “for any assessment [it would be] very, very risky to include people when in the end you rely on information that are not peer reviewed”, suggesting that “if you want to have broader stakeholder involvement then you have to set up a research project”.

In GEA documents, this category of objectives was linked directly to all four of the engagement activities described in Table 1, and in particular focused on diversifying the sources of information. For example, the IPCC WGIII AR5 selected authors and participants for the scoping meeting based on “the following criteria: Scientific, technical and socio-economic expertise, including the range of views; geographical representation; a mixture of experts with and without previous experience in IPCC; gender balance; experts with a background from relevant stakeholder and user groups, including governments” [25]. In GEO-5 regional consultations, an official objective was to engage “geographically representative and gender balanced regional groups” [24].

In the literature, this category of objectives has also often been discussed with regards to diversifying the sources of information to an assessment. Similarly to what was found in GEA documents, this includes the engagement of authors from diverse disciplines, institutional affiliations, geographic locations, and with a variety of areas of topical expertise [9,14,41]. However, stakeholders can also provide information in other roles stemming from a broader diversity of worldviews, beliefs and knowledge systems [7,42,43,44,45,46]. For example, non-scientific stakeholders could contribute information based on cultural beliefs or experiences to the problem framing and to analyzing the potential success or failure of solution options, including co-benefits and side effects [47,48].

3.1.2. Dialogue

The category of objectives to foster a dialogue was common across interviews and was the second most commonly mentioned category in the material analyzed overall, but was actually not mentioned explicitly in GEA documents. Within this category, interviewees focused on two points in particular. Firstly, some interviewees characterized this category of objectives in an overarching sense, as an objective which applies to stakeholder engagement over the whole course of the GEA process and has benefits transcending the process itself. This standpoint was mainly described by government representatives and target audience members. For example, one target audience member stated bluntly that “we lose if we do not discuss, we win if we discuss”, while another advocated taking a “more deliberative approach which explicitly acknowledges that there are alternative views on what the problem is and why it matters”. Some interviewees focusing on the overarching nature of this category of objectives described dialogue as leading to a broader, desirable outcome. For example, a government representative involved in IPCC WGIII AR5 described how dialogue processes within a GEA could help “bring new actors into the broader discussion, reset priorities, re-frame the agenda”, while a government representative from GEO-5 argued this could help “deal with lack of trust from some countries”. Another government representative for IPCC WGIII AR5 described how regional-scale dialogues in particular could “build up a regional common understanding on issues, and then move forward to a global process” while a target audience member explained that “when you have the chance to discuss […] this interaction is a positive thing towards collaboration at the regional level”.

A second standpoint within this category of objectives had to do with describing dialogue as an objective with more practical implications for reaching the targeted goals of a specific meeting or engagement activity. This perspective came up primarily during interviews with authors and other involved scientific experts. For example, a GEO-5 author saw a high value in promoting “interdisciplinary discussions to clarify the scientific findings and methodology” in regional consultations. Regarding the SPM negotiations, an expert involved in the publication of GEO-5 felt that striving to foster dialogue would help to overcome “misunderstandings between policy makers and scientists”. An author involved in the IPCC WGIII AR5 explained that in some cases which come up during the SPM negotiations “there is no scientific way of saying that this way of representing data to carry this political message is better than that way—you can say it’s marginally better but you can’t say it’s right or wrong”. In these situations, this interviewee argued, a broader dialogue must be fostered since science alone cannot justify the choice of presentation.

The main emphasis in the literature when addressing dialogues at the science–policy interface focuses on an open, back-and-forth flow of information between different actors [49]. This can often be observed in practice in GEAs [10,17,20,21], even though the official GEA documents do not explicitly state that fostering a dialogue is an objective. Similar to interviewees, some researchers have argued that dialogue can build trust and mutual understanding, and clarify representations of problems, impacts and potential solutions [9,50]. Others have emphasized in particular the importance of fostering dialogue in situations not necessarily requiring consensus as an end point [31,51]. Fostering dialogue, and ensuring that stakeholders feel they are not only listened to but have a more meaningful role in an ongoing discussion, can contribute to strengthening the legitimacy and salience of the assessment report, important criteria for success in GEAs [2,46].

3.1.3. Communication and Understanding

The third category of objectives for engaging with stakeholders in GEAs, mentioned most often by government representatives, is to improve the communication and understanding of the main messages and findings of the assessment. There were three general ways of describing this objective in the interview material, all of which have a relatively pragmatic basis. Firstly, and most commonly, this category of objectives was described with regards to how different findings were worded and presented. From this standpoint, the pragmatic end goal is to ensure that information shared with target audiences is clear and comprehensible to them. One IPCC WGIII AR5 author described “the act of translation to a policy context” as a crucial objective of the SPM negotiation, further explaining that “policy makers can help shape that language in a way that communicates the ideas better”. A GEO-5 author stated that “we do not all speak the same language, we need to translate the researchers language into the stakeholders language”. In an unrecorded interview, a government representative involved in IPCC WGIII AR5 explained that some of the technical language often used by scientists in the GEA process can be difficult to understand for many policy makers, in particular in less developed countries. However, some interviewees felt that too much emphasis on communication could be detrimental to scientific credibility, in particular when negotiating the SPM document. For example, many interviewees from all groups felt that the findings in the SPM were “watered down” as a result of the negotiation process. However, as one GEO-5 author pointed out, this is a “trade-off—by going through with that [negotiation] the report gets more visibility, but some information gets lost”.

A second line of thinking focuses on the types of outputs produced. From this standpoint, a major aspect of engaging with stakeholders with the objective of improving communication and understanding is to determine which types of products are most appropriate and useful for different audiences. Here, the practical goal is to produce these appropriate and useful products in order to increase the likelihood that they will actually be used. This perspective was mentioned most prominently by authors. For example, one GEO author suggested that “if they want to have an impact on teaching, they should provide PowerPoint slides”, which could be best designed together with experienced teachers to be used more easily in classrooms. An IPCC WGIII AR5 author described how the GEA should tailor-make “presentations in some developing countries, so that policy makers can understand better the issues”.

Finally, a third line of thinking has to do with the strategic dissemination of GEAs’ findings in order to exert an influence on global governance more broadly. This was most often mentioned by government representatives with regards to engaging with stakeholders to secure a place for the GEA in milestone global environmental governance events. For example, one government representative in GEO-5 described the role of the assessment with regards to the United Nations Conference on Sustainable Development (Rio+20), stating that “the more [GEO-5] findings are shared the more a wide spectrum of the stakeholders gain the support for consequential actions in the right directions to reach agreements at the international stage”. An unrecorded interview with an IPCC government representative involved since the first assessment report stressed the connection between the IPCC reports and the United Nations Framework Convention on Climate Change (UNFCCC) Conference of the Party meetings, a major target audience for the IPCC and a means of exerting influence.

In GEA documents, this category of objectives was mentioned most prominently with regards to the SPM negotiations. In GEO-5 this was done quite directly. For example, an important objective of the GEO-5 SPM was to effectively “communicate the findings of the GEO-5 assessment and maximize accessibility of GEO-5 information [to target audiences], including in terms of format and languages” [22] (p. 8). This is very much in line with the first standpoint focusing on the act of translating scientific findings. GEO-5 also strove to “make use of a range of multi-media and tools and innovative approaches” to reach a diverse target audience [22] (p. 8), more in line with the second standpoint on the types of outputs envisaged. In IPCC WGIII AR5 this objective was mentioned more indirectly. For example, government representatives involved in the IPCC review process and SPM negotiation must provide “integrated comments on the accuracy and completeness of the scientific and/or technical content and the overall scientific and/or technical balance of the drafts” [23] (p. 16). This would in part involve ensuring that the wording and framing are appropriate for the target audience (in this case, for governments).

In the literature, discussions about communication and understanding primarily relate to the efficient and effective transmission of information to target audiences. The literature stresses that while scientists involved in GEAs are experts in their fields, they are not necessarily experts in communication, requiring input from a broader group in order to improve understanding. This includes in particular ensuring that GEA messages fulfill a demand from end-users [5,20] and are framed appropriately [42,49]. Another crucial aspect is the role of stakeholders in interpreting information, making often complex and technical findings more usable [52], a highly pragmatic rationale akin to those which came up during interviews regarding this category of objectives.

3.1.4. Ownership

The fourth category of objectives is to build up a sense of ownership over the GEA process or its products amongst target audience members. While not mentioned in official GEA documents, interviewees who discussed ownership saw this category of objectives as very important. Interviewees often use the term ownership interchangeably with buy-in. Ownership was chosen as the heading for this category because, as some interviewees described it, ownership can be a means of encouraging stakeholders to buy into (or believe) a GEA process or products, whereas buy-in on its own does not necessarily require engagement.

There were two general ways of describing this category of objectives in the interview material, both mentioned primarily by authors and other scientific experts as well as by government representatives to some extent, and both referring primarily to building a sense of ownership amongst target audiences and in particular governmental actors. Firstly, interviewees talked about the objective to build up a sense of ownership as a unique way that GEAs in particular can increase impact. One government representative involved in IPCC WGIII AR5 described how “the degree of authority and ownership is something that other good reports, like World Bank or other similar global reports, don’t quite have”. An author involved in GEO-5 described how “people who are part of the process become ambassadors for the results, and that’s how the assessment would increase its impact”. One scientific expert involved in the production of IPCC WGIII AR5 stated that “government buy-in is the critical part of the whole process […] that’s what gives the whole thing weight and allows it to have credibility above and beyond normal scientific work”. Many interviewees linked this in particular to the SPM negotiation (see Figure 1). One scientist with many years of experience working with GEO assessments stated directly that “by having these really negotiated versions, governments feel a sense of ownership […] and do something with the report”.

The second way of describing this category of objectives was a more pragmatic perspective regarding how exactly the objective of fostering ownership amongst different stakeholders might take place in a GEA. One author involved in GEO-5 felt that this was a central concern for future GEAs, asking “how do we download it to the country level how do we get buy-in from policy makers and politicians?” One government representative involved in IPCC WGIII AR5 felt that “the larger buy-in is there when all countries can participate”. A producer involved in GEO-5 explained how particular strategies can help in building ownership, but come with trade-offs: “focusing on policy successes probably did some good in achieving some buy-in from policy makers, but maybe the trade-off was that problems most in need of policy attention were downplayed or not featured that strongly because they didn’t have success stories in them”.

In the literature, building a sense of ownership is often linked to the GEA having a pragmatic impact or influence along the lines of the first perspective above, with the assumption being that stakeholders who are involved are more likely to feel ownership and in turn are more likely to actually use the end products [2,5,44,52,53]. The most prominent example is the SPM negotiation, where the engagement of government representatives is thought to increase their sense of ownership and ultimately increase the influence of the final document on policy decisions [54].

3.1.5. Exerting Control

The fifth category of objectives involves different actors exerting some form of control over the GEA process or its products. This category of objectives was never explicitly mentioned in official GEA documents, but still came up in interviews to some extent. Very few interviewees described themselves as holding this objective (rather discussing their observation that others exerted control), and all descriptions referred to a specific instance of a stakeholder or group exerting control.

By far the most common examples given by interviewees had to do with government stakeholders exerting control over the SPM document. Both scientific experts as well as government representatives themselves discussed this standpoint. For example, one scientific expert who participated in the IPCC WGIII AR5 SPM negotiation meeting felt that governments were “trying to skew the science in their favour”. A government representative involved in that same meeting noted that many other governments “were exerting a certain influence which was based on the political priorities of their countries”. A high-level scientific expert involved in GEO-5 explained that “political influence had too much influence on the Summary for Policy Makers”. One author wrote in a public letter after the IPCC WGIII AR5 SPM negotiation meeting that “nearly all delegates in the meeting demonstrated the same perspective and approach, namely that any text that was considered inconsistent with their interests and positions in multilateral negotiations was treated as unacceptable” [55]. More broadly, one government representative involved in GEO-5 observed that there has been a push for “much more governmental control of all the sessions”, while a GEO-5 producer reflected that “you see governments wanting more and more control over the assessment process”, implying that government control may actually be increasing. As explained briefly in Section 3.1.3 above, government control is often seen as negative, for example when the final SPM becomes too “watered down” as a result of their involvement, sacrificing scientific credibility. Thus, increasing government control would not likely be welcomed by many expert scientists. However, many scientists do still see the value of relinquishing some control in particular in order to foster more ownership or buy-in. For example, one highly experienced individual who has participated as an author and producer in both GEO and IPCC assessments explained that while “the negotiation process eliminates a lot of the stuff which the scientists consider important, at least it’s some common point that has the buy-in from some policy makers”, also indicating a trade-off with the objective of building a sense of ownership.

In addition, there were a few cases where scientists exerted control. One author involved in GEO-5 described how they observed cases where “authors seemed to want to push what went into these final chapters as a function of anticipated funding needs”. Another expert with a long history of engaging with GEAs including all GEO assessments recalled numerous cases where “people [are] coming in with their pet projects and favourite subjects”, insisting these are included in the assessment. However, at least one scientific expert who has been involved in many GEA processes made the case that scientific control did not have to be a negative thing. This interviewee stated that “a lot of time you really have to sneak stuff in that you feel is a greater good”.

In the literature, control has long been linked with power relations, for example as used as leverage during debates, in particular where competing values play a strong role [56]. Control has been described as being exerted both between and within stakeholder groups. For example, some individual governments or negotiating blocks may be more powerful than others [57], or some scientific disciplines may be taken more seriously in debates [28]. In general, non-scientific stakeholders exert comparatively less control over GEA processes, at least in part due to continuing emphasis on quantitative and science-based knowledge in place of other perspectives [45,58].

3.1.6. Learning

The final category of objectives identified is learning. Learning via stakeholder engagement, though not mentioned explicitly in GEA documents, was mentioned to some extent during interviews, in particular by target audience members who had not actually been involved in GEO-5 or IPCC WGIII AR5. These interviewees spoke primarily about the high potential for learning during GEAs, how learning could or should be an objective. One target audience interviewee stated bluntly that “knowledge exchange and learning should become one of the main objectives and one of the main results of this kind of exercises”. Another described how GEAs are unique in bringing together such a diversity of actors at the global scale, stating that “people can learn a lot from each other, across different scales, different government levels, but also among government levels”. Another still focused on how learning could take place by building connections to other groups, stressing that “the focus should be on the communities that are built, you know, the social networks that are cultivated”.

Expert authors also mentioned learning as an objective of stakeholder engagement, though to a lesser extent. Scientists had a different overall perspective on learning, focusing more on learning by doing. For example, one author involved in GEO-5 explained how “working with people from different backgrounds, disciplines, countries—I think we all learned something and gained capacities through the process”. Other authors confirmed this statement and described, for example, learning to work in an interdisciplinary environment, learning how to think collectively, or learning about problems and potential solutions from entirely different perspectives. Another GEO-5 author described the value added of learning through GEAs as opposed to other channels, explaining how “in those [stakeholder] meetings, you could gather a lot of information and feelings and ideas that are not in the research papers”. A highly experienced IPCC WGIII AR5 author stressed that GEAs provided an important opportunity for learning amongst early career researchers, who often “have gone on to become scholars in their own right, writing fabulous stuff—think about what a great training ground it is”.

Government representatives also mentioned learning to some extent, but focused more on what they needed to learn, emphasizing the demand-side of this category of objectives. One government representatives involved in IPCC WGIII AR5 described how governments wanted to learn “what’s new since the last report”. A government representative involved in GEO-5 described how “we learn from China, Japan and others in the Pacific Region where they have similar problems and much better approaches”.

However, while learning was generally described positively, there were still reservations. One government representative involved in IPCC WGIII AR5 in particular described how “governments have better and better understood their respective constraints and what is at stake for each nation and where they are situated globally and the things they can do, they have learned and learned and learned”. This interviewee went on to explain that despite the clear evidence that learning has taken place over many years of IPCC reports, there was still a lack of concrete policies in place, implying that learning may simply not be sufficient on its own.

Described in the literature in a pragmatic sense, learning occurs through an uptake of relevant and comprehensible information from the products of a GEA, and can be facilitated by participation [43,49,53]. Stakeholder engagement can engender social learning and behavioral change at multiple levels and can also translate upwards to organizational learning, in GEAs and in other processes at the science-policy interface [59,60,61].

3.2. Linkages between Objectives and Broader Deliberative Goals

The keyword analysis described in Section 2.2 yielded a series of text segments from interview transcripts and GEA documents which described objectives for stakeholder engagement in GEAs while also describing how these might contribute positively towards the concepts represented by the keywords associated with deliberative policy learning. We found a total of 234 such overlaps between categories of objectives and building blocks: 93 with representation, 71 with capacity building, 59 with spaces for deliberation, and 11 with empowerment.

Table 4. The percentage of sources (interview material and GEA documents) who described a particular category of objectives and also mentioned a keyword associated with a building block of deliberative policy learning.

	Communication	Exerting Control	Learning	Source of Information	Ownership	Dialogue
Representation	0.19	0.23	0.11	0.41	0.25	0.14
Empowerment	0.00	0.04	0.00	0.02	0.06	0.04
Capacity building	0.22	0.06	0.38	0.17	0.09	0.18
Spaces for deliberation	0.12	0.10	0.06	0.17	0.10	0.28

Source: Own elaboration.

shows the number of overlaps per source (per interview or per GEA document) averaged by number of sources which mention each category of objectives overall. This is representative of the percentage of sources which have described a category of objectives which also explicitly mention a keyword associated with one of the building blocks of deliberative policy learning.

The two strongest linkages found through this analysis are between the category of objectives to foster learning outcomes and the “capacity building” building block, and between the category of objectives to provide a source of information and the “representation” building block. The strongest contribution to the building block of providing “spaces for deliberation” comes from the category of objectives to foster dialogue. The two building blocks mentioned most explicitly in official GEA documents [24,25], those of ensuring balanced representation and of contributing towards capacity building, are the two which are most strongly linked with the highest number of categories of objectives. GEA documents nearly always linked the building blocks “representation” and “capacity building” with the category of objectives to provide a source of information. Only in one case did the GEO-5 document explicitly link the category of objectives to improve communication to the building block “capacity building”.

As mentioned in Section 3.1.5, not all interviewees painted the objective of exerting control in a negative light, with many implicitly connecting this objective with the building block “representation” in the sense that the control exerted by some stakeholders (where interviewees referred primarily to relatively more powerful government representatives) could be balanced by others exerting their own control (here, interviewees referred to either relatively weaker governments or scientists). Overall, the “empowerment” building block was nearly never mentioned during interviews in connection with any of the objectives. The keyword analysis further highlights that within a single category of objectives there are many different perspectives regarding what exactly is meant by a specific objective. This has been demonstrated by the fact that different interviewees connected certain categories of objectives with certain aspects of deliberative policy learning whereas others did not. This can in turn have a strong impact on the extent to which broader goals such as deliberative policy learning can realistically be envisaged by targeting specific objectives from the different perspectives of the interviewees.

4. Discussion

Based on the diversity of (sometimes potentially incompatible) objectives described in Section 3.1 and the links between objectives and deliberative policy learning in Section 3.2, objectives for stakeholder engagement in GEAs must be evaluated and prioritized in future studies with the end goal of improving stakeholder engagement in GEAs. Going beyond the scope of this paper, this presupposes a careful analysis of the direct and indirect (desirable and undesirable) effects of implementing these objectives in GEA processes [10]. Among other things, such an evaluation will require more attention to the feasibility of implementing objectives (Section 4.1); to metrics of success associated with engagement objectives (Section 4.2); and to trade-offs between objectives (Section 4.3).

4.1. Feasibility and Selection of Objectives

One issue that connects all the findings in the Results Section is the extent to which different objectives for stakeholder engagement or combinations of objectives are actually practically feasible in a GEA. In order to avoid frustrations and unmet expectations at a later stage of the GEA process, it is very important to ensure that the selected objectives can actually be achieved given available resources and other contextual factors. An effective alignment of engagement objectives, methods and resources has been shown to be absolutely crucial [10].

A first step towards addressing this issue would be to have open and explicit discussions of objectives early in a GEA process, based on the results presented in Section 3. Such discussions could help to identify potentially over-ambitious combinations of objectives and to ensure that the objectives actually selected are clearly defined. This could paint a more accurate picture of the feasibility of the objectives for engagement, including the required number and diversity of stakeholders, given the limitless pool of potential stakeholders. For example, while the categories of objectives in Section 3 do not necessarily imply clear criteria for stakeholder selection, more specific objectives such as diversifying sources of information can have severe resource implications regarding stakeholder engagement activities. Clarifying the objectives selected and explicitly acknowledging the diversity of potential objectives can help individuals distinguish and find a balance between their own personal perspective on objectives, the perspective of the institution they represent, and the officially sanctioned objectives for engagement.

Selecting the objectives for stakeholder engagement in GEAs can be facilitated and more clearly justified by linking the objectives to a higher-level rationale. A rationale such as deliberative policy learning can be easier to agree on, and can then help to ground discussions over which objectives to prioritize and ultimately select for stakeholder engagement activities.

4.2. Indicators for Success

The development of (context-dependent) metrics of success for stakeholder engagement in GEAs, for example in the form of indicators, is crucial to strengthen evaluation and foster iterative improvement, especially with a view towards future decisions about objectives for engagement and related funding requirements. Our interview results demonstrate that such indicators can and should be developed in parallel to early discussions about objectives with input from a broad diversity of actors. The characteristics and perspectives associated with the categories of objectives in Section 3.1 already often point towards potential indicators for success, in particular when interviewees specifically discuss whether or not the objectives were achieved. This can be seen, for example, when interviewees describe the mix of participants as sufficiently (or insufficiently) diverse based on institutional affiliation or country of origin. Thus, tracking the diversity of actors based on criteria such as these could be seen as one indicator of success for the objective to provide a source of information, in particular with regards to the standpoint that the objective should serve to diversify the sources of information. Another example can be highlighted within the category of objectives to improve communication and understanding, specifically with the focus on tailoring wording and presentation to the target audience. Here, an indicator could be that, from the perspective of government representatives as the target audience, the messages are clearer and more useful after stakeholder engagement activities such as the SPM negotiation. Linking categories of objectives to higher-level rationales for stakeholder engagement, such as deliberative policy learning, can also serve to specify what types of indicators could be appropriate. For example, stating that providing a source of information is an important objective might not necessarily imply a specific indicator for success, and could be interpreted in many different ways. However, explicitly linking this category of objectives to the underlying rationale of ensuring that different viewpoints are represented in a fair and balanced manner, a building block of deliberative policy learning, points more clearly towards specific indicators. These indicators could include, for example, measures of diversity or representation, and could be measured, for example, using stakeholder analysis or mapping techniques.

There have been many examples of successful stakeholder engagement in GEAs or similar global mechanisms with regards to different criteria. For example, the Millennium Ecosystem Assessment was largely praised for its broad representation of multiple actor groups. Engagement activities during this assessment process, in multi-stakeholder meetings for example, but also in developing scenarios and in other instances, explicitly sought information from a broad diversity of knowledge systems relevant to decision-makers at multiple scales and to many different scientific disciplines [5,9]. The UNFCCC has, beginning in 2013, conducted four Structured Expert Dialogue sessions closely linked with IPCC AR5, which successfully brought together scientific experts from multiple disciplinary backgrounds and national government representatives in open discussions away from the pressures of a negotiation to foster learning [1,62]. A third example of success with regards to the indicator of high participants diversity can be found in the case of IPBES, which places more emphasis on indigenous and local knowledge than any other GEA to date, and already includes actors coming from a diversity of cultural backgrounds and with different types of knowledge in meetings such as the pre-plenary Stakeholder Days [10,14,15].

A process of determining indicators for success would ideally involve the diversity of relevant stakeholders from the outset. However, this involves practical challenges. For example, even though a large number of interviews were conducted for this paper (99), more marginalized stakeholders were not reached. The lack of marginalized or less empowered viewpoints in the analysis could be one reason that the building block of deliberative policy learning “empowerment” was nearly never mentioned in conjunction with any of the categories of objectives. The method of linking objectives to higher-level rationales, such as deliberative policy learning (Section 3.2) could be improved by adapting it to actual practice in GEAs. For instance, GEA producing organizations could widely circulate a well-designed survey to gather even more information on different perspectives (including more marginalized viewpoints) on indicators for success of stakeholder engagement.

4.3. Trade-Offs between Objectives

As mentioned, in addition to selecting objectives based on their individual characteristics, it is also important to understand how they might play out in different combinations. To this end, understanding potential trade-offs and synergies between objectives, and how these relate to the different perspectives, can be very important. Some of the different viewpoints on objectives which imply potential trade-offs have been highlighted in the Results, including different perspectives on the extent to which the sources of information should be diversified, on how much emphasis to put on communication in light of the potential trade-offs with scientific credibility, or on how desirable control is as an objective and in which cases it might be acceptable. We can also consider a hypothetical example where the objective for a particular stakeholder engagement method within a GEA is to foster dialogue, a sense of ownership or learning exclusively with regard to national governments rather than amongst a more diverse group. On the one hand, this may, under certain conditions, reduce funding needs and perhaps even reduce some tensions within GEA processes by limiting the number of potential stakeholders to engage with. However, this would create significant trade-offs with other possible engagement objectives, particularly with the ambition to ensure the inclusion in the GEA process of a diverse and balanced set of viewpoints stemming from multiple perspectives and to give diverse actor groups the chance to exert some form of control over GEA processes.

The most prominent example of trade-offs between objectives to come out of the analysis is between securing ownership amongst governments on the one hand and allowing governments some form of control on the other. Many scientists explicitly pursue governmental buy-in in order to ensure that their research and their work on a GEA have an influence on policy. In exchange for this, many are willing to relinquish some degree of control over the process and the final products. Governments often seek to exert control inter alia to maintain a say over the information which will become crucial inputs to international negotiations (for example the UNFCCC negotiations, where the IPCC SPM documents are generally taken as agreed text, as highlighted by the interviewees and in the literature [63]). Based on our interview results, it seems clear that there is an implicit deal struck at least between parts of these groups, where each accepts the position of the other to achieve their own objectives. While some interviewees merely alluded to this deal, some (and in particular highly experienced contributors to GEAs) mentioned it explicitly (Section 3.1.4 and Section 3.1.5).

This deal is one way to visualize the very important social dynamics at play between expert scientists and government representatives which occur at the very center of the GEA enterprise. The engagement of these two groups in particular is absolutely crucial to the continued production of GEAs, since after all they are often the ones who mandate, provide an audience for, and actually write the GEA products among many other roles. In light of this, it becomes highly important to consider what might happen to GEAs if this deal were to break down, as may be the case according to some interviewees and other claims that, for example, the “watering down” of the SPM has gone too far (Section 3.1.3 and Section 3.1.5) [10,55]. One way forward would be to explicitly avoid prioritizing the engagement of these two particular groups above others and to purposefully open up the entire governance structures of GEAs, providing balance to the inputs from governments and scientific authors by including a diversity of perspectives. Governance of GEAs along the lines of the Pragmatic Enlightened Model [64], already attempted in part in IPCC WGIII AR5, would even go a step further. In addition to involving a diversity of stakeholders, this model advocates for highly inclusive exploration of policy alternatives in GEAs, giving scientists the opportunity to make significant contributions to policy debates by freely exploring policy pathways, while also ensuring that the experts are not prescribing a particular policy pathway, which many governments would appreciate. This model would also in general limit the control exerted by both governments and scientists by including many more stakeholder perspectives for example when determining guiding questions or frameworks, or when evaluating different policy pathways. Such a model could effectively pave the way for an updated deal to guide GEAs and their engagement of stakeholders into the future.

5. Conclusions

This study has shed light on the large variety of perspectives on objectives for stakeholder engagement in GEAs, which are crucial tools in international environmental governance. Engaging with a diversity of stakeholders in these processes is important inter alia to create robustness and resilience in environmental governance [16,65]. Understanding the diversity of existing objectives for stakeholder engagement in GEAs can help to improve these activities. Categorizing and characterizing the objectives for stakeholder engagement in GEAs (Section 3.1) provides a crucial first step towards grounding normative discussions and clarifying the objectives for engagement in future GEAs. This is in turn is a prerequisite for justifying stakeholder engagement, especially in light of the high costs and challenges involved, and improving the methods for engagement, including selecting stakeholders based on an improved understanding of the actor constellations and social dynamics central to GEAs. Improvements to engagement activities can then take place through an evolutionary process, encouraging systematic learning from success and failures (based on objectives and explicitly taking into account trade-offs between objectives) (Section 4), and by explicitly adapting engagement to the new environmental governance contexts (Section 1). To our knowledge, this study presents the most comprehensive systematic exploration of current objectives for stakeholder engagement in GEAs to date.

Successful stakeholder engagement in GEAs can increase the effectiveness and adaptiveness of GEAs more broadly, particularly when moving towards more deliberative and inclusive assessments—going beyond the traditional objectives for stakeholder engagement focusing in particular on the dissemination of results and governmental buy-in. The Paris Agreement, for example, has made the stakes much more concrete when it comes to solutions to climate change. In light of this, the IPCC, and in particular WGIII focusing on mitigation options, must adapt its processes and procedures in order to more explicitly take into account these increasingly high and diverse stakes, and must govern its assessment processes in a more inclusive and interactive manner regarding non-state actors [16]. As argued in Section 4, this presupposes not only selecting feasible, clear and well-justified objectives, but also indicators of success and appropriate methods for engagement.

Future research may focus on: (1) the motivations and rationales, but also on the external factors underlying diverging perspectives on objectives for stakeholder engagement; (2) normative evaluation of objectives along the lines outlined in Section 4; and (3) more systematic research on specific stakeholder engagement methods and outcomes in GEAs and beyond, including a more in-depth exploration of the trade-offs between objectives as well as implications of objectives for the selection of stakeholders, for the timing of engagement, and for the required resources and procedures. In addition, understanding and strengthening the relationships between the individuals involved in GEA processes is of central importance for future research and practice.

Improving our understanding of the diversity of objectives for stakeholder engagement in GEAs can help support appropriate evaluation, can motivate engagement, and can strengthen the relationships which make GEAs function effectively and efficiently. This is a crucial point to address for future GEAs in order to adapt to the evolving environmental governance landscape.