Comparing Approaches for the Integration of Stakeholder Perspectives in Environmental Decision Making

Abstract

Including stakeholder perspectives in environmental decision making is in many countries a legal requirement and is widely seen as beneficial as it can help increase decision legitimacy, likelihood of implementation, and quality of the outcome. Whereas the theoretical literature on stakeholder engagement is large, less attention has been devoted to comparing and discussing different methodological approaches. Here, we compare three approaches—multi-criteria analysis, plural rationality theory, and scenario construction—that include stakeholders’ perspectives in environmental decision making. We find differences between the approaches concerning the assumptions about stakeholder rationality and whether experts and/or stakeholders are in charge of framing the problem. Further differences concern the type of data input from stakeholders and how it is used by the experts, as well as the role of stakeholders and whether they are involved early—already for identifying options—or later in the process, for evaluating or ranking alternatives analyzed by the experts. The choice of approach thus predetermines the type and depth of stakeholder engagement. No approach is “better” than another, but they are suited for different problems and research aims: the choice of the approach, however, has a large impact on the results.

1. Introduction

There is an increasing recognition that it is important to integrate stakeholders’ perspectives in the development of alternatives, technical-policy options, scenarios or models used as support for environmental decision making. The inclusion of their perspectives in the decision-making process may lead to the identification of new solutions; improve the legitimacy of decisions; and increase the likelihood that policy implementation will be more effective, efficient, and sustainable [1,2,3,4,5,6,7,8,9,10,11,12]. Examples can be found in the sectors of disaster risk management, urban planning, water management, waste management repositories, and infrastructural projects [4,8,9].

However, representing and translating the heterogeneity of stakeholders’ perspectives and dealing constructively with value-based issues in decision making is far from simple. Problems may derive from the representativeness of the participants, the selection bias (concerning who is willing and able to participate and how stakeholders are chosen), the resource and power imbalances between actors, the time and resources needed from researchers or mediators to understand and deal with fairness issues, the difficulties in finding impartial facilitators, etc. As a result, the inclusion of stakeholders may have the adverse effects to slow decision making, or to facilitate preferred solutions of already powerful actors rather than to foster democratic decision making processes [13,14,15,16,17].

The integration of stakeholder perspectives and knowledge in the tools, methods, or models used as support to decision making [18] may be also problematic. This integration is the starting point of knowledge co-production processes which shape how stakeholders’ perspectives are taken into account and translated into inputs for decision making [19]. Even if a growing number of decision support tools and frameworks have been developed to co-produce knowledge between stakeholders and experts in environmental decision-making, there are still several open issues. While there is a large theoretical literature on knowledge co-production [2,20,21,22,23,24,25,26], much less attention has been devoted to comparing and contrasting approaches and methodologies.

In this paper, we focus on the modes, approaches, and methodologies to co-produce knowledge, with a specific focus on the integration of stakeholder perspectives and their “translation” into options for solving environmental problems. The aim is to foster the debate on the selection of the most appropriate approach and to reveal some critical choices that a decision maker or analyst should be aware of. We select three approaches—multi-criteria, plural rationality, and scenario based—describe their aims and key phases, and provide some examples of practical applications. Afterwards we describe the key differences between approaches concerning assumptions about the stakeholders’ rationality and the result/most desirable outcome; the problem and solution framing; the experts’ and stakeholders’ tasks in knowledge co-production; and qualitative/quantitative data analysis. We conclude with some reflections about the consequences of methodological choices on the process design and its outcome.

2. Inclusion of Stakeholders’ Perspectives and Knowledge Co-Production in Environmental Decision Making

An academic focus on understanding environmental decision making and classifying approaches has existed for a long time. For example Gregory and colleagues [27] distinguish between science-based, consensus-based, economics/multi-criteria-based, and structured decision-making approaches. Science-based approaches highlight that factual information and science will never, by itself, make a decision. Yet, this information will shape values, which do determine choices. Consensus-based processes instead focus on the endpoint of bringing a group to a consensus agreement. Economics- and multi-criteria analysis-based approaches focus on monetization techniques and emphasize the analysis of a preferred solution. Finally, structured decision making involves six steps from clarification of the decision context to definition of objectives and performance measures, alternatives development, consequence estimation, trade-off evaluation and selection, and implementation.

However, in this and other classifications, little attention is devoted to the inclusion of stakeholders’ perspectives in decision-making and to the related knowledge co-production processes. Yet, there are exceptions. Mielke and colleagues [17] develop four ideal types for integration of stakeholder perspectives: the technocratic, the neoliberal, the functionalist, and the democratic one. Their theoretical analysis reveals that stakeholders’ main role may be to: (i) provide issue-specific and objective information (technocratic type); (ii) carry out the battle of power and authority (neoliberal type); (iii) trigger learning processes that can make science more sensitive for societal problems (functionalist type); (iv) generate (new) knowledge on the basis of their scientific as well as local expertise (democratic type) [17]. This and other similar typologies focused on theoretical aspects (see e.g., [28]) certainly help to better understand the role of stakeholders in decision-making processes, but do not address methodological issues.

With a focus on methods to integrate stakeholders’ perspectives, Toth and Hizsnyik [29] for example distinguish between: building models together, teaching and training games, policy exercises to explore alternative futures, and Integrated Environmental Assessment focus groups to involve citizens and other stakeholders in decision-making processes. Van Asselt and Rijkens-Klomp [30] identify eight different methods such as focus groups, scenario analysis, scientific stakeholder workshops, policy exercises, participatory modelling building, citizens’ juries, consensus conferences, and participatory planning. Geurts and Joldersma [31] distinguish between gaming/simulations, consensus conferences, and electronic meetings. Tuler and colleagues [32] identify five different ways of bringing stakeholders together to summarize how a system works: participatory modelling, mediated modelling, cooperative modelling, group model building, and computer-mediated collaborative decision making.

Beside the classifications, numerous criteria for evaluating the quality of stakeholder engagement processes can be found in the literature [7,33,34]. For example, Renn and colleagues [12] measure the performance of participation processes by developing a normative framework around the central principles of procedural fairness and competence in knowledge verification. Rowe and Frewer [34] provide a set of nine criteria divided in two sub-groups: acceptance criteria, leading to public acceptance of processes (i.e., representativeness, independence of participants, early involvement, influence on policy, and transparency) and process criteria, leading to an effective process (i.e., resource accessibility, task definition, structured decision making, and cost effectiveness). Webler and Tuler [35] identify seven principles for public participation, including access to process, power to influence process and outcomes, facilitation of constructive interactions, access to information, adequate analysis of available data, and enabling the conditions for future processes.

Although the typologies, classifications, and criteria presented above are certainly helpful, they leave several open issues, such as how to choose the most appropriate approach according to the features of the issue and the goals pursued, how and when to include stakeholders’ perspectives, how to do it effectively, how to integrate qualitative and quantitative information, what role the experts play in the process of knowledge coproduction, as well as if and how the different methods and tools affect the outcome. In this paper, we contribute to the discussion about these issues by comparing and contrasting three approaches for the integration of stakeholder perspectives and knowledge co-production.

3. Approaches for the Integration of Stakeholder Perspectives and Knowledge Co-Production

In this section, we describe the aims, key assumptions, typical works steps, types of data used, and results generated in three different approaches—respectively grounded in multi-criteria analysis, plural rationality theory, and scenario analysis—and we provide examples of their application. We selected these approaches because of our knowledge/research experience and because of the differences in the integration of stakeholder perspectives and in the co-production of knowledge between experts and stakeholders—i.e., the key aspects under study. It is also important to highlight that these approaches are not at all new in decision-making theory and practice. They have been developed in different disciplinary fields (e.g., economics, cultural anthropology, political sciences, engineering, operational research, etc.), often without deep contacts between research groups. This is also one of the reasons of the limited comparative studies. Also, we are aware that the boundaries between these approaches may sometimes be blurred, and that methods from more than one approach may be used in the same decision making process. For example, scenario analysis is, at least in principle, applicable as a method within both the other approaches. Nevertheless, they are also distinct approaches frequently used to address and solve problems in environmental decision-making processes.

In the sections below, we describe each of the approaches: the aims and key phases are summarized in

Table 1. Summary of the aims and key phases of the approaches.

Criteria	Multi-Criteria	Plural Rationality	Scenario-Based
Aim	Identify the most desirable alternative	Find a clumsy/compromise solution	Show impacts of different solutions to an environmental problem
Key phases	Development of alternatives, identification of criteria, assessment of the criteria for the different alternatives (with stakeholders), ranking of alternatives	Identification of stakeholder perspectives and needs, option co-production, working groups, compromise proposal	Storyline development, quantification of input parameters, interaction with model or model results

. The approaches are compared and their similarities and differences discussed in Section 4.

3.1. Multi-Criteria Based Approaches

Multi-criteria analysis (MCA) is a formalized and well-established decision support method used to reach a consensus regarding the ranking of different alternatives [36]. By allowing impacts to be measured in different units, MCA supports decision makers to identify the most desirable alternative [36]. More precisely, MCA addresses trade-offs between different alternatives and clarifies values associated with decisions, by considering one problem description and evaluating different alternatives to solve it [37,38] and by putting emphasis on including additional criteria to the traditional economic ones. The alternatives are broken down into criteria, which are valued and/or weighted separately and then recombined and projected back onto one common scale.

The criteria are typically economic, technical, social, and environmental and their combination varies depending on the problem at hand. Multi-criteria analysis has been applied in many different sectors relevant for environmental decision making, such as ecosystem services, water, forest, natural hazards, renewable energies, electricity infrastructure, and urban sustainability [38,39,40,41]. Usually MCA-based approaches use predominantly quantitative data. Stakeholders’ perspectives—usually described by using qualitative information—are not always explicitly addressed in MCA. Hence, traditional MCA methods alone often prove insufficient when confronted with the stakeholders’ different definitions of the problem to be addressed or with the integration of stakeholders’ perspectives on their preferred alternative.

In order to overcome these limitations, MCA has been combined with other social research methods allowing a better representation of stakeholders’ perspectives. Different methods and techniques can be used to integrate diverging stakeholders’ views, such as surveys, discourse based evaluation, narrative analysis, and value integration methods [42].

As a result, several variants of multi-criteria methods have been developed to accommodate the participation of stakeholders [40,43,44,45,46]. Stakeholders may play a relevant role in the identification of alternatives, in the selection of criteria and their weighting, and/or in the evaluation of alternatives [47,48].

In this section, we focus on one of the variants, Social Multi-criteria Evaluation (SMCE) because it pays particular attention to combining stakeholder perspectives and participative/institutional approaches and it puts particular emphasis on stakeholder engagement [38,49]. The phases of SMCE can be summarized as follows: (i) institutional analysis, identification of main stakeholders, definition of the decision problem and objectives; (ii) identification of alternatives and of economic, social, environmental and technical criteria to compare them; (iii) assessment/evaluation of the criteria for the alternatives and of stakeholders’ preferences for each alternative; (iv) identification of the most desirable alternative (Figure 1).

SMCE begins with an institutional analysis, which identifies the key stakeholders and describes their tasks, responsibility, and views about the problem to be addressed. The identification of alternatives is followed by the selection of relevant criteria and finally different methods are used to compare the alternatives. Alternatives and criteria are usually identified by experts. More precisely different matrixes are elaborated: the impact matrix evaluates the alternative on the basis of the criteria and the social impact matrix evaluates the alternatives based on stakeholders’ opinions, preferences and views. The social impact matrix allows to visualize the dendrogram of coalition formation, which shows possible alliances among stakeholders and pictures the coalitions which might be established according to similarities in opinions (more details in [38]). The social impact matrix and the dendrogram of coalitions are the two key tools to integrate stakeholders’ perspectives in the decision making process, to take into account their views and to anticipate conflicts.

For example, Paneque Salgado and colleagues [39] did an SMCE to evaluate the urban water supply alternatives in Costa del Sol Occidental (Malaga, Spain). The engagement of stakeholders allowed to explore a non a priori foreseen alternative and proved extremely helpful for problem structuring in a collective, flexible, and iterative way, unveiling existing water management conflicts and their motivations and improving the quality and effectiveness of information exchange.

More precisely, the social impact matrix and the dendrogram of coalitions revealed that the public authorities’ problem framing in terms of water scarcity and “structural water deficit” did not coincide with the stakeholders’ perspectives. The majority of stakeholders framed the problem in terms of inadequate management, criticized the lack of forward planning in a geographic area suffering the effects of unbounded growth, lack of coordination among the authorities (e.g., those responsible for water management and for land use planning). This matrix provided useful insights not only into better understanding of stakeholder views but also into which alternatives were more likely to be accepted.

3.2. Plural Rationality Approaches

Plural rationality approaches (PRA) do not seek a consensus on a single best option, but rather a compromise solution reached through explicit elicitation of stakeholders’ perspectives on the nature and cause of the problem and its solution [50,51].

This approach draws upon the theory of plural rationality which has demonstrated that in every policy discussion, there is a limited number of socially constructed stakeholders’ perspectives: hierarchy, individualism, egalitarianism, and fatalism [52,53]. These perspectives are characterized by shared attitudes, behaviors, interests, and views on what is relevant and why for different stakeholders.

For example, the hierarchical perspective is pro-control and insists that problems demand expertly planned solutions. This translates into top-down planning through government authorities with their network of experts. The individualist voice is instead pro-market. It calls for de-regulation, for the freedom to innovate and take risks, and for the explicit recognition of trade-offs among competing uses of resources. The egalitarian perspective is strident and critical. Deeply sceptical of both the individualist notion of trade-offs (especially when lives and other “sacred” values are at issue) and the hierarchists’ claim that experts know what is best. This perspective usually argues for a more holistic, moralistic, and natural approach to problems. Finally the fatalist perspective sees no possibility of affecting change for the better (for details, see [50,53]). These perspectives can be adapted to a number of different environmental issues—e.g., climate change, natural hazards, urban sustainability—and are a point of reference to construct stakeholders’ discourses, i.e., shared, structured ways of speaking, thinking, interpreting, and representing ideas and set of ideas [54]. Discourses are usually elicited by means of social science methods such as interviews, focus groups, questionnaire surveys, documentary analysis, cognitive mapping methods, e.g., [55]. They form the basis for knowledge co-production: technical options, scenarios, integrated assessment models, or other tools can be used to “translate” the qualitative information gained through the discourses into quantitative data/technical option used for supporting the decision-making process, e.g., [56].

Figure 2 provides an illustration of the key phases of the process design in PRA: (i) stakeholder perspectives’ elicitation and discourse generation; (ii) generation of technical-policy options; (iii) working groups to discuss the options and identify priorities for action; (iv) facilitated discussion to reach a clumsy/compromise solution.

In this approach, knowledge co-production plays a vital role when experts/researchers translate stakeholders’ perspectives into technical options. Moreover, experts provide the background to reach a compromise—a clumsy solution by interacting with stakeholders and providing scientific inputs. The pre-requisites for a clumsy solution are accessibility (each perspective able to make itself heard) and responsiveness (each perspective engaged with, rather than dismissive of the others). This means that a critical point is the respect for heterogeneity of stakeholders’ views rather than convergence or persuasion towards a consensus.

PRA has been applied to several environmental issues such as the handling of radioactive materials, ropeways in Nepal, climate change and cities, insurance for flood risk, disaster risk management [18,56,57,58,59,60]. For example, in a case of landslide risk mitigation in Southern Italy [18], the experts involved in a participatory process played a unique role by providing technical options that corresponded to the different perspectives held by the stakeholders. Three different risk mitigation options reflecting respectively the hierarchical, egalitarian, and individualistic perspectives (respectively called Safety First, Careful stewardship of the mountains, and Rational Choice) were provided by the scientific advisors/experts on the basis of the results of interviews with stakeholders, questionnaire surveys, and focus groups. The decision-making process involved 18 residents and the social context was characterized by high levels of conflict. Indeed, a few years before the participatory process, a €24.5 million risk mitigation project prepared by the regional authorities was rejected by the Municipal Council supported by several stakeholders, including citizens. During the process, the citizens discussed their preferred option in working groups and identified priorities. The results of the working groups showed that most participants agreed on the priorities to reduce landslide risk—i.e., an integrated system to monitor landslides, stabilization of the open slopes with naturalistic engineering works, and an improved warning system and maintenance of existing risk mitigation measures. On the basis of the results of the facilitated dialogue, experts drafted a compromise/clumsy solution. In this way, a suitable risk mitigation plan gradually moved from a contested terrain to increasing convergence on a clumsy solution [61].

3.3. Scenario-Based Approaches

Scenario-based approaches (SBA) focus on specifying the impacts of different solutions to a given problem and are widely used in different sectors, including climate modelling (in particular also through integrated assessment models), water management or energy system modelling. Generally, the scenario process stops with presenting the scenarios and their impacts, and does not include a decision-making process: this is usually the decision-makers’ task, based among others (or exclusively) on the scenario analysis. However, most scenario analyses typically show that one scenario is cheaper or more profitable, implicitly recommending the adoption of that option. Further, SBA can be a precursor for an MCA of the developed solutions, it can be used in combination with PRA, or it can be used on a stand-alone basis.

Scenario analyses are often done as pure desktop exercises with a computer model and no dedicated outreach to stakeholders, following a strict rational choice framework, typically seeking the least-cost option for a specific aim (e.g., least-cost energy strategy for 80% greenhouse gas emission reduction [62]). In scenario approaches foreseeing the involvement of stakeholders, the most important task is to link stakeholders’ perspectives with the scenario development. The stakeholder engagement can be done in many ways, from merely ranking the finished scenarios to iteratively creating narratives, quantifying input data, and setting scenario boundary conditions together with the modellers [63,64,65,66,67]. In most cases, and especially for the approaches with high levels of stakeholder empowerment and engagement, the scenario changes depending on the stakeholders’ evaluation of different parameters. Therefore, several interactions between experts and stakeholders may be necessary in order to develop the final scenarios [10,68,69]. The stakeholders’ perspectives are usually analyzed at the beginning of the process and linked to the models in an iterative procedure. A critical point is the translation of stakeholders’ qualitative visions and narratives into quantitative information that is useful to build scenarios. This means, for example, to ask stakeholders for qualitative or quantitative information about demographic, economic, and other trends. This translation from qualitative to quantitative is challenging, and several approaches for how to do this exist [70]. Examples include fuzzy method approaches to derive consistent and transparent numerical values directly from stakeholder [65] or expert-supported quantification of stakeholders perspectives [71,72]. Figure 3 summarizes the key phases of stakeholder engagement in scenario-based approaches (see [66,71,72,73,74]: (i) elicitation of stakeholders’ perspectives; (ii) quantification of inputs to feed the model parameters; and (iii) scenario ranking and identification of actions to enact a particular scenario (see Figure 3).

In many cases, SBAs struggle with the ontological divide between their positivistic view and the stakeholders’ normative and constructivist perspectives. A particular challenge is to ensure that the scenarios are comprehensive (i.e., do not leave out important developments) and internally consistent. The formalized methods for this, such as the cross-impact-balance method [75,76], require experts to judge—and if necessary ‘correct’—the stakeholder input in order to maintain “the primacy of science” [17]. For example, [68], in a project about the decarbonisation of the UK power system, researchers found out that stakeholders “overestimate [the] irreplaceability of carbon capture and storage”, but “underestimate (…) the role of nuclear power”. This means that stakeholders’ perspectives were considered ‘wrong’ by the modellers accompanying the process, which largely nullifies the usefulness of stakeholder involvement. This problem is difficult to solve, as models generally follow one specific rationality, often within a cost-minimizing rational choice framework, whereas stakeholders may follow another (or several) logics. Hence, the (perverse) effects of the discursively very powerful models and their scenarios can sometimes be to close down rather than open up spaces for discussion of alternative pathways to sustainability [77,78].

Yet, there are many good examples of how SBA can be used to enrich the sustainability policy debate and open up the discussion by laying open the impacts of different options. For example, Trutnevyte and colleagues [73] analyzed the decision for which energy strategy to adopt in Urnäsch, a small municipality in north-eastern Switzerland, for it to become more climate friendly and energy secure. A team of researchers, supported by stakeholders of a “steering board” (including the local energy supplier and the mayor), defined a list of six basic energy storylines, each representing one possible strategy. These storylines were subsequently quantified by the researcher team using an energy system model, into a set of technically feasible energy scenarios for Urnäsch. Following this, the researchers carried out an MCA based on criteria elaborated together with the steering board, and let the citizens of Urnäsch, local companies active in energy or energy-relevant (e.g., insulation, boilers) products, and experts/academics evaluate the scenarios. Importantly, the scenarios were evaluated both based on their short narrative description and on the detailed quantified view, to see how the stakeholders react to the hard numbers compared to the catchy stories. The results show that the narratives of local energy autarky appealed to the villagers but not to the experts, who instead preferred a high energy-efficiency future; when the impacts of the narratives were quantified and specified in detail, the autarkic future lost its appeal and crashed in the ranking—faring well in terms of greenhouse gas emissions, employment, and fully lacking expensive imports, but coming at high economic costs, higher local air pollution and disturbance of the landscape. The authors conclude that stakeholders, especially non-experts, follow an intuitive mode when evaluating options, but that information and modelling of scenarios may change their decision.

4. Discussion

In the previous section, we described multi-criteria, plural rationality, and scenario-based approaches and provided examples of their applications. Our analysis reveals four key differences between these approaches (see

Table 2. Key differences between multi-criteria, plural rationality, and scenario-based approaches.

Criteria	Sub-Criteria	Multi-Criteria	Plural Rationality	Scenario
Assumptions	Stakeholder rationality	Bounded rationality	Value-centred	Rational choice
	Result	Identification of option that maximizes positive economic, environmental, social impacts	Compromise/new solution	Identification of the maximum-utility or least-cost option
Problem and solution framing	Problem framing	Experts, sometimes with stakeholders	Primarily stakeholders	Experts, sometimes with stakeholders
	Main method to frame solution	Expert analysis	Facilitated dialogue	Expert analysis
Knowledge co-production	Experts’ tasks	Provide solution by comparing alternatives	Provide different options and an informed opinion to design co-produced options	Specify impacts of options; sometimes frame problem, specify solution options, evaluate criteria
	Stakeholders’ tasks in the process	In definition of relevant criteria, discussion/weighting of criteria and alternatives	In problem scoping, coproduction of technical policy options, identification of compromise solution	Story development and quantification of input parameters, evaluation of scenarios
	Knowledge generation	From experts to stakeholders and back	From stakeholders to experts and back	From experts to stakeholders and back
Data analysis	Information/data	Predominance of quantitative data and analysis	Mixed approach: quantitative and qualitative data and methods	Mixed approach: quantitative and qualitative data and methods
	Information flow	Two-way	Two-way and iterative	Two-way (typically iterative)

): the assumptions about stakeholders’ rationality and the result/most desirable outcome; the problem and solution framing; the experts’ and stakeholders’ tasks in knowledge co-production; and the data analysis.

4.1. Assumptions

Key differences between the approaches lie in the assumptions of respectively: (i) one universal (be it bounded or not) rationality vs. plural rationalities; (ii) an outcome that minimizes costs or maximizes utility and/or positive impacts vs. a clumsy solution—a compromise—reached through facilitated dialogue.

In MCA, the most desirable outcome is the one that maximizes the positive environmental, social, and economic impacts, most of the times by avoiding trade-offs and achieving the greatest win-win solution. Usually one of the alternatives—generally presented by the experts, sometimes including stakeholders’ inputs—emerges as the most desirable outcome [79]. The underlying assumption is that the stakeholders will agree on the results of the expert analysis (even if this is not always the case in actual applications). In MCA approaches, bounded rationality is what drives decision and actions. This means that stakeholders’ rationality is limited to the information they have, the cognitive limitations of their minds, and the finite amount of time that they have to make a decision [80]. Another assumption is that their preferences change during the process and converge towards the most desirable alternative identified by experts.

The SBA is similar to MCA (and sometimes identical) in that it is a rational choice approach, seeking the utility-maximized or cost-minimized solution to an environmental problem. This optimal solution can be the outcome of a model, especially a cost-optimization model, or it can be the result of a stakeholder valuation of the model output (the scenarios). In both cases, stakeholder data can be inserted before or during the modelling, whereas the ranking must happen after the modelling. A central assumption here concerns the rationality of stakeholders, which are both able to correctly (in a positivist sense) provide quantitative data and to rationally evaluate the options. Similar to MCA, it is generally assumed that the model and the stakeholder engagement will lead to objective results accepted by all stakeholders (see [10]). Moreover the decision-making is, in principle, not a part of the scenario approach, which instead seeks to reveal the impacts of options and the trade-offs between them, without telling which option is ‘best’; in reality, scenario exercises almost always identify the ‘best’ solution, and often provide policy recommendations.

As opposed to MCA and SBA, a central assumption of PRA is that there are multiple rationalities. As described in 3.2., PRA identifies four rationality types (individualist; egalitarian; hierarchical; fatalist) which reflect socially constructed stakeholders’ perspectives, evoking conflicts between cost minimization and equity, trust and disbelief in authorities, etc. As opposed to the other approaches, in PRA the understanding that stakeholders have different rationalities, and hence problem framings, is a key point. Moreover, rationalities and stakeholder preferences do not change during the process (as opposed to MCA). Explicit elicitation and structuring of stakeholders’ rationalities and views on the nature and cause of the problem and its solution is the starting point in PRA [50,51]. Instead of identifying one option as superior, PRA seeks a compromising solution behind which everyone can stand. Thus, the outcome (i.e., the clumsy solution) is often a new solution, different from all the initial options. The initial options reflect the perspectives of each stakeholder group, but are often unacceptable to the others: in PRA, facilitated discussion among stakeholders makes these differences explicit, pushes towards convergence on shared priorities, and allows identification of a compromise solution.

4.2. Problem and Solution Framing

Depending on the approach, the problem and the solution framing changes considerably.

Concerning problem framing, the following aspects are particularly crucial to draw a line between approaches: (i) who (stakeholders, experts, decision makers) has the right, knowledge, and power to define what is the problem; (ii) what are the critical issues that need to be addressed to solve the problem; (iii) what information/knowledge is needed to tackle the problem. Yet, only in PRA do stakeholders play a decisive role in problem framing. In MCA and SBA, they may also play a role, but most of the time they are not involved in the early phase of problem framing. Usually decision makers and/or experts are in charge of defining the problem, highlighting critical issues, and deciding how to tackle them, e.g., by identifying alternatives.

Solution framing is also a critical step in which the heterogeneity of stakeholders’ priorities, values, and interests must be reduced in order to converge towards a compromise/clumsy solution in PRA or a desirable outcome (i.e., an outcome which maximizes positive impacts and/or utility, in MCA and SBA). In this respect, one of the main differences between approaches is the method used to frame the solution, i.e., expert analysis vs. facilitated dialogue. More precisely experts may frame the solution by explicating and/or evaluating the impacts of different options (in MCA or SBA) or by mediating/facilitating the dialogue among stakeholders and designing co-produced technical-policy options (in PRA). It is important to point out that experts with different specializations can (and should) be involved in these processes, e.g., there may be experts in scenario-making, in collaborative learning, in negotiated compensation, in discourse analysis, etc. They may also have completely different disciplinary backgrounds, from social/political sciences to engineering, economics, or ecology.

4.3. Knowledge Co-Production

A key difference between approaches is when and how knowledge is co-produced between experts and stakeholders.

In MCA, experts open up the complexity of alternatives, compare them, and identify what alternative suits better a certain decision problem [45]. They may provide alternatives and/or the criteria to compare them. They may also elicit the criteria via discussion with stakeholders who may play a relevant role also in the prioritization of the criteria and/or in the evaluation of the alternatives. Therefore, the stakeholder’s inputs to knowledge co-production are guided by experts.

The same is true in SBA, but in this case, specific attention is devoted to long-term targets to be reached in the future (e.g., for climate change, land use etc.) and stakeholders are expected to quantify their narratives and storylines by providing parameters to feed the models. Experts provide the background for designing storylines and for identifying model parameters, for creating the model, and often for evaluating results.

In PRA, the stakeholder inputs to knowledge co-production are instead guided by their own needs, interests and perspectives. A key difference between PRA and the other approaches is that experts provide informed opinions to design co-produced options based on their specialized knowledge as well as local knowledge and stakeholders’ values. Their role is not to provide information on the present state of knowledge in their disciplinary field, nor to ‘speak truth to power’ following a one-way linear knowledge transfer model, nor to deliver solutions for policy. Their main task is to balance evidence; uncertainties; institutional, legal, and social contextual factors; and to co-produce options designed for and with stakeholders. They also provide the background to reach a co-produced compromise—a clumsy solution—by interacting with stakeholders and providing scientific inputs that take into account the priorities emerging from the facilitated dialogue.

4.4. Data Analysis

A key difference between approaches concerns the integration of qualitative information into models, and the translation of qualitative into quantitative data.

In PRA, the quantification of qualitative stakeholders’ views, narratives and discourses is generally not a necessary step, and often it is not done at all: some—or all—aspects may remain qualitative throughout the process, which instead focuses on mediating between standpoints.

In SBA, in contrast, data from stakeholders is often quantified and in many cases included in model runs. This is done in different ways. In some cases, stakeholder perceptions of likely/unlikely storylines are elicited, which are then translated by the modellers themselves into quantitative scenarios (e.g., [66]), the range of possible futures is developed jointly by stakeholders and experts (e.g., [73]), or the experts translate the stakeholder stories to model input data themselves (e.g., [72]). In these cases, to varying degrees, the stakeholders become indirect sources of data (as the experts still help or do the quantification of qualitative statements themselves). This link between qualitative stakeholder narratives and their quantification (or monetization) into meaningful input data for models and scenario construction remains the ‘soft spot’ of participatory SBA.

In MCA, the data is typically also quantified by experts. Sometimes stakeholders are asked to quantify the importance of each of a set of criteria and/or to quantify how well each option fulfils each criterion. These responses may then be aggregated into a final score for each option. Critically, in most cases both the criteria to be weighted and evaluated by stakeholders, as well as the rules for aggregating the responses into a final score are defined by the experts: hence, the experts remain dominant in the evaluation and ranking process, although they generally do not produce the evaluation data themselves.

Hence, whereas in PRA data quantification generally is not a central issue), in MCA, stakeholders and experts do different quantification steps but the stakeholders usually play a decisive role in the final score aggregation. Finally, in SBA experts are strongly dominant in the data quantification.

5. Conclusions

Our analysis shows that the choice of approach makes a large difference for the integration of stakeholder inputs into research and environmental decision making, and that this has consequences for the results of the research and—ultimately—for the decision made. The first difference lies on the ontological level, where especially SBA—but also MCA—approaches assume (bounded) rationality in stakeholders, whereas PRA assumes that there are several equally valid but different stakeholders’ rationalities.

Further, stakeholder engagement is performed at different stages and for different purposes. If stakeholders’ perspectives are included early in the process, i.e., to frame the problem (most common in PRA), then the knowledge produced in the process grounds on these perspectives. This may be challenging for the experts, because they may have to choose the methods, models, scenarios, etc., on the basis of stakeholders’ needs and expectations. If stakeholders are involved later in the process (most common in MCA and SBA), for example to evaluate or rank ready-made alternatives, experts decide a priori what approach to use to address the problem. Stakeholders still influence the results, but the depth of their inputs is reduced.

Stakeholders’ perspectives can be included in the shape of qualitative, quantitative, or mixed data, and often a translation of qualitative views into alternatives, technical options, models, or scenarios—including quantification efforts—is necessary, which adds difficulty and complexity to the process, but can also enrich the final outcome.

Overall, another key difference lies in the roles of experts and stakeholders in knowledge co-production. In the case of MCA or SBA, the experts usually define the options or scenarios—sometimes supported by stakeholders—and identify the most desirable solution, based on the evaluations of stakeholders (for which the experts still determine the variables and the aggregation rules). In PRA, the stakeholders—supported by the experts—define the options, on the basis of their own opinions, needs, and values. Stakeholders also identify a compromise or ‘clumsy solution’, which is usually very different from the initial option. Also in this approach expert support, advice and co-design of the clumsy solution plays a critical role, but it is less dominant than in MCA and SBA.

All of this leads to different types of results: whereas MCA outcome reveals the most desirable alternative, SBA generally shows the least-cost (or maximum-utility) option, PRA provides a non-optimal compromise solution that is preferred by no one, but acceptable to all.

Hence, by choosing a particular approach, one also chooses how to address a problem (and often also what problem to address), which role is played by the experts in the process and which possible solutions are looked at; how, when, which and for what purpose stakeholder data is collected; and what the results will look like and how they can be used. The choice of an approach enforces a particular view (the expert’s normative view) about what knowledge co-production should be, how stakeholder perspectives should be integrated with scientific knowledge, and how options should be generated and discussed. Clearly, the choice of the approach is important and will affect the outcome and, consequentially, the decision made.

Still, it is not possible to say that one approach is superior to another: they all have strengths and weaknesses, and they are all practical or impractical in different ways. For example, if the alternatives to address a certain problem are already decided and cannot be changed, then multi-criteria-based approaches may be a good choice, as the ranking or relative evaluation of these alternatives reflects the policy problem (i.e., what alternative to choose?). If the decision addresses a long-term and broad problem, such as climate change mitigation, then a scenario approach may be a suitable choice, as the solutions need to be defined and their impacts analyzed. If a problem is very contested, a plural rationality approach may be suited, as it seeks to first lay open the points of controversy, it acknowledges different viewpoints as equally valid, and then it seeks a compromise/clumsy solution. In many cases, a combination of several approaches may be useful, as is already often the case for SBA and MCA, but also PRA methods can be used in combination—especially at early research stages—with the two other approaches. The choice of the most appropriate approach thus depends on the research question and the type of problem at hand; yet, it is crucial for the experts and decision makers to be aware of the consequences of that choice, and in this paper, we have contributed to understanding what these consequences are.