Assessing the Potential, Performance and Feasibility of Urban Solutions: Methodological Considerations and Learnings from Biogas Solutions

Abstract

Many cities of the world are faced with multiple sustainability challenges, for example related to food and energy supply, transportation, waste management, clean air, and more. Preferably, these challenges are addressed with broad and interconnected solutions with the ambition of addressing several challenges simultaneously, in this paper referred to as multi-functional urban solutions. Implementation of multi-functional urban solutions requires well informed decisions, supported by knowledge about the potential contributions that the solutions can make to a more sustainable city as well as on issues that may hinder or facilitate their implementation. Thus, in this paper, we suggest a soft multi-criteria decision analysis method that can be used to gather and structure this knowledge. This method acknowledges the importance of incorporating local knowledge, is based on life-cycle thinking, and is flexible and open-ended by design so that it can be tailored to specific needs and conditions. The method contributes to existing practices in sustainability assessment and feasibility studies, linking and integrating potential and performance assessment with issues affecting solutions’ feasibility of implementation. This method offers a way for local authorities, researchers and exporting companies to organize and structure the diverse range of knowledge to be considered for more informed decisions regarding the implementation of multi-functional urban solutions. While the main contributions of the paper are methodological, brief descriptions of two studies that have applied this method to assess biogas solutions are shown as clarifying examples. One of these studies was performed in Chisinau, Moldova and the other in Johannesburg, South Africa.

1. Introduction

Worldwide, many cities are faced with a diverse range of urgent urban challenges, related to food and energy supply, transportation, waste management, climate change, air pollution or other sustainability issues [1,2]. These challenges are multi-faceted and interconnected—affecting societies in several ways while being closely linked to each other. Attempting to address such challenges one at the time can be inefficient and result in fragmented policies and management. Because of potential synergistic benefits, narrow perspectives may also mean missed opportunities for enhancing resource efficiency [3,4]. Furthermore, limited perspectives bring a higher risk of unintended problem shifting, i.e., to cause new or even worse problems in other areas [5]. Thus, it may be wise for representatives of cities and regions, as well as other relevant actors to strive for a broad perspective and adopt solutions that can contribute to several challenges simultaneously [6,7]. In this paper, these solutions are referred to as multi-functional urban solutions.

Many cities in Sweden (and elsewhere) have implemented biogas solutions, here used as examples of a multi-functional urban solution and a platform for learning. Biogas solutions commonly involve cross-sectoral co-operation, where waste streams and wastewater flows are transformed into valuable products such as renewable energy and biofertiliser [8,9,10]. Accordingly, they can contribute to several urban challenges as they can be an important part of the waste treatment system, contribute to the local economy, decrease local emissions and climate impact and enhance nutrient and soil management. During the last decades, the successful utilization of biogas solutions in Swedish cities has become increasingly known to the international audience [11,12], and delegates from different parts of the world have come to Sweden asking whether they can implement biogas solutions in their hometown and what performance they may expect.

In response to this, the Swedish government has set up an arena to demonstrate and disseminate knowledge regarding various multi-functional urban solutions, including biogas solutions [13]. Moreover, Swedish cleantech companies consider opportunities to export parts of, or entire, multi-functional urban solutions along with the knowledge to operate these [14]. For these kinds of companies, it is of strategic importance to select favourable regions for their products and services. Consequently, there is a need for methods that can be used to assess the potential, performance and feasibility of multi-functional urban solution in different contexts and, if required, compare them with other possible solutions.

In this paper, departing from the research area of sustainability assessment, we clarify existing knowledge, needs and central issues related to assessing multi-functional urban solutions. Thereafter, based on existing knowledge and needs, we propose a method to assess multi-functional urban solutions, capable of both capturing a solution’s contribution to a variety of sustainability-related challenges and whether the solution is possible and reasonable to implement. Following this, we briefly present two studies applying the method to assess biogas solutions in Chisinau and Johannesburg. Thereafter, we hold a discussion about the contributions, strengths, weaknesses and practical applications of the method. Finally, we conclude the paper by underlining the methodological contributions of the paper.

2. Assessing Multi-Functional Urban Solutions

One way to assess multi-functional urban solutions is through their contribution to the sustainability of the city: in other words, by performing a sustainability assessment. Here, sustainability assessment refers to a family of methods that aim to provide decision-makers with knowledge on what actions to take, or not to take, to improve the sustainability of society [15]. Most sustainability assessments provide normative requirements for actions to be considered sustainable, often combining universal, broad and generic requirements with case-specific requirements [16,17,18]. To assess whether an action meets these requirements, sustainability assessments may raise questions about peoples’ perceptions, value orientation and priorities, as well as questions about material flows, emissions, waste, costs and other physical or monetary aspects [19].

In general, sustainability assessments are more concerned with solutions’ impacts on society than with what is required for their actual implementation. However, Ramos [20] and Turnheim and Nykvist [21] identify a need to further consider feasibility aspects in relation to sustainability, that is, how to implement the solution and realise its contribution to sustainability. This is rarely included outside of economic feasibility e.g., [22,23,24] or techno-economic feasibility e.g., [25]. The feasibility assessment of a solution is usually considered as a separate task from assessing its potential scale of contributions to sustainability and its relative performance in doing so. This separate task can, for example, be done through a feasibility study. Feasibility studies are concerned with understanding and assessing the key factors relevant for the successful implementation of a solution in practice [26]. Nevertheless, it is common to find feasibility studies with narrow perspectives, focusing solely on technical feasibility e.g., [27,28,29] or techno-economic feasibility e.g., [30,31,32], much like some sustainability assessments. However, since multi-functional urban solutions often require the integration of several different socio-technical systems, proper identification and understanding of barriers for their implementation in a given context require a broader feasibility assessment, for example, also including organisational and institutional aspects.

In order to simultaneously assess potential contributions of a multi-functional urban solution to the sustainability of the city, as well as how feasible it is to implement, we propose that the assessment should try to answer two questions: How sustainable? and How feasible? This may require understanding how large the contribution of the studied solution can be and whether it is big enough to make it attractive for implementation e.g., [33,34,35]: In other words, to provide knowledge on the potential scale of operation or amount of resources that can be mobilized by that solution. It may also require looking into its sustainability performance, be it from an environmental, economic, or social perspective e.g., [36,37,38]. Finally, it requires knowledge about other social, technical, organizational, or institutional issues that can significantly affect the feasibility of implementation e.g., [39,40].

As previously mentioned, sustainability assessment may raise questions about peoples’ perceptions, value orientations and priorities. As such, what denotes the sustainability of a society may differ from community to community [41] and stakeholder to stakeholder [42]. Hence, a research approach that can utilize local knowledge and take local value orientations and priorities into consideration is preferable. One such approach is transdisciplinary research. Defined as the integration of knowledge from different academic disciplines and practitioners’ experience in order to solve real-world problems [43,44,45], transdisciplinary research aims to increase the legitimacy and ownership of the research for participating stakeholders through collaborative efforts [46]. This cooperation between stakeholders and the performers of the study allows the stakeholders to convey valuable information about societal needs, local conditions and value orientations, while allowing performers to provide general knowledge to the stakeholders, increasing their understanding of the subject [47].

Finally, assessing the potential, performance and feasibility of multi-functional urban solutions requires a multi-dimensional method. It requires a method capable of structuring information in many different areas while also facilitating the synthesis and integration of knowledge gathered. One such family of methods are multi-criteria decision analysis (MCDA) methods [48,49,50], which are suitable for our purpose because: (a) they can include a wide range of aspects that are deemed important; (b) they can incorporate different types of objectives and value orientations [49]; (c) it is possible to use them in transdisciplinary research [51,52]; and finally, (d) they can handle both quantitative and qualitative information [50].

MCDA methods are concerned with structuring and addressing complex problems involving several criteria in order to support decision-making e.g., [53,54,55,56]. In many cases, these methods focus on the quantitative evaluation of alternatives—via some form of weighing, algorithm or hierarchical process of elimination—in order to arrive at a decision based on optimal compromises among potentially conflicting issues; see, for example, [57] or [58] for an overview of such methods. However, they can also be used in a more flexible and non-deterministic manner to structure and organize a large amount of relevant information for decision-making e.g., [56]. This type of MCDA is referred to as a soft MCDA and aims at problem structuring, rather than problem solving [59]. Thus, providing a procedure for expanding the scope of issues that should be considered in decision-making but may be neglected if not systematically approached. This is the type of MCDA that the suggested method in this paper is based upon. A soft MCDA is chosen because of its ability to handle data constraints (such as data gaps and mixed data); the fact that they are usually simpler and more transparent than their harder counterpart, meaning that they are perhaps better suited to participatory settings; and finally, their allowance of value frameworks other than utilitarianism [59].

3. The Suggested Multi-Criteria Method

Our suggested multi-criteria method for assessing the potential, performance and feasibility of multi-functional urban solutions involves five main steps. These steps build upon the general procedure common for many MCDAs (see for example [58,60]). The five steps are shown in Figure 1 and they are: (1) problem or goal definition, (2) identifying alternatives to assess, (3) defining the multi-criteria assessment framework, (4) assessing each alternative using the assessment framework and (5) interpreting the results. Stakeholders should be involved in all steps but, stakeholders may often be more involved in Steps 1, 2, 3 and 5 and less involved in Step 4, unless they happen to be an expert in a particular area or on an alternative being assessed. Gathering, structuring and discussing the inputs and reflections of stakeholders will require revision and iteration, and as highlighted by Dijk et al. [19] effective sustainability assessments should strive to be reflexive. Finally, because we are concerned with the entire multi-functional urban solution, a life-cycle perspective is of relevance [61]. This means considering the life-cycle of the solution when studying its potential, performance and feasibility.

The first part of the method (Steps 1 and 2) involves defining the goal of the assessment and a few alternative ways of achieving it, or in other words, what types of challenges are supposed to be addressed and what possible alternatives can be considered. For the sake of comparison, alternatives ranging from limited technological fixes to more integrative solutions can be included. One should also think about whether to include any existing or conventional techniques (that may sometimes be contributing to some sustainability issues) as a reference alternative. Illustrated with a simplified example: to solve issues surrounding traffic and transportation, a limited technological fix may be to attach a new exhaust filter to existing vehicles, a more integrative solution may be switching to a bus rapid transit system powered by electricity, and the reference alternatives may be the current transportation system.

Next, the assessment framework should be defined (Step 3). This encompasses three elements: the criteria, key areas and indicators. The elements within the assessment framework will be specific to each study and depend on the stakeholders involved, the identified alternatives, the aim and framing of the study and the local context of the studied city. If any of these changes, it is likely that some of the elements of the multi-criteria assessment framework will differ, even though the overall structure remains the same. Thus, studies with, for example, the same aim will have similarities in their elements, but they will not be the same.

The criteria are generally broad and originate from the aim and problem definition. With regards to implementation of multi-functional urban solutions and the possible alternatives, one can choose as many criteria as deemed important, but they are expected to at least cover feasibility and performance to assess whether the considered alternatives are difficult or easy to implement, and how they would possibly perform. In addition, their potential contribution (scale, scope, or significance) to the urban problems that they are supposed to address should also be considered. Based on these minimum expectations on the assessment of multi-functional urban solutions, we suggest including at least four general criteria in the framework: the criterion of potential (or significance), the criterion of environmental performance, the criterion of economic performance, and the criterion of feasibility. One can also consider adding other criteria, such as a criterion of equity. In these types of assessments, having somewhat overlapping criteria is not necessarily a problem, as long as they are conscious choices and are made explicit. Therefore, excessive—and perhaps unjustifiable—attempts to create mutually exclusive criteria can be avoided. For example, economic performance is clearly linked with feasibility (poor economic performance may imply lower feasibility for implementation), and the overlap can be solved by considering the criterion of feasibility to include indirect economic issues such as policy implications, while direct economic issues are included in the criterion of economic performance. It is central to make sure that important areas are covered; overlaps can be solved by explicitly pointing them out and acknowledging them during the interpretation of the assessment (Step 5).

In order to make it easier to identify and structure relevant information for assessing each criterion, a few key areas are defined. These should reflect what areas are of relevance to fulfil each criterion. The decomposition of each criterion should be carefully deliberated upon in order to keep a balance between lucidity and completeness. Having too many key areas may mean that each key area cannot be thoroughly assessed without massive efforts, leading to analytical paralysis [62]. Furthermore, having many key areas may also make it more difficult to assess the parent criterion. Too few key areas, on the other hand, may miss central issues. To make the key areas more understandable and easier to communicate, each key area can be rephrased as a key question to clarify what information is needed.

The last element of the assessment framework is the indicators, which are used to operationalize the assessment of the key areas, that is, answering the key questions. Since the risk of analytical paralysis is also of concern here, we suggest defining one to three indicators for most of the key areas. If a certain key area demands more indicators, one can consider separating the key area (if possible), to reduce the risk of analytical paralysis, or aggregating indicators to reduce the workload. A reasonable amount of indicators simplifies the gathering of information, the assessment of this information (Step 4) and the interpretation of the results (Step 5). For each indicator, a scoring table should be defined so that it becomes possible to clearly and consistently assign a score to it. We suggest a five-level scale, indicating assessment scores of Very Poor, Poor, Fair, Good, and Very Good. These normative terms are more suitable than terms such as “Very High” or “Very Low”, because depending on the type of the indicator, Low may be considered desirable or not desirable. Moreover, for each indicator, each of these scores should be formally defined. The actual definition of each score is usually case-specific. However, to exemplify,

Table 1. An example of a scoring table with a quantitative (top) and a qualitative (bottom) indicator usable in the multi-criteria assessment framework. Adapted from Feiz and Ammenberg [56].

Indicator 1:Greenhouse Gas Emission Reduction Compared to Fossil Reference
Very poor	Poor	Fair	Good	Very good
0–20%	21–40%	41–60%	61–80%	>81%
Indicator 2:Public Perception
Very poor	Poor	Fair	Good	Very good
The technological solution is seen as unacceptable by a large share of the population without any significant supporters	The technological solution is questioned by a large share of the population, but a few seem to support it	Most people are neutral toward the technological solution, but there are a few supporters and critics as well	The technological solution is supported by a large share of the population, but there are some critics	The technological solution is widely supported and there is no significant critique

shows a scoring table defined for two indicators, one quantitative and one qualitative.

With all the elements presented,

Table 2. The suggested criteria, key areas, and indicators for assessing multi-functional urban solutions.

Criterion of Potential (Significance) The Considered Alternative should Have a Significant Potential to Contribute Towards Solving the Studied Problem.

Key Area 1: Potential scale of supply Can this alternative supply enough products, services (or functions) relevant to contribute in a significant way to solve the considered problem?

Indicators should be defined based on the type of problem and the considered urban solutions. If a bio-based solution such as biogas production in the city context is considered, one can include indicators that can indicate the amount of biomass, bio-products, bio-energy, and bio-nutrients that can be supplied.

Key Area 2: Potential scale of demand Is there significant demand, relevant to the considered problem, for the products, services (or functions) that are delivered in this alternative?

Indicators should be defined based on the type of the problem and the considered urban solutions. Indicators will be similar to those of Key Area 1, but will focus on the demand of products and services considered.

Criterion of Environmental Performance The considered alternative should have good environmental performance.

Key Area 3: Greenhouse gas emissions reduction Considering the life-cycle, is this alternative good (or better than the reference) from the perspective of greenhouse gas emissions?

Indicators should be defined based on the type of the problem and the considered urban solutions. However, greenhouse gas emissions due to their potential contribution to climate change are commonly relevant and can be expressed in t CO2-eq for each alternative (provided that the function, boundary conditions, and main assumptions are specified).

Key Area 4: Primary energy balance Considering the life-cycle, is this alternative good (or better than the reference) from the perspective of primary energy use?

Indicators should be defined based on the type of the problem and the considered urban solutions. However, in most energy-related applications it is important to keep track of the life-cycle energy efficiency, expressed in the primary energy expended for delivering a unit of energy (provided that the function, boundary conditions, and main assumptions are specified).

Key Area 5: Local and regional environmental impacts Considering the life-cycle, is this alternative good (or better than the reference) in regard to various local and regional environmental impacts?

Indicators should be defined based on the type of the problem and the considered urban solutions. Depending on the context, local or regional environmental issues related to air, water, soil, or biodiversity could be of relevance. To represent various local and regional environmental impacts an aggregated indicator may be sufficient, but if needed, some environmental aspects can be represented by individual indicators.

Criterion of Economic Performance The considered alternative should have good economic performance, be it from a business or societal perspective.

Key Area 6: Profitability or cost-efficiency Considering the life-cycle, is this alternative relatively profitable; or if profitability is not the purpose, is it cost-efficient?

Indicators should be defined based on the type of the problem and the considered urban solutions. However, an indication of the operational costs, investments, and revenues (or otherwise costs that are saved) can be represented via either aggregated or separate indicators.

Key Area 7: Degree of control and competing applications Considering the life-cycle, are there competing solutions to the considered alternative? Can the involved actors have a reasonable degree of control over the supply of relevant inputs and demand?

Indicators should be defined based on the type of the problem and the considered urban solutions. Generally, indicators focusing on the degree of control of input materials and competing interests over these input materials are relevant.

Criterion of Feasibility The considered alternative should be feasible to implement.

Key Area 8: Institutional feasibility Considering the life-cycle, is this alternative relatively supported by existing and foreseeable regulations and institutional conditions?

Indicators should be defined based on the type of the problem and the considered urban solutions. However, indicators that describe current relevant legislation, economic incentives as well as public opinion are recommended, as some alternatives may be prohibited or actively hindered while other alternatives may be actively supported.

Key Area 9: Technical feasibility Considering the life-cycle, is this alternative relatively feasible from a technical perspective?

Indicators should be defined based on the type of the problem and the considered urban solutions. Common indicators for this key area are indicators referring to the technological readiness of the alternative, whether there is suitable infrastructure to support the alternative as well as how well the geography suits the alternative.

Key Area 10: Organizational feasibility Considering the life-cycle, is this alternative relatively feasibility from an organizational perspective?

Indicators should be defined based on the type of the problem and the considered urban solutions. Here it is common to include indicators related to the knowledge and learning capacity of involved actors as well as whether the alternatives are in line with the involved actors’ strategies. Furthermore, indicators can also consider the number of actors needed to involve in the implementation of the solution, indicating the challenge cooperation.

gives an overview of our suggestions for the criteria and key areas (as well as the clarifying key questions) to include for the assessment of multi-functional urban solutions. These criteria and key areas should in some way be included but can be adapted, merged or separated if desired. Furthermore, possible indicators for each recommended key area are briefly deliberated upon to guide potential future users of the method.

The suggested criteria and key areas in Table 2 are derived from different origins and are the results of several processes. Clearly, in obtaining the list of the key areas we have been influenced by the literature that we have related to in this paper (for example [33,34,35,36,37,38,39,40]), but we have also relied on our previous studies (for example [56,63]). We have also considered a theoretical understanding that emphasizes efficient resource utilization with least environmental harm over the life-cycle of the considered alternatives. For example, including environmental issues such as greenhouse gas emissions, which are of global relevance, and those with regional or local relevance, such as soil quality. Furthermore, energy is what animates most technological systems and therefore we have considered it to be of typical importance. Finally, to some extent these key areas are derived from common sense. For example, it is common sensual that if we want to assess the economic performance of a certain alternative, we can consider its profitability or its cost-efficiency. Similarly, if a technological solution is going to be implemented, it appears common sensual to consider its commercial availability or maturity. Ultimately, our aim here is not to provide a definitive and final list to be used as-they-are in future studies. Instead, we provide a sufficiently broad list of knowledge areas that we think may be worthy of consideration when assessing multi-functional urban solutions.

During this assessment, it is important to reflect upon the uncertainty of the assessments. We suggest performing a simple qualitative uncertainty assessment for all indicator scores. If the assessment is based on consensus among the involved stakeholders as well as reliable and relevant knowledge, it can be assessed as “rather certain” and marked with three asterisks (***). If there are contradictions, disagreements, and less reliable or relevant knowledge, the certainty can be assessed as “rather uncertain” and marked with a single asterisk (*). The in-between situation can be assessed as “somewhat uncertain” and marked with a double asterisk (**). This allows those performing the interpretation (Step 5) of the result to get a rudimentary understanding of the quality of information in the assessment and it highlights what indicators may need further investigation.

Speaking of which, at this point, the results can be interpreted and discussed (Step 5). This is the step where many MCDA methods perform an evaluation of the alternatives; mostly through analytical and mathematical methods [64], here referred to as hard MCDA [59]. In principle, such approaches to evaluation could also be used here, but we instead avoid evaluations that aim to arrive at certain preferences through algorithmic means and instead this step aims at interpreting the results through qualitative and critical discussion. To enable this, all key areas and related information that have been previously gathered can be discussed to arrive at qualitative and non-deterministic judgement on what alternative is preferred. Such a discussion-based interpretation is in agreement with views such as those of Martinez-Alier et al. [49], who emphasize the benefits of soft approaches to enable inclusion of the human dimension in ecological change and perception changes along the assessment process. A discussion-based approach to the assessment can confront and resolve potential disagreements and conflicts of interests, instead of avoiding them by hiding discussions behind quantitative weighing and algorithms [65]. Furthermore, deliberation between stakeholders of different opinions can change the stakeholders’ value orientation towards becoming more altruistic, in part because they gain a deeper understanding of other stakeholders’ perspectives [66]. This may help stakeholders to compromise with each other. Finally, a discussion-based interpretation of the result allows the inclusion of knowledge gaps in the assessment in a way that is not possible in a mathematical evaluation. If algorithms are to be used, knowledge gaps need to be filled by approximations and assumptions, which may sometimes prove false. The discursive nature of the discussion-based approach also gives more lenience to possible overlaps and interrelations between different criteria and key areas. With a mathematical approach, overlaps and interrelations may need to be solved by overcomplicated definitions and complex mathematical operations, which may reduce the traceability and transparency of the result for decision makers.

4. The Studied Cases

In this section, two studies applying the multi-criteria method to assess biogas solutions in Chisinau and Johannesburg are briefly presented. This section should be seen as an illustration of the type of study design and results one may expect from applying the suggested method, not as a guide on how to perform the method. The studies were performed as master’s thesis projects by four students (two per study) with the supervision of the authors of this paper. In Appendix A, the reader will find more information on how the assessment frameworks were set up and assessed in the two studies. Furthermore, in-depth information about the studies can be found in Alander and Nylin [67] for the Chisinau study, and in Niklasson and Bergquist Skogfors [68] for the Johannesburg study.

4.1. Biogas Solutions in Chisinau

The study in Chisinau was initiated by Chisinau city hall through the Swedish environmental institute IVL and the Swedish city of Borlänge. These actors, together with researchers from Linköping University, co-designed the study. Participating stakeholders were the Swedish embassy, two deputy mayors at Chisinau City Hall, four waste management companies and a representative of the ministry of environment in Moldova. The definition of the multi-criteria assessment framework and identification of alternatives to assess was performed together with the waste management companies as well as the representative from the ministry of environment. Throughout the study, workshops and interviews with the stakeholders were held as well as one formal reflection session that allowed the stakeholders to reflect and give comments on preliminary results.

Due to Moldova’s ratification of the Paris Agreement [69], climate change was placed high on the environmental performance agenda. Furthermore, the country’s reliance on Russian natural gas has led to increased interest in finding alternative and local ways to substitute Russian gas and increase national energy security. Thus, the study put emphasis on the assessment of different end-use applications for biogas, assessing biogas use for electricity generation, heat production, upgrading (Upgrading the gas means increasing the methane concentration to 95–99%.) of the gas for use in public transportation and finally upgrading the gas in order to feed it into the gas grid. Preliminary results in the feasibility criterion indicated that electricity generation was the most feasible end-use alternative. Therefore, it was decided, together with the stakeholders, that the performance assessment would only be done for this end-use alternative in order to provide more in-depth information about this particular application.

4.1.1. Potential

Two different waste-to-biogas flows were assessed. For the first, municipal organic waste, it was estimated that between 102,000 and 112,000 tonnes of wet weight organic municipal solid waste were generated in the city each year. If that amount was sorted and digested anaerobically, the waste could generate 470 to 510 TJ of methane annually. At the time, the majority of this organic municipal solid waste was sent to landfills, resulting in a diminishing of its energy potential and creating worse conditions to recycle nutrients. Furthermore, the wastewater treatment plants in the city treated around 17,000 tonnes of dry matter each year. Through an anaerobic treatment step, the wastewater treatment plants could generate up to 190 TJ of biomethane per year. In total, the different flows investigated would generate around 252 TJ of waste-based electricity generation per year, assuming an electrical efficiency of 37 percent. This would be enough to cover around 6 percent of the city’s electricity consumption and since around 70 percent of its electricity originates in foreign gas and oil [70], this would be a significant reduction in climate impact as well as an improvement to energy security. Using the digestate and sludge as fertilisers on farmlands could enable a nutrient recycling of 900 tonnes of nitrogen and almost 200 tonnes of phosphorous. Nutrient content in the digestate and sludge were based on a UK case study [71] and as such, the actual nutrient content in Chisinau may vary from that of the UK, due to differences in digestate quality.

4.1.2. Environmental Performance

Because of time limitations, environmental performance was only assessed for climate impact; this is not recommended, however, since local and regional impacts should also be addressed in the assessment. Nevertheless, the estimated annual reduction in greenhouse gas emissions totalled approximately 24,000 tonnes of CO₂ equivalents. This was based on substituting natural gas-based electricity with biogas-based electricity. The greenhouse gas emission factor used for natural gas-based electricity was 110 g CO₂ equivalents per MJ [72] and 14 g CO₂ equivalents per MJ for the biogas-based electricity [73].

4.1.3. Economic Performance

The economic performance of biogas solutions in Chisinau was difficult to assess. Context-specific costs were missing because neither pilot studies nor thorough pre-studies on the topic had been previously performed. While the revenue from electricity sales would equal approximately 0.01 € per kWh, including feed-in tariffs, the revenue possible from the collection and treatment of wastes and biofertiliser sales was difficult to assess. Biogas solutions capable of charging for waste collection and treatment as well as sales of both electricity and biofertiliser could be profitable using Swedish cost conditions based on Vestman et al. [74]. However, a more thorough study would be needed to be able to increase the certainty of these results.

4.1.4. Feasibility

The current political situation in Chisinau, as of the time of the study, was quite uncertain. The recent mayor had been detained on charges of corruption, and a re-election was upcoming. This meant that public investments were put on hold. Despite this, the institutional support for biogas solutions was deemed to be good. A waste management strategy and feed-in tariffs did support waste-based biogas solutions, increasing the profitability of such solutions. While the political situation did hinder public authorities to act, the institutional support could make it possible for private biogas solutions to grow. Furthermore, some household waste fractions were already being sorted, such as glass, paper and plastics. This could make it easier to implement a source separation of organic wastes than if such an attempt was made without previous sorting experiences.

Since most of the organic municipal solid waste in Chisinau was landfilled at a fee, a biogas treatment company could likely be able to charge a fee for treatment. While being a financial benefit to the biogas producer, shifting from landfills to a waste treatment with energy and nutrient recovery is also an improvement from an environmental perspective [75].

Looking at the biogas supply side, Chisinau already uses a large amount of natural gas. While the biogas would need upgrading to be used in conjunction with natural gas through a gas grid, the current gas infrastructure opens for more possibilities and makes biogas solutions in Chisinau more flexible.

Perhaps the largest barrier for implementation of biogas solutions in Chisinau is the lack of trust and knowledge regarding the use of sludge and digestate on farmlands. If the by-product of the biogas generation cannot be utilized, a large portion of the economic and environmental benefits of the biogas solution will be lost. To avoid this, field studies and information campaigns on the effects of using local sludge and digestate on farmlands should be performed in the area around Chisinau.

4.2. Biogas Solutions in Johannesburg

The second study, in Johannesburg, was initiated through a collaboration between the University of Johannesburg and the Biogas Research Center, based at Linköping University, Sweden. The study was performed together with the following local stakeholders: the director of energy and the director of waste management and regulation at the City of Johannesburg, one farmer, three researchers, a bus operating company, a transport solution provider, two waste treatment companies, three food industries and two independent consultants in relevant fields. The stakeholders contributed to the results of the study through their expertise and knowledge. Additionally, some stakeholders also contributed to the definition of the multi-criteria assessment framework and the identification of alternatives to assess.

Due to the unsustainable use of natural resources [76] and waste-related challenges such as the lack of new landfill space [77], the emphasis of the study was placed on finding specific promising waste flows to produce biogas from. Furthermore, the city desired to find ways to combat climate change and air pollution problems, much of which originated from the transportation sector. This, in combination with the participation of a transport solution provider, meant that the end-use application considered was biogas used in transportation.

4.2.1. Potential

The assessment of the criterion of potential of the biogas solutions was based on the total generated biogas from identified waste streams as well as the total amount of waste treated in the process of generating the biogas. The study identified and assessed several promising organic waste streams in Johannesburg. Waste management companies, market places, wastewater treatment facilities and food processing industries were among the sources of organic waste identified. While the study was not a complete study of the entire organic waste generation in Johannesburg, it still showed a theoretical energy potential of around 3300 TJ per year, and the investigated waste streams totalled 407,000 tonnes of organic waste (wet weight) yearly. The energy in the biogas produced could, for example, fuel 2600 buses, based on an average fuel consumption by Stafford et al. [78]. This is more than the roughly 1000 buses in operation today in the city, which leaves room for expansion of the bus fleet or use in other vehicles.

The nutrient recycling potential was assessed based on the phosphorous and nitrogen content in each of the organic waste flows investigated. The study found that the implementation of biogas production would produce an annual amount of 46,500 tonnes of digestate (dry weight). The subsequent use as biofertiliser could lead to a recycling of 10,000 tonnes of nitrogen and 2300 tonnes of phosphorous. For South Africa, being a net importer of nitrogen, this should be of interest in order for the country to secure its food production. Calculations for the nutrient content in the digestate were based on a UK case study [71], hence the amount may vary between South Africa and the UK due to potential differences in digestate composition.

4.2.2. Environmental Performance

The environmental performance of biogas solutions in Johannesburg was assessed based on climate impact and impact on air quality. Implementing the biogas solution and using the fuel in buses, instead of conventional diesel fuel, would result in a reduction of greenhouse gas emissions by 256,000 tonnes CO₂-eq per year. This emission reduction could be even higher if a system expansion method is used instead of energy allocation when calculating the contribution of the biofertiliser [79].

Air quality was assessed based on indicators for particle emissions and nitrogen oxide emissions. For these emissions, gas-fuelled vehicles have proven to have lower emissions than diesel-fuelled ones [73,80,81,82]. However, a quantitative assessment of both the production and use of biogas in and around Johannesburg was not possible. This was mainly due to a lack of place-specific data and uncertainties concerning the size of the emission reduction. The information gathered does, however, suggest that while the production may have some local negative effects on the air quality, the use of biogas over diesel in buses should improve the ambient air quality in Johannesburg in areas where the buses operate.

4.2.3. Economic Performance

Assessment of the economic performance of biogas solutions in Johannesburg was based on investment costs, operating costs and revenue from treating waste, and selling biogas and biofertiliser. Finding place-specific data for this criterion was difficult. Thus, the main result here was that a large knowledge gap exists in this area in Johannesburg. However, while no quantitative results could be reached, a previous study with a South African perspective by Stafford et al. [78], and other studies on the economic performance of similar biogas solutions [8,79,83], led to a qualitative result that indicated that the studied biogas solutions would be economically viable in Johannesburg.

4.2.4. Feasibility

The institutional and economic support for biogas solutions were proven to be low in Johannesburg. Clean fuel support systems existed but did not mention biogas, while other alternative fuels were mentioned. Furthermore, the City of Johannesburg’s director of waste management and regulation thought current waste management regulation might even hinder the development of biogas solutions. This was mainly due to the demanding permit process to get all the necessary permits to handle organic waste. Two positive trends here were that a landfill and carbon tax were in discussion as of the spring of 2018. These two economic control measures could benefit the implementation of waste-based biogas solutions. The landfill tax would be positive for biogas solutions due to the fact that a large fraction of the investigated organic waste streams ended up in landfills. Since landfilling is bad from a resource management perspective and the city was running short on landfill space, biogas was seen as a solution to this problem. Due to the large fraction of organic waste being landfilled at a, in the future, significant cost, biogas producers may be able to charge fees in exchange for the organic waste treatment. This would aid the economic performance of the biogas solution.

Two large barriers to the distribution of produce were also found. The first was the lack of farmlands in proximity of the city. This would mean large transportation distances for the biofertiliser, affecting the costs of distribution negatively. Possibly, the lack of close farmlands may require biogas producers to invest in biofertiliser refining techniques to dry and separate the nutrients in order to improve the biofertiliser’s value density. Secondly, the gas network owner in the city does not allow for the injection of biomethane into the gas grid—no matter the quality of the gas. This would mean that a producer of biogas must distribute the gas by truck, which would be more expensive than using the existing gas grid. Furthermore, not having access to the grid would reduce the available market significantly.

5. Discussion

The contribution of the suggested method is twofold. First, the method presents a way to integrate knowledge on potential, performance and feasibility of considered urban solutions. This complements the tendency to do separate assessments for each; instead, these assessments can be integrated and used in combination within the suggested method. The purpose of sustainability assessments is to help decision-making and to facilitate the implementation of solutions that can meaningfully contribute to sustainability. Therefore, obtaining knowledge regarding all three aspects of potential, performance and feasibility is relevant. Secondly, our suggested method broadens the concept of feasibility by including additional relevant feasibility aspects—beyond techno-economic feasibility—and applying a life cycle perspective. Because the method goes beyond techno-economic feasibility, it allows for discussion and deliberation of solutions’ feasibility of implementation. For example, in the studies, the life-cycle perspective and broadened feasibility concept meant that feasibility issues regarding the legislative conditions for processing organic waste were assessed. This aspect may have been overlooked if the focus had been on techno-economic feasibility of the biogas production facility.

Furthermore, something that sets this method apart from other multi-criteria decision analysis (MCDA) methods is the focus of the methodological contributions. While many other MCDA methods are categorised by the evaluation method applied to select the most favourable alternative [57,58,64], our contributions are focused on the entire processes of performing an MCDA—from problem definition, to interpreting the results. With the increasing amount of MCDA studies integrating stakeholders and group discussions in the research design and selection of indicators and alternatives [84], this paper provides valuable methodological insights on how this can be done systematically and practically.

Presented in the paper is the structure and approach of the method as well as some recommendations for what elements (criteria, key area and indicators) to include (Table 2). Because of study-specific differences, elements in one multi-criteria assessment framework will rarely be the same as another. Even though the two studies in Chisinau and Johannesburg shared the same aim, differences between stakeholders involved and the local context of each city meant that emphasis of the study and subsequently, the elements of the assessment frameworks, differed. In Chisinau, due to the desire to reduce the dependency on Russian natural gas, the end-use application of the biogas was left open. Instead, the focus was on how to replace as much natural gas as possible with locally produced biogas. In Johannesburg, due to issues with air pollution, climate change and the participation of a transport solution provider, the end-use of the biogas was fixed to utilization as a vehicle fuel. Thus, the approach lends itself well to bottom-up studies as the method allows the user to consider local conditions and capacities as well as involves local stakeholders [85,86].

Nevertheless, the suggested method does bring with it some limitations. Due to the differences between different assessment framework elements, assessments are usually incommensurable. This is because different assessment frameworks will use different indicators and scales, and thus no common unit of measurement can be found between them. However, incommensurability does not mean incomparability. As Martinez-Alier et al. [49] state, incommensurable results of this kind are still weakly comparable, meaning that the comparison is valid from one perspective without it being valid for all. You might say that biogas solutions perform better in Johannesburg because of the larger potential. At the same time, you may say that they perform better in Chisinau because of the existing economic incentives and infrastructure. Through the understanding gained in the studies it is possible to judge these differences and if desired, make comparative decisions. Such a decision may be valuable to companies looking for new markets. Furthermore, the method may be time-consuming since it requires stakeholder interaction on multiple levels. In addition, in some cases knowledge may already exist for a certain indicator while for others it may require a more in-depth study utilizing other methods, something that may put further pressure on the time constraints of a study. Additionally, as with many participatory methods, stakeholder involvement can also lead to issues with biases. Getting a balanced stakeholder group and giving all parties equal room is important in order to counteract these issues [87,88].

Finally, the method presented in this paper is open-ended and flexible in its use. It can work as a complement to existing methods that are more focused. More focused methods have many benefits, such as in-depth results, extensive user support and assistance with focusing and prioritization. However, certain knowledge may be incompatible with these methods and hence, downplayed or ignored. This could be due to the type of information (qualitative or quantitative); information being outside the focus of the method, even if it is relevant to the study; or simply that the method shifts users away from certain kinds of information toward other kinds. The suggested method allows for the integration of knowledge from these types of methods to ensure a broad assessment where any relevant knowledge can be used. Moreover, many different types of actors may use the method; some examples are public authorities, researchers, exporting cleantech companies and non-governmental organisations. For these actors, the method can offer a way of gathering and integrating various knowledge areas and types to be used as a complement to current decision-making practices. This enables broader and more informed decisions, which is crucial when assessing whether to, and how to, implement solutions aimed at solving several sustainability challenges simultaneously.

6. Conclusions

Because they address many different complex urban challenges simultaneously, assessment of multi-functional urban solutions requires a broad approach. It requires the integration of knowledge on the solution’s potential contributions, the solution’s performance and the solution’s feasibility of implementation. This paper provides a method and methodological insights to enable broad assessment of multi-functional urban solutions’ potential, performance and feasibility. The suggested method is based on soft multi-criteria decision analysis. The methodological contributions and insights in the paper provide guidance when applying soft multi-criteria decision analysis in this way. This integrated assessment is needed to compliment the tendency to do separate assessments for each topic, making it possible to judge a solution’s contribution to sustainable development and its feasibility of implementation simultaneously within the same assessment.

The methodological insights provided in the paper are mainly focused on the early stages of multi-criteria decision analysis, namely the problem definition, selection of alternatives and creation of assessment framework. Explicitly, they deal with how to involve stakeholders that may improve the relevance and legitimacy of the assessment, and what knowledge areas to include in the assessment. Here, the paper suggests a list of some knowledge areas (key areas) to consider in the assessment of multi-functional urban solutions. However, these should not be seen as a final or definitive list of areas. Studies in different contexts, involving other stakeholders, may add to or modify this list of knowledge areas to suit their specific need. Because these methodological contributions are aimed at earlier stages in the analysis, these insights may be useful to consider for other types of multi-criteria decision analysis studies. Even those that aim to utilize mathematical techniques for the interpretation of results in order to produce a final preferred alternative.