**2. Methodology**

A critical point in multidimensional managemen<sup>t</sup> problems is the evaluation and combination of di fferent types of available information that are ultimately able to lead to an optimal solution [27]. Multi-criteria methods provide the framework for collecting, registering, and ultimately promoting all relevant information, thus making the decision-making process detectable and transparent [28,29]. In this light, the adoption of a decision is based on the result of the analysis of the conflicting parameters and goals of socio-political, economic and environmental nature, thus creating a multidimensional problem that needs special treatment [30]. The nature of decision-making processes makes it di fficult to represent them in descriptive models. Furthermore, uncertainty and inaccuracy are inalienable elements of their structure.

Multidisciplinary analysis can be defined as a systematic and mathematically standardized e ffort to solve problems arising from conflicting goals in an e ffort to reconcile them [31]. Making a decision is the study of identifying and selecting alternative solutions based on the preferences of the recipient's decision. Decision making also implies that there are alternatives being considered. In this case, the goal is not only to identify as many of the alternatives as possible, but to choose the one that best fits the goals and desires of the decision maker [32]. Decision making with the use of multi-criteria analysis is realized in four discrete steps, as follows: The first step comprises the determination of the alternative scenarios for the selection of environmental indicators. The second step includes the selection of the criteria, the scoring scale of the alternative scenarios and the assessment of the weighting factors by the decision makers. The next step includes the application of the multi-criteria analysis and the extraction of the results, followed by a last step, where the decision is realized for the selection of the appropriate set of indicators, taking into consideration the use results of a sensitivity analysis.

The selection of environmental indicators for investment evaluation is a very complex process. A considerable number of alternatives, often presenting equivalent weightings, need to be evaluated [33]. To e fficiently achieve such an assessment, it is necessary to analyze and grade a series of critical parameters, or other criteria. In particular, the formulation of an integrated policy regarding the creation of environmental indicators for investment evaluation is considered a rather complex process, given that the number of environmental indicators (alternatives) can be quite large, and at the same time, each indicator shows advantages and disadvantages on di fferent levels, namely economic, environmental, social or technological (criteria).

The combination of all the parameters that appear makes the selection of an environmental indicator a rather complex problem. The final selection of the most appropriate buddle of indicators between alternative scenarios requires the consideration and evaluation of several parameters, therefore, it is necessary to apply the multi-criteria analysis. In the literature, there are over 50 multi-criteria analysis techniques [34,35], and a different classification can be attempted according to their content and scope.

In the present work, the ELECTRE III technique is selected to process the collected data via the distributed questionnaires. ELECTRE III is a well-known method of multi-criteria analysis, with a long history of successful practical applications internationally [29,36,37]. The method was used often in the past to compare different scenarios in different thematic areas, such as energy, construction, waste management, services, public policy, and transportation. An important advantage of the ELECTRE III method over other methods is its usefulness in examining environmental problems [38]. In addition, ELECTRE III has the ability to include a fairly large number of evaluation criteria for the selection of environmental indicators, combined with the ability of a large number of decision makers [39]. The method requires the determination of three threshold values of the criteria used, i.e., the indifference, the preference and the veto threshold. These thresholds allow the uncertainties of the evaluation criteria to be incorporated into the decision-making process [40].

Determining the recipient's preference data for a decision expressed as a criterion is one of the most important factors of ELECTRE III. It is already noted that the method uses the thresholds of preference and indifference, and includes an additional parameter, the concept of veto [41]. By using these parameters, the method examines not only the two extremes of the problem, strong and weak, but also a whole family of intermediate levels, from the overall strongest to the overall weakest alternative. The process is achieved by assessing, comparing and finalizing the various environmental selection indicators (alternatives) over the criteria considered.

As a ranking technique, ELECTRE III calculates a ranking hierarchy among alternatives. The ranking is based on concordance (*cj*) and non-discordance (*dj*) binary outranking. In brief, concordance is valid for the cases where alternative *X* outranks alternative *Y* when most of the criteria *X's* performance is better than the alternative's *Y*. Respectively, non-discordance is valid for the cases where none of the criteria in the minority are strongly opposed to alternative's *Y* outranking by alternative *X*. The ELECTRE III methodology calculates a credibility index, which characterizes that *X* outranks *Y*. The credibility index shows the real degree of the aforementioned assertion [42].

Following this, alternatives are pairwise compared for every criterion by inserting two more pseudo-criteria, namely the preference (*pj*) threshold and the indifference (*qj*) threshold. In the case where the difference between the performances of *X* and *Y* is lower than the indifference threshold for a specific criterion, then the two alternatives are regarded as similar for that criterion *j*, and the credibility index *cj(X,Y)* equals zero. On the other end, in the case where the difference between the performances of *X* and *Y* is larger than the preference threshold for a specific criterion, then *X* is strongly preferred to *Y* for that specific criterion *j*, and the credibility index *cj*(*X,Y*) equals one. In this context, the concordance index *cj(X,Y)* for any criterion *j* is mathematically described with Equation (1).

$$\begin{aligned} V\_j(\mathbf{X}) - V\_j(\mathbf{Y}) \le q\_j \Longleftrightarrow c\_j(\mathbf{X}, \mathbf{Y}) = 0\\ q\_j < V\_j(\mathbf{X}) - V\_j(\mathbf{Y}) < p\_j \Longleftrightarrow c\_j(\mathbf{X}, \mathbf{Y}) = \frac{V\_j(\mathbf{X}) - V\_j(\mathbf{Y}) - q\_j}{p\_j - q\_j} \\ V(\mathbf{X}) - V(\mathbf{Y}) \ge p\_j \Longleftrightarrow c\_j(\mathbf{X}, \mathbf{Y}) = 1 \end{aligned} \tag{1}$$

Taking into account all the concordance indices calculated for each criterion *j*, and also the weighting factor (relative importance) of each criterion *j*(*wj*), a global concordance index is calculated for every pair of alternatives (*X*,*Y*). The global concordance index (*CXY*) is mathematically described with Equation (2).

$$\mathbf{C}\_{XY} = \frac{\sum\_{j=1}^{n} w\_j \times c\_j(\mathbf{X}, \mathbf{Y})}{\sum\_{j=1}^{n} w\_j} \tag{2}$$

As a next step in the methodology, a discordance index (*dj*) is calculated, with the assistance of preference (*pj*) and indifference (*qj*) thresholds, as those were described above, and with the use of a third threshold, namely veto (*vj*), that gives the acceptable (maximum) difference between the performances of two given alternatives *X* and *Y* in criterion *j* for not rejecting the assertion that *X* outranks *Y*, regardless of the alternative's performance in all other criteria. More specifically, in the case that the difference between the performances of *X* and *Y* is lower than the preference threshold for a specific criterion, then no discordance exists and *dj*(*X*,*Y*) equals zero. On the other end, in the case that the difference between the performances of *X* and *Y* is larger than the veto threshold for a specific criterion, then *Y* is globally preferred to *X,* no matter the performances in all other criteria, and the discordance index *dj*(*X*,*Y*) equals one. In this context, the discordance index *dj*(*X*,*Y*) for any criterion *j* is mathematically described with Equation (3).

$$\begin{cases} \begin{aligned} V\_j(Y) - V\_j(X) \le p\_j \Leftrightarrow d\_j(X,Y) = 0\\ p\_j < V\_j(Y) - V\_j(X) < v\_j \Leftrightarrow d\_j(X,Y) = \frac{V\_j(Y) - V\_j(X) - p\_j}{v\_j - p\_j} \\ V\_j(Y) - V\_j(X) \ge v\_j \Leftrightarrow d\_j(X,Y) = 1 \end{aligned} \tag{3} \end{cases} \tag{3}$$

The credibility index δ*X*Υ of the assertion "*X* outranks *Y*" is mathematically formulated with the use of Equation (4).

$$\delta\_{XY} = \Pi\_{j \in \overline{F}} \frac{1 - d\_j(X, \mathcal{Y})}{1 - \mathcal{C}\_{XY}}, \text{ where } \overline{F} = \left\{ j \in F, d\_j(X, \mathcal{Y}) > \mathcal{C}\_{XY} \right\} \tag{4}$$

In order to determine the optimal set of environmental indicators (alternatives) for the assessment of potential investments within the present work, a questionnaire was used, aiming at the collection of the required data that would feed the ELECTRE III methodology. The questionnaire considered a number of indicators that are commonly used to evaluate the environmental performance of a company in operation, the indicators for the evaluation of environmental performance in highly industrialized countries (e.g., USA, UK, EU, Japan) [43], as well as the ISO 14031/2013 standard [44]. The aim was to explore the dominant aspect of tackling the problem, namely the optimal choice of environmental indicators, through the views of experts. A total of 80 experts' responses were collected, representing all different stakeholders involved in the decision-making process, namely NGOs, business, academia, authorities, certified assessors, and scientist/practitioners activated in the field, in order to capture different needs and requirements. More specifically, half of the experts' sample (40 out of 80) represented the private sector (31 practitioners and business consultants activated in industrial investments and nine senior staff in private companies), while the other half represented the public sector (17 representatives from public authorities at local-to-regional level, 9 representatives from the academia sector, 3 certified assessors and 11 representatives from NGOs activated in the field of environmental protection and sustainability). The aim was to evaluate alternative indicators from all different aspects, with the involvement of a balanced sample of experts, since the former (practitioners, business consultants, enterprises) focus mostly on the business success of the proposed investment, while the latter (public authorities, academia, NGOs) place emphasis on the common public interest.

The questionnaire consists of 18 indicators (alternatives) that were coded to facilitate the processing of the results (Table 1). Potential indicators of sustainable development were selected taking into account the pillars of sustainable development and research work in international literature [43–45] and assessed over four discrete criteria by the experts involved in the framework of the present work. The criteria used for the assessment of indicators' suitability are grounded on a set of sustainable development's pillars and principles. The pillars are the environment (Criterion C1), society (Criterion C2), and economy (Criterion C3). To efficiently evaluate an investment, the above criteria are the main priorities for its successful operation and a more sustainable future. In addition, technology (Criterion C4) was selected as the criterion for evaluation, given that environmental technology and technological infrastructure provide the basis for faster and cost-effective development.


**Table 1.** Alternative indicators for the environmental assessment of investments.

#### **3. Results and Discussion**

Within the framework of the present study, an assessments' matrix was formed, consisting of the alternative scenarios of environmental indicators, and criteria over which the selected alternative scenarios were assessed by the experts. In Table 2, the evaluations of 80 experts are depicted in a scale from 1 to 9, where 1 represents the worst-case assessment and 9 the best-case one. The assessment of each alternative for each criterion is calculated as follows:

$$V\_{ij} = \frac{\sum\_{1}^{N} v\_{ij}}{N}, \; i \in (A\_1, A\_2, \dots, A\_{18}), \; j \in (\mathbb{C}\_1, \mathbb{C}\_2, \mathbb{C}\_3, \mathbb{C}\_4) \tag{5}$$

where:

*Vij*: Assessment of alternative scenario *i* based on criterion *j* for all experts *vij*: Performance of an alternative scenario *i* based on criterion *j* for each expert *N*: Total number of experts


**Table 2.** Assessment matrix.

In the present work, the weighting factors of each criterion were determined by the experts participating in the research. In particular, experts were required to assign a percentage of importance to each criterion according to their personal opinion. The values of the criterion weighting factors emerged as averages of the views of the various experts, who took part in the qualitative evaluation of the environmental selection scenarios. For the assessment of the importance of the selected criteria (environment, society, economy, technology) a weight scale from 0 to 100% was used.

In respect to the preference and indifference thresholds that are required for the ELECTRE III methodology, the following equations were considered [38,46–49]

$$p\_i = \frac{(v\_i \text{max} - v\_i \text{min})}{N} \tag{6}$$

$$q\_i = 0.3 \times p\_i \tag{7}$$

where

*vimax*: The maximum value displayed by alternative *i* for criterion *j vimin*: The minimum value displayed by alternative *i* for criterion *j N*: The number of alternative scenarios (here *N* = 18)

According to the ELECTRE III technique, two distillations are calculated (namely, ascending and descending), before the determination of the final order of the available alternatives. In the case under study, the LAMSADE software was used. In Figure 1, the distribution of the ascending and descending distillations is illustrated for the Baseline Scenario (BS), i.e., taking into account: (a) the opinion of the 80 experts with respect to their assessment for the performance of the 18 alternatives over the four selected criteria, (b) the opinion of the 80 experts with respect to the weighting of the four criteria's importance, (c) Equations (6) and (7) for the calculation of the preference and

indifference thresholds, in particular, in the *x*-axis, the ascending distillation (i.e., from the worst to the optimal alternative), while in the *y*-axis, the descending distillation (i.e., from the optimal to the worst alternative), are provided. For instance, alternative A03 (resources' savings) is the optimal one in both distillations. Correspondingly, alternative A01 (recycling) is ranked third (following A03 and A14) when considering the optimal hierarchy from the worst to the best one (ascending distillation), while ranked second (only after A01) when considering the optimal hierarchy from the best to the worst one (descending distillation).

**Figure 1.** Distribution of ascending and descending preference of alternative environmental indicators for the application of the Baseline Scenario.

The final ranking of the alternatives (indicators) is calculated taking into account the two aforementioned distillations. For the BS, the ranking of the alternative indicators is the following; (i) Resources' savings [A03], (ii) Recycling [A01], (iii) Research, Innovation, Development [A14], (iv) Impact restoration [A05], (v) Health and safety [A08], (vi) Gas emissions [A02], (vii) Social actions [A11], (viii) Pollution prevention [A12], (ix) Alternative energy forms [A06], (x) Information across supply chain [A10], (xi) Legal framework [A16], (xii) Communication and public awareness [A09], (xiii) Environmental policy [A15], (xiv) Environmental accounting [A13], (xv) Biodiversity [A04], (xvi) Education [A07], Environmental standards [A17], Corporate governance [A18].

In addition to the Baseline Scenario (BS), nine (9) additional scenarios were considered for sensitivity analysis purposes. In other words, additional scenarios are examined to study whether changes in the parameters of the problem affect the final solution, with the aim of providing further confidence in decision-making. More specifically, in order to globally assess the alternative indicators (taking into account their performance in the four described criteria, namely environment, society, economy and technology), the following parameters need to be determined: (a) the weighting factor (relative importance) of each criterion, and (b) two pseudo-criteria, namely the preference and the indifference threshold. In the present study, sensitivity analysis is selected to be applied in comparison to the Baseline Scenario. In Table 3, the key parameters (thresholds and weighting factors) of the scenarios (Sx) examined are depicted. The Baseline Scenario (BS) reflects the solution of the mathematical algorithm (Equations (1)–(5)), taking into account the weighting factors (relative importance) of the criteria as averages of the experts' views. In this light, the weighting factor for the environmental

criterion [C1], the social criterion [C2], the economic criterion [C3] and the technological criterion [C4], are 31.9%, 23.6%, 25.1% and 19.4%, respectively. Moreover, for the determination of the preference and indifference thresholds, we used the referenced Equations (6) and (7). However, since both the views of the experts are subjective, while the equations for the determination of the pseudo-criteria (preference and indifference thresholds) are based on assumptions, the algorithmic model presented in the methodology is solved with alternative values with respect to weighting factors and thresholds, so as to provide a "what-if" analysis. In this context, the hierarchy of the alternative indicators is re-assessed with the use of different weighting factors (putting more emphasis on different criteria) in each scenario examined. More specifically, in Scenario 1 (S1), all criteria are equally weighted (25%), compared to the Baseline Scenario (BS), where more emphasis is placed on the environmental pillar of sustainability. In Scenario S2, more emphasis is put on the economic criterion [C3] which weighs 50%, while each of the environmental [C1] and social [C2] criteria weigh 25% and the technological criterion [C4] is neglected. Similarly, different weight factors are applied in the case of the rest of the different scenarios (S3–S9). For the scenarios S10 and S11, the weighting factors are based on the experts' views (similarly to the Baseline Scenario), while the preference and indifference thresholds are altered compared to the values calculated with Equations (6) and (7) in order to assess the sensitivity of those thresholds in the optimal hierarchy of the indicators. The analysis, apart from providing robustness to the ranking of the Baseline Scenario, can be used for altering the weights of the criteria based on the individual needs of the funding programmes.


**Table 3.** Modifications of weighting factors and thresholds for sensitivity analysis purposes.

In Table 4, the final ranking (hierarchy) of the alternative indicators are illustrated for the scenarios examined. Apparently, the ranking shows significant changes in cases of change in the thresholds of preference and indifference. However, the results provide adequate information in the selection of a bundle of indicators to be used for the assessment of alternative investments. More specifically, for the Baseline Scenario, Alternative A03 (Resources' savings) is the highest ranked indicator to be considered in the environmental assessment of potential investments. The same applies for all examined scenarios, except S9, where the focus is solely on economic and technological criteria, which provides robustness to the optimal solution and confidence to the decision-maker so as to always consider the indicator when assessing the environmental performance of any given investment.

Since, in most programmes, more than one criterion is simultaneously considered in order to select the optimal investment, the bundle of five top indicators in the Baseline Scenario was comprised, apart from Alternative A03, by Alternatives A01 (Recycling), A14 (Research, Innovation, Development), A05 (Impact restoration) and A08 (Health and safety). Research, Innovation, Development (A14) is included in the top five criteria for all examined scenarios (being the highest ranked alternative indicator for 3 out of 11 scenarios), while recycling (A01) and impact restoration (A05) are included in the top five criteria for 10 out of 11 examined scenarios (A01 is ranked 6th for S8, while A05 is ranked 6th for S10). In this light, the aforementioned criteria should be considered in the top bundle of indicators

to be selected when designing a funding programme, accompanied by indicators like A14 (Research, Innovation, Development), A11 (Gas emissions) and A06 (Alternative energy forms), which are placed comparatively highly for most of the examined scenarios. On the contrary, Alternative A18 (Corporate governance) is the last indicator to be considered for the assessment of investments, as it is ranked 18th for 8 out of 11 scenarios.

**Table 4.** Final ranking of environmental indicators (alternatives) for investments' evaluation for each scenario studied.


It should be highlighted from the analysis of the variation of the coefficients that the ranking of the alternative indicators in the Baseline Scenario (BS) and the Scenario S1 where all criteria (environmental, social, economic, technological) are equally considered (with a weighting factor of 25%), are identical in all ranking positions. Consequently, the experts' opinion on the significance of the criteria does not significantly affect the ranking of the indicators. In this light, all criteria could be equally considered (as realized in S1) in a real-world case.

Overall, the ranking of environmental indicators for investments' assessment was observed to be significantly influenced by the weighting factors and preference and indifference thresholds. The latter demonstrates the choice of A03 as the optimal alternative indicator, but, on the other hand, also reveals that the final choice of the optimal bundle of environmental indicators for the assessment and evaluation of investments is left purely to the decision-maker and to the thresholds used. This provides the funding authority with appropriate "freedom" to apply the most suitable weighting factors and thresholds that best suit the particular needs of the funding programme and the pillar of sustainable development that should be promoted.
