Interactive Selection of Reference Sets in Multistage Bipolar Method

Abstract

In this paper, the Multistage Bipolar method is developed. The paper presents a synthesis of three streams related to multiple criteria decision-making: the reference point-based approach, the interactive approach and multistage decision processes. A significant problem, the solution of which is a prerequisite for the application of the Multistage Bipolar method, is the determination of the sets of reference objects for subsequent stages. This paper addresses the question of how to utilize an interactive multi-criteria approach to select subsets of ‘good’ and ‘bad’ objects for each stage of the considered process, which the decision-maker will accept as sets of reference objects. Its objective is to propose an interactive procedure for generating these sets. The approach proposed in this paper is illustrated by the utilization of stage sets of reference points, generated via the proposed interactive procedure, within a mathematical model for resource allocation in a multistage regional development planning problem. The problem addressed constitutes a mathematical economics model, while simultaneously demonstrating that multi-criteria methods are widely applicable in management. Of fundamental importance here is the expenditure of public funds in a manner that yields maximum benefits for citizens.

1. Introduction

Multi-Criteria Decision Analysis (MCDA) is a rapidly developing branch of Operations Research. One way of developing multi-criteria methods is the utilization of reference points. This originated as Compromise Programming [1,2]. The reference point was the ideal point, defined as a point in the criterion space where all considered evaluation criteria for the decision problem achieved their optimal values. The goal of this approach was to identify the solution closest to the ideal solution using a predetermined distance metric. A similar method for determining the final solution is found in the highly popular Goal Programming approach [3], as well as in Wierzbicki’s Reference Point Method [4]. The reference point is no longer the ideal solution, but rather an arbitrary point in the criterion space. The next step involves the combined use of both ideal and anti-ideal points, as realized in the Bi-Reference Procedure [5] or the Fuzzy Bi-Reference Procedure [6].

It is worth referring to ideal point analysis as a family of multi-criteria methods, applied in psychometrics for attitude modeling [7] and in political science to position politicians relative to an ideal point on an ideological axis [8], as well as in decision sciences, where the TOPSIS method [9] is particularly prominent. Many variants and variations of this method have appeared [10]. A natural extension of the approach utilizing ideal and anti-ideal points, or another reference point indicated by the decision-maker, is the employment of a set of such points. One of the earliest papers in this domain is [11]. The Bipolar method was presented by Konarzewska-Gubała [12,13].

In the Bipolar method, two sets of reference points are considered: a set of ‘good’ points, possessing properties desired by the decision-maker, and a set of ‘bad’ points, which the decision-maker deems undesirable. Decision alternatives are not compared directly against each other, but rather against the reference points belonging to sets of ‘good’ and ‘bad’ objects using the ELECTRE methodology [14].

Another widely utilized method in multi-criteria optimization is the interactive approach. It assumes that the decision-maker is capable of providing information of a local nature—that is, information regarding a specific alternative or a small subset of decision alternatives. In a classical approach, information about the decision-maker’s preferences is obtained before the calculation procedure is launched, the purpose of which is to determine the solution to the problem. In contrast, in an interactive approach, this information is obtained step by step. In each iteration, the decision-maker is tasked with evaluating a specific decision alternative or a set of alternatives. The information thus obtained is then used to determine the next proposal, which is subsequently presented to the decision-maker. The procedure continues until a solution is found that the decision-maker considers satisfactory.

The first interactive methods were proposed in the 1970s and were utilized for solving multi-criteria linear or convex programming problems [15,16,17,18]. Their scope of application was subsequently extended to discrete problems—that is, those wherein the number of decision alternatives is finite and relatively small—including problems of multi-criteria decision-making under risk [19,20].

The third approach in the development of multi-criteria methods considered in this paper comprises multistage decision processes and discrete multi-criteria dynamic programming [21]. This approach employs a vector criterion function for the overall evaluation of the process, which is a composition of stage-wise criterion functions.

In this paper, the Multistage Bipolar method [22,23,24] is used. We consider a multistage decision process (with a finite number of stages) in which stage decisions (hereinafter referred to as stage alternatives) are evaluated with respect to criteria selected by the decision-maker. This method allows for overcoming a certain difficulty encountered when using Multi-Criteria Dynamic Programming. Decision-makers are often interested not only in the multistage values of criteria; stage values are often equally or even more important to them than the multistage values.

In the Multistage Bipolar method, these difficulties are overcome by utilizing two stage sets of reference points—hereinafter referred to as sets of reference objects—at each stage to compare the stage criterion values of the considered stage decisions (hereinafter referred to as stage alternatives). These include a set of ‘good’ objects defining stage values desired by the decision-maker and a set of ‘bad’ reference objects defining undesirable values. These sets may differ in subsequent stages of the process.

A significant problem, the solution of which is a prerequisite for the application of the Multistage Bipolar method, is the determination of the sets of reference objects for subsequent stages. Both existing data, such as stock market company ratings [25], and expert opinions, which form the basis for generating these sets, can be used for this purpose. In both cases, it may turn out that the size of the obtained sets is excessive, necessitating their reduction by the decision-maker.

This paper addresses the question of how to utilize multi-criteria approach to select subsets of ‘good’ and ‘bad’ objects for each stage of the process. It is assumed that a set of candidates for reference objects is available. The aim of the procedure is to select those among them that the decision-maker accepts as ‘good’ or ‘bad’ ones. To solve this problem, we use an interactive approach. The proposed objects are presented to the decision-maker. The decision-maker’s decision to accept or reject an object as ’good’/’bad’ is then used to determine the next proposals.

The approach proposed in this paper is illustrated by the utilization of stage sets of reference points, generated via the proposed interactive procedure, within a mathematical model for resource allocation in a multistage regional development planning problem. This is one of the fundamental problems of public management—how to allocate available resources in order to improve citizens’ living conditions as much as possible. We employ synthetic indicators to structure the expenditures related to the economic, social and environmental order in an example region in order to best achieve the sustainable development goals defined by the decision-maker.

The Multistage Bipolar method has never been used in the field of regional planning. Ref. [26] is highly relevant to our research as it bridges the gap between theoretical control models and their practical application in strategic planning. The author provides a formalized framework for multistage decision-making that effectively incorporates uncertainty and evolving socio-economic priorities.

Resource allocation is a frequently discussed dynamic optimization problem in the operational research literature [27]. It can be formulated as follows: a specific resource is given, and it should be allocated to maximize the total effect. We will use this approach to model the problem of shaping a regional sustainable development plan, which aims to equalize intra-regional disparities. Development goals apply to all aspects of life, referred to as economic order, social order and environmental order, respectively [28,29,30,31].

The assessment of sustainable development is, on the one hand, multi-criteria in nature and, at the same time, involves the use of aggregated measures [32]. The most common way to consider the problem is to calculate synthetic measures encompassing various factors. There are numerous studies comparing regional development using these integrated measures. Their goal is most often to rank regions and position them relative to others [33,34]. We will further deal with synthetic measures of regional development, but we will use them not for interregional comparisons but instead for shaping the expenditure related to the economic, social and environmental order in an exemplary region.

This paper consists of six chapters. Following the introduction in Section 1, the Multistage Bipolar method is briefly presented in Section 2. Section 3 describes the proposed interactive procedure subsequently used for generating ‘good’ and ‘bad’ objects in the successive stages of the Multistage Bipolar method. A numerical illustration of the proposed procedure is presented in Section 4 (Example 1), while its application in constructing a long-term sustainable development plan is found in Section 5 (Example 2). Conclusions are provided in Section 6.

2. Multistage Bipolar Procedure

The Multistage Bipolar Procedure is a procedure that enables the sorting of multistage decision alternatives, their ranking and the selection of the best multistage alternative. Below, we present a brief description of this procedure based on previous works [22,23,24]. We consider a multistage decision process consisting of T stages. At each stage, we consider sets of stage alternatives ${\mathbf{A}}_t$ (t = 1, …, T), composed of stage alternatives. These are evaluated using K stage criterion functions, $f_t^k$ (k = 1, …, K). We assume that for each stage, the following have been defined:

• The stage sets of good reference objects, ${\mathbf{G}}_t$ (t = 1, …, T), consisting of objects ${\mathbf{g}}_t$;
• The stage sets of bad reference objects, ${\mathbf{B}}_t$ (t = 1, …, T), consisting of objects ${\mathbf{b}}_t$;
• Weights, $w_t^k$, for the stage criteria (t = 1, …, T, k = 1, …, K).

At each stage, the reference set of good objects is disjoint from the reference set of bad objects.

Stage alternatives are not compared directly with one another; instead, their position is determined relative to the stage sets of reference objects ${\mathbf{G}}_t$ and ${\mathbf{B}}_t$.

By combining these stage alternatives, we obtain the set of multistage alternatives, $\mathbf{A}$. Each multistage alternative $\mathbf{a}\in \mathbf{A}$ is a combination of different stage alternatives from different stages and is in the following form:

$$\mathbf{a}=\left({\mathbf{a}}_1,\dots ,{\mathbf{a}}_T\right).$$

(1)

We determine the position of multistage alternatives by calculating, for each multistage alternative, the multistage failure avoidance degree and the multistage success achievement degree defined for the classic (one-step) Bipolar method by Konarzewska-Gubała [12,13].

Next, we sort the multistage alternatives into three predefined classes: A^H (class number 1), A^M (class number 2) and A^L (class number 3). Each multistage alternative from class A^H is superior to any other multistage alternative from classes A^M and A^L, whereas each multistage alternative belonging to class A^M is superior to any multistage alternative belonging to class A^L. Precise definitions will be introduced during the presentation of Example 2.

To determine the best multistage alternative, we identify the non-empty class with the lowest number. We rank the multistage alternatives using a quality index, starting with the multistage alternative for which this index has the highest value. A detailed description of this procedure can be found in [22,23,24].

To sum up, the Multistage Bipolar Procedure consists of the following 5 steps.

Step 1: Stage alternatives are compared with reference objects. Well-known procedures, such as ELECTRE, PROMETHEE and AHP (see [35]), can be employed.

Step 2: The position of each stage alternative with respect to the bipolar stage reference system is established.

Step 3: Relationships within the set of multistage alternatives are established.

Step 4: Multistage alternatives are assigned to predefined classes.

Step 5: The final solution is selected from the multistage alternatives belonging to the class with the lowest number.

3. Selection of Good and Bad Reference Objects—Interactive Procedure

3.1. Introductory Remarks

The purpose of the procedure is to enable the decision-maker to identify ${\mathbf{G}}_t$ and ${\mathbf{B}}_t$ for each stage. Initially, sets of candidates for good and bad objects are created. They can be generated in various ways, for example, based on historical data. In this paper, we assume that, for each stage, the sets of candidates are generated randomly.

In order for the procedure to run smoothly, candidates for good and bad objects should be presented to the decision-maker in the appropriate order. For this reason, we ask them to define a hierarchy of criteria for each stage, starting with the criterion that they consider most important. We assume that the sets of good and bad objects should be appropriately diversified, i.e., they should not contain only objects that are highly rated according to the most important criterion. For this reason, during the procedure, the objects with the best values for the subsequent criteria are sequentially proposed to the decision-maker.

In ensuring that the procedure is well received by the decision-maker, they should not be faced with the need to evaluate an object that, based on previously expressed opinions, should be automatically eliminated from further analysis. For this reason, when determining the set of good objects, we do not present the decision-maker with an object that is dominated by a previously analyzed one. We define the dominance relation in terms of the Pareto principle [36,37], which means that object c dominates object c’ if c is not worse than c’ with respect to any criterion and is better than c’ with respect to at least one criterion.

Similarly, when determining the set of bad objects, we do not propose to the decision-maker an object that dominates the one that was previously proposed. This approach also ensures that both sets will consist of non-dominated objects, which guarantees that the generated sets of good and bad objects will be sufficiently diverse.

Taking into account the assumptions of the Bipolar method, it can be assumed that the sets of good and bad objects should contain at least three elements. The maximum size of these sets is not predetermined. In the procedure presented here, it is assumed that the number of good and bad objects is determined either by the decision-maker or by the analyst assisting them in solving the problem. Both sets should be sufficiently diversified. In real-life applications [25], the number of good and bad objects usually does not exceed 10. Adding more usually does not improve the results. However, as a result of the procedure, it may turn out that it was not possible to identify the number of objects that was initially assumed. If this number is not less than the accepted minimum (e.g., 3 objects), the sets obtained can be accepted and then used in the Bipolar method. Otherwise, the procedure for determining good and bad objects should be repeated for the same or a new set of candidates.

3.2. Description of the Procedure

• Notation:

• ${\mathbf{C}}_t$—a stage set of candidates for good or bad objects in stage t;
• $n_t^{\mathbf{G}}$—the required number of good objects in stage t;
• $n_t^{\mathbf{B}}$ —the required number of bad objects in stage t;
• $m_t^{\mathbf{G}}$—the current number of good objects in stage t;
• $m_t^{\mathbf{B}}$—the current number of bad objects in stage t;
• ${\tilde{\mathbf{C}}}_t$—the set of objects that, in the current iteration, can still be proposed to the decision-maker as a good or bad object;
• ${\mathbf{R}}_t$—the set of objects that have been suggested to the decision-maker as good or bad objects and have not been accepted by them;
• $h_t^i$—the position of criterion i in the hierarchy of criteria at stage t.

• Initial phase:

• Step I1: Determine the values of $n_t^{\mathbf{G}}$ and $n_t^{\mathbf{B}}$.
• Step I2: Assume ${\mathbf{G}}_t:=\varnothing$, ${\mathbf{B}}_t:=\varnothing$, $m_t^{\mathbf{G}}:=0$, $m_t^{\mathbf{B}}:=0$.
• Step I3: Define the hierarchy of criteria—specify the value of $h_t^i$ for i = 1, …, K.

• Identification of the stage set of good objects:

• Step G1: Assume ${\tilde{\mathbf{C}}}_t:={\mathbf{C}}_t$, ${\mathbf{R}}_t≔\mathit{\varnothing },s=1$.
• Step G2: Identify the criterion for which $h_t^i= s$, and assume $l=i$.
• Step G3: Identify ${\overline{\mathbf{c}}}_t\in {\tilde{\mathbf{C}}}_t$ for which $f_t^l$ is maximized; if there is more than one such object, select one of them using a hierarchical approach, applying the hierarchy defined by the decision-maker.
• Step G4: If ${\mathbf{R}}_t=\mathit{\varnothing }$ and ${\mathbf{G}}_t=\mathit{\varnothing }$, proceed to Step G6.
• Step G5: If ${\overline{\mathbf{c}}}_t$ is dominated by at least one ${\mathbf{c}}_t\in {\mathbf{R}}_t\cup {\mathbf{G}}_t,$then proceed to Step G8.
• Step G6: Present ${\overline{\mathbf{c}}}_t$ and $f_t^k\left({\overline{\mathbf{c}}}_t\right)$ for k = 1, …, K to the decision-maker.
• Step G7: Ask the decision-maker whether ${\overline{\mathbf{c}}}_t$ can be considered a good object. If the answer is ‘yes’, proceed to Step G9.
• Step G8: Add ${\overline{\mathbf{c}}}_t$ to ${\mathbf{R}}_t$ and remove it from ${\tilde{\mathbf{C}}}_t$:

  $${\mathbf{R}}_t:={\mathbf{R}}_t\cup \left\{{\overline{\mathbf{c}}}_t\right\}$$

  (2)

  $${\tilde{\mathbf{C}}}_t:={\tilde{\mathbf{C}}}_t\backslash \left\{{\overline{\mathbf{c}}}_t\right\}.$$

  (3)

  and proceed to Step G3.
• Step G9: Add ${\overline{\mathbf{c}}}_t$ to ${\mathbf{G}}_t$ and remove it from ${\tilde{\mathbf{C}}}_t$:

  $${\mathbf{G}}_t:={\mathbf{G}}_t\cup \left\{{\overline{\mathbf{c}}}_t\right\}$$

  (4)

  $${\tilde{\mathbf{C}}}_t:={\tilde{\mathbf{C}}}_t\backslash \left\{{\overline{\mathbf{c}}}_t\right\},$$

  (5)

  $$m_t^{\mathbf{G}}:=m_t^{\mathbf{G}}+1.$$

  (6)
• Step G10: If $m_t^{\mathbf{G}}=n_t^{\mathbf{G}}$ or ${\tilde{\mathbf{C}}}_t=\mathit{\varnothing }$, the identification of good objects is complete; go to the phase Identification of the stage set of bad objects.
• Step G11: Assume $s:=s+1$. If $s>K$, assume $s:=1$. Proceed to Step G2.

The flowchart of the procedure of good-object identification is presented in Figure 1. The identification of bad objects is very similar to that of good objects, except that in each case, the decision-maker is presented with an object that has a very low value for each of the criteria considered in turn.

• Identification of the stage set of bad objects:

• Step B1: Assume ${\tilde{\mathbf{C}}}_t:={\mathbf{C}}_t\backslash {\mathbf{G}}_t$, ${\mathbf{R}}_t≔\mathit{\varnothing },s=1$.
• Step B2: Identify the criterion for which $h_t^i= s$, and assume $l=i$.
• Step B3: Identify ${\overline{\mathbf{c}}}_t\in {\tilde{\mathbf{C}}}_t$ for which $f_t^l$ is minimized; if there is more than one such object, select one of them using a hierarchical approach, applying the hierarchy defined by the decision-maker.
• Step B4: If ${\mathbf{R}}_t=\mathit{\varnothing }$ and ${\mathbf{B}}_t=\mathit{\varnothing }$, proceed to Step B6.
• Step B5: If ${\overline{\mathbf{c}}}_t$ dominates at least one ${\mathbf{c}}_t\in {\mathbf{R}}_t\cup {\mathbf{B}}_t,$then proceed to Step B8.
• Step B6: Present ${\overline{\mathbf{c}}}_t$ and $f_t^k\left({\overline{\mathbf{c}}}_t\right)$ for k = 1, …, K to the decision-maker.
• Step B7: Ask the decision-maker whether ${\overline{\mathbf{c}}}_t$ can be considered a bad object. If the answer is ‘yes’, proceed to Step B9.
• Step B8: Add ${\overline{\mathbf{c}}}_t$ to ${\mathbf{R}}_t$ and remove it from ${\tilde{\mathbf{C}}}_t$:

  $${\mathbf{R}}_t:={\mathbf{R}}_t\cup \left\{{\overline{\mathbf{c}}}_t\right\}$$

  (7)

  $${\tilde{\mathbf{C}}}_t:={\tilde{\mathbf{C}}}_t\backslash \left\{{\overline{\mathbf{c}}}_t\right\}.$$

  (8)

  and proceed to Step B3.
• Step B9: Add ${\overline{\mathbf{c}}}_t$ to ${\mathbf{B}}_t$ and remove it from ${\tilde{\mathbf{C}}}_t$:

  $${\mathbf{B}}_t:={\mathbf{B}}_t\cup \left\{{\overline{\mathbf{c}}}_t\right\}$$

  (9)

  $${\tilde{\mathbf{C}}}_t:={\tilde{\mathbf{C}}}_t\backslash \left\{{\overline{\mathbf{c}}}_t\right\},$$

  (10)

  $$m_t^{\mathbf{B}}:=m_t^{\mathbf{B}}+1.$$

  (11)
• Step B10: If $m_t^{\mathbf{B}}=n_t^{\mathbf{B}}$ or ${\tilde{\mathbf{C}}}_t=\mathit{\varnothing }$, the identification of bad objects is complete.
• Step B11: Assume $s:=s+1$. If $s>K$, assume $s:=1$. Proceed to Step B2.

4. Numerical Illustration

The artificial sets of candidates for good and bad objects were generated randomly and are presented in Appendix A. Below, it is presented in detail how the sets of good and bad objects were generated for stage t = 1.

• Initial phase:

• Step I1: The following values were assumed for the number of good and bad objects: $n_1^{\mathbf{G}}=3$, $n_1^{\mathbf{B}}=3$.
• Step I2: ${\mathbf{G}}_1:=\mathit{\varnothing }$, ${\mathbf{B}}_1:=\mathit{\varnothing }$, $m_1^{\mathbf{G}}:=0$ and $m_1^{\mathbf{B}}:=0$.
• Step I3: The hierarchy of criteria was defined as $h_1^1=1$, $h_1^2=2$, $h_1^3=3$—the first criterion was the most important, the next was the second criterion and criterion no. 3 was the least important.

• Identification of the stage set of good objects for stage t = 1:

• Step G1: ${\tilde{\mathbf{C}}}_t:={\mathbf{C}}_t$, ${\mathbf{R}}_t≔\mathit{\varnothing },s=1$.
• Step G2: As $h_1^i= 1$ for $i=1$, $l=1$.
• Step G3: The object that achieved the highest score for the first criterion was ${\mathbf{c}}_{67}$.
• Step G4: As ${\mathbf{R}}_1=\mathit{\varnothing }$ and ${\mathbf{G}}_1=\mathit{\varnothing }$, the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_{67}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{67}\right)=100$, $f_1^2\left({\mathbf{c}}_{67}\right)=10$ and $f_1^3\left({\mathbf{c}}_{67}\right)=59$.
• Step G7: The decision-maker stated that, due to the insufficient value of the second criterion, ${\mathbf{c}}_{67}$ could not be considered a good object.
• Step G8: ${\mathbf{c}}_{67}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{67}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{67}\right\}$; the procedure proceeded to Step G3.
• Step G3: There were two objects in ${\tilde{\mathbf{C}}}_1$ that achieved the highest value of the first criterion, equal to 99: ${\mathbf{c}}_{62}$ and ${\mathbf{c}}_{85}$; since criterion no. 2 was the second most important according to the hierarchy defined by the decision-maker, we selected ${\mathbf{c}}_{62}$ as a candidate for a good object: $f_1^2\left({\mathbf{c}}_{62}\right)=85$ vs. $f_1^2\left({\mathbf{c}}_{85}\right)=37$.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: As ${\mathbf{c}}_{62}$ was not dominated by ${\mathbf{c}}_{67}$ (the only object in ${\mathbf{R}}_1$), the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_{62}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{62}\right)=99$, $f_1^2\left({\mathbf{c}}_{62}\right)=85$ and $f_1^3\left({\mathbf{c}}_{62}\right)=52$.
• Step G7: The decision-maker accepted ${\mathbf{c}}_{62}$ as a good object; the procedure proceeded to Step G9.
• Step G9: ${\mathbf{c}}_{62}$ was added to ${\mathbf{G}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{G}}_1:={\mathbf{G}}_1\cup \left\{{\mathbf{c}}_{62}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{62}\right\}$ and $m_1^{\mathbf{G}}=1$.
• Step G10: As $m_1^{\mathbf{G}}<n_1^{\mathbf{G}}$ and ${\tilde{\mathbf{C}}}_1\ne \mathit{\varnothing }$, the identification of good objects was continued.
• Step G11: $s:=2$; the procedure proceeded to Step G2.
• Step G2: As $h_1^i= 2$ for $i=2$, $l=2$.
• Step G3: The object that achieved the highest score for the second criterion was ${\mathbf{c}}_{53}$.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: As ${\mathbf{c}}_{53}$ was neither dominated by ${\mathbf{c}}_{67}$ (the only object in ${\mathbf{R}}_1$), nor by ${\mathbf{c}}_{62}$ (the only object in ${\mathbf{G}}_1$), the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_{53}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{53}\right)=38$, $f_1^2\left({\mathbf{c}}_{53}\right)=100$ and $f_1^3\left({\mathbf{c}}_{53}\right)=5$.
• Step G7: The decision-maker stated that, due to the insufficient value of the third criterion, ${\mathbf{c}}_{53}$ could not be considered a good object.
• Step G8: ${\mathbf{c}}_{53}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{53}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{53}\right\}$; the procedure proceeded to Step G3.
• Step G3: There were two objects in ${\tilde{\mathbf{C}}}_1$ achieving the highest value of the second criterion equal to 98: ${\mathbf{c}}_{17}$ and ${\mathbf{c}}_{35}$; taking into account the hierarchy of criteria, ${\mathbf{c}}_{35}$ was selected as a candidate for a good object.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: Object ${\mathbf{c}}_{35}$ was dominated by ${\mathbf{c}}_{53}\in {\mathbf{R}}_1:f_1^1\left({\mathbf{c}}_{35}\right)=34<f_1^1\left({\mathbf{c}}_{53}\right)=38$, $f_1^2\left({\mathbf{c}}_{35}\right)=98<f_1^2\left({\mathbf{c}}_{53}\right)=100$, $f_1^3\left({\mathbf{c}}_{35}\right)=1<f_1^3\left({\mathbf{c}}_{53}\right)=5$; thus, the procedure proceeded to Step G8.
• Step G8: ${\mathbf{c}}_{35}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{35}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{35}\right\}$; the procedure proceeded to Step G3.
• Step G3: The second object for which the second criterion was equal to 98 (${\mathbf{c}}_{17})$ was selected as a candidate for a good object.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: Object ${\mathbf{c}}_{17}$ was dominated by ${\mathbf{c}}_{53}\in {\mathbf{R}}_1:f_1^1\left({\mathbf{c}}_{17}\right)=5<f_1^1\left({\mathbf{c}}_{53}\right)=38$, $f_1^2\left({\mathbf{c}}_{17}\right)=98<f_1^2\left({\mathbf{c}}_{53}\right)=100$, $f_1^3\left({\mathbf{c}}_{17}\right)=4<f_1^3\left({\mathbf{c}}_{53}\right)=5$; thus, the procedure proceeded to Step G8.
• Step G8: ${\mathbf{c}}_{17}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{17}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{17}\right\}$; the procedure proceeded to Step G3.
• Step G3: The object that achieved the highest score for the second criterion was ${\mathbf{c}}_6$.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: As ${\mathbf{c}}_6$ was neither dominated by ${\mathbf{c}}_{62}$ (the only object in ${\mathbf{G}}_1$), nor by objects in ${\mathbf{B}}_1$(${\mathbf{c}}_{67},{\mathbf{c}}_{53},{\mathbf{c}}_{35},{\mathbf{c}}_{17})$, the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_6$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_6\right)=85$, $f_1^2\left({\mathbf{c}}_6\right)=95$, $f_1^3\left({\mathbf{c}}_6\right)=13$.
• Step G7: The decision-maker accepted ${\mathbf{c}}_6$ as a good object; the procedure proceeded to Step G9.
• Step G9: ${\mathbf{c}}_6$ was added to ${\mathbf{G}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{G}}_1:={\mathbf{G}}_1\cup \left\{{\mathbf{c}}_6\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_6\right\}$, $m_1^{\mathbf{G}}=2$.
• Step G10: As $m_1^{\mathbf{G}}<n_1^{\mathbf{G}}$ and ${\tilde{\mathbf{C}}}_1\ne \mathit{\varnothing }$, the identification of good objects was continued.
• Step G11: $s:=3$; procedure proceeded to Step G2.
• Step G2: As $h_1^i= 3$ for $i=3$, $l=3$.
• Step G3: The object that achieved the highest score for the second criterion was ${\mathbf{c}}_{54}$.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: As ${\mathbf{c}}_{54}$ was neither dominated by objects from ${\mathbf{G}}_1$ (${\mathbf{c}}_{62},{\mathbf{c}}_6$), nor by objects from ${\mathbf{B}}_1$(${\mathbf{c}}_{67},{\mathbf{c}}_{53},{\mathbf{c}}_{35},{\mathbf{c}}_{17})$, the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_{54}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{54}\right)=16$, $f_1^2\left({\mathbf{c}}_{54}\right)=77$, $f_1^3\left({\mathbf{c}}_{54}\right)=100$.
• Step G7: The decision-maker stated that, due to the insufficient value of the first criterion, ${\mathbf{c}}_{54}$ could not be considered a good object.
• Step G8: ${\mathbf{c}}_{54}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{54}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{54}\right\}$; the procedure proceeded to Step G3.
• Step G3: The object that achieved the highest score for the second criterion was ${\mathbf{c}}_{84}$.
• Step G4: As ${\mathbf{R}}_1\ne \mathit{\varnothing } \mathrm{a}\mathrm{n}\mathrm{d} {\mathbf{G}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step G5.
• Step G5: As ${\mathbf{c}}_{84}$ was neither dominated by objects from ${\mathbf{G}}_1$ (${\mathbf{c}}_{62},{\mathbf{c}}_6$), nor by objects from ${\mathbf{B}}_1$(${\mathbf{c}}_{67},{\mathbf{c}}_{53},{\mathbf{c}}_{35},{\mathbf{c}}_{17},{\mathbf{c}}_{54})$, the procedure proceeded to Step G6.
• Step G6: Object ${\mathbf{c}}_{84}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{84}\right)=67$, $f_1^2\left({\mathbf{c}}_{84}\right)=76$, $f_1^3\left({\mathbf{c}}_{84}\right)=98$.
• Step G7: The decision-maker accepted ${\mathbf{c}}_{84}$ as a good object; the procedure proceeded to Step G9.
• Step G9: ${\mathbf{c}}_{84}$ was added to ${\mathbf{G}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{G}}_1:={\mathbf{G}}_1\cup \left\{{\mathbf{c}}_{84}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{84}\right\}$, $m_1^{\mathbf{G}}=3$.
• Step G10: As $m_1^{\mathbf{G}}=n_1^{\mathbf{G}}$, the identification of good objects was completed.

• Identification of the stage set of bad objects for stage t = 1:

• Step B1: ${\tilde{\mathbf{C}}}_1:={\mathbf{C}}_1\backslash \left\{{\mathbf{c}}_{62},{\mathbf{c}}_6,{\mathbf{c}}_{84}\right\}$, ${\mathbf{R}}_1≔\mathit{\varnothing },s=1$.
• Step B2: As $h_1^i= 1$ for $i=1$, $l=1$.
• Step B3: The object that achieved the lowest score for the first criterion was ${\mathbf{c}}_{58}$.
• Step B4: As ${\mathbf{R}}_1=\mathit{\varnothing }$ and ${\mathbf{B}}_1=\mathit{\varnothing }$, the procedure proceeded to Step B6.
• Step B6: Object ${\mathbf{c}}_{58}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{58}\right)=1$, $f_1^2\left({\mathbf{c}}_{58}\right)=79$ and $f_1^3\left({\mathbf{c}}_{58}\right)=11$.
• Step B7: The decision-maker accepted ${\mathbf{c}}_{58}$ as a bad object; the procedure proceeded to Step B9.
• Step B9: ${\mathbf{c}}_{58}$ was added to ${\mathbf{B}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{B}}_1:={\mathbf{B}}_1\cup \left\{{\mathbf{c}}_{58}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{58}\right\}$, $m_1^{\mathbf{B}}=1$.
• Step B10: As $m_1^{\mathbf{B}}<n_1^{\mathbf{B}}$ and ${\tilde{\mathbf{C}}}_1\ne \mathit{\varnothing }$, the identification of bad objects was continued.
• Step B11: $s:=2$; the procedure proceeded to Step B2.
• Step B2: As $h_1^i= 2$ for $i=2$, $l=2$.
• Step B3: The object that achieved the lowest score for the second criterion was ${\mathbf{c}}_{55}$.
• Step B4: As ${\mathbf{B}}_1\ne \varnothing$, the procedure proceeded to Step B5.
• Step B5: As ${\mathbf{c}}_{55}$ did not dominate ${\mathbf{c}}_{58}$ (the only object in ${\mathbf{B}}_1$), the procedure proceeded to Step B6.
• Step B6: Object ${\mathbf{c}}_{55}$ was presented to the decision-maker: $f_1^1\left({\mathbf{c}}_{55}\right)=44$, $f_1^2\left({\mathbf{c}}_{55}\right)=1$, $f_1^3\left({\mathbf{c}}_{55}\right)=23$.
• Step B7: The decision-maker accepted ${\mathbf{c}}_{55}$ as a bad object; the procedure proceeded to Step B9.
• Step B9: ${\mathbf{c}}_{55}$ was added to ${\mathbf{B}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{B}}_1:={\mathbf{B}}_1\cup \left\{{\mathbf{c}}_{55}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{55}\right\}$, $m_1^{\mathbf{B}}=2$.
• Step B10: As $m_1^{\mathbf{B}}<n_1^{\mathbf{B}}$ and ${\tilde{\mathbf{C}}}_1\ne \mathit{\varnothing }$, the identification of bad objects was continued.
• Step B11: $s:=3$; the procedure proceeded to Step B2.
• Step B2: As $h_1^i= 3$ for $i=3$, $l=3$.
• Step B3: The object that achieved the lowest score for the third criterion was ${\mathbf{c}}_{35}$ (which had been previously considered as a candidate for a good object).
• Step B4: As ${\mathbf{B}}_1\ne \mathit{\varnothing }$ procedure proceeded to Step B5.
• Step B5: As ${\mathbf{c}}_{35}$ did not dominate any of the objects from ${\mathbf{B}}_1$ (${\mathbf{c}}_{58},{\mathbf{c}}_{55}$), the procedure proceeded to Step B6.
• Step B6: Object ${\mathbf{c}}_{54}$ was presented to the DM: $f_1^1\left({\mathbf{c}}_{35}\right)=34$, $f_1^2\left({\mathbf{c}}_{35}\right)=98$, $f_1^3\left({\mathbf{c}}_{35}\right)=1$.
• Step B7: The decision-maker stated that, due to the high value of the second criterion, ${\mathbf{c}}_{35}$ could not be considered a bad object.
• Step B8: ${\mathbf{c}}_{35}$ was added to ${\mathbf{R}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{R}}_1:={\mathbf{R}}_1\cup \left\{{\mathbf{c}}_{35}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{35}\right\}$; the procedure proceeded to Step B3.
• Step B3: There were three objects in ${\tilde{\mathbf{C}}}_1$ that achieved the lowest value of the third criterion, equal to 2: ${\mathbf{c}}_2,{\mathbf{c}}_{69}$ and ${\mathbf{c}}_{90}$; taking into account the hierarchy of criteria, ${\mathbf{c}}_{90}$ was selected as a candidate for a good object.
• Step B4: As ${\mathbf{B}}_1\ne \mathit{\varnothing }$, the procedure proceeded to Step B5.
• Step B5: As ${\mathbf{c}}_{90}$ did not dominate any of the objects from ${\mathbf{B}}_1$ (${\mathbf{c}}_{58},{\mathbf{c}}_{55}$), nor the object ${\mathbf{c}}_{35}$ (the only object from ${\mathbf{R}}_1$), the procedure proceeded to Step B6.
• Step B6: Object ${\mathbf{c}}_{90}$ was presented to the DM: $f_1^1\left({\mathbf{c}}_{90}\right)=6$, $f_1^2\left({\mathbf{c}}_{90}\right)=73$, $f_1^3\left({\mathbf{c}}_{90}\right)=2$.
• Step B7: The decision-maker accepted ${\mathbf{c}}_{90}$ as a bad object; the procedure proceeded to Step B9.
• Step B9: ${\mathbf{c}}_{90}$ was added to ${\mathbf{B}}_1$ and removed from ${\tilde{\mathbf{C}}}_1$: ${\mathbf{B}}_1:={\mathbf{B}}_1\cup \left\{{\mathbf{c}}_{90}\right\}$, ${\tilde{\mathbf{C}}}_1:={\tilde{\mathbf{C}}}_1\backslash \left\{{\mathbf{c}}_{90}\right\}$, $m_1^{\mathbf{B}}=3$.
• Step B10: As $m_1^{\mathbf{B}}=n_1^{\mathbf{B}}$, the identification of bad objects was complete.

The sets of good and bad objects for stage 1 are presented in

Table 1. The sets of good and bad objects for stage 1.

Good Objects				Bad Objects
Object	${\mathit{f}}_1^1\left(·\right)$	${\mathit{f}}_1^2\left(·\right)$	${\mathit{f}}_1^3\left(·\right)$	Object	${\mathit{f}}_1^1\left(·\right)$	${\mathit{f}}_1^2\left(·\right)$	${\mathit{f}}_1^3\left(·\right)$
${\mathbf{c}}_{62}\equiv {\mathbf{g}}_1^{\left(1\right)}$	99	85	52	${\mathbf{c}}_{58}\equiv {\mathbf{b}}_1^{\left(1\right)}$	1	79	11
${\mathbf{c}}_6\equiv {\mathbf{g}}_1^{\left(2\right)}$	85	95	13	${\mathbf{c}}_{55}\equiv {\mathbf{b}}_1^{\left(2\right)}$	44	1	23
${\mathbf{c}}_{84}\equiv {\mathbf{g}}_1^{\left(3\right)}$	67	76	98	${\mathbf{c}}_{90}\equiv {\mathbf{b}}_1^{\left(3\right)}$	6	73	2

. The same procedure was applied for identifying good and bad objects for stages 2 and 3. At each of these stages, the decision-maker set the number of good and bad objects at three. In stage two, the hierarchy of criteria was established as follows: the most important criterion was No. 2, followed by criterion No. 3, and the least important was criterion No. 1. In stage 3, the hierarchy was established as follows: criterion 3, criterion 1, criterion 2. The calculations were similar to those for stage 1: first, the object with the highest rating for the most important criterion was proposed, followed by the objects with the highest ratings for the subsequent criteria in accordance with the established hierarchy. The results are presented in

Table 2. The sets of good and bad objects for stage 2.

Good Objects				Bad Objects
Object	${\mathit{f}}_2^1\left(·\right)$	${\mathit{f}}_2^2\left(·\right)$	${\mathit{f}}_2^3\left(·\right)$	Object	${\mathit{f}}_2^1\left(·\right)$	${\mathit{f}}_2^2\left(·\right)$	${\mathit{f}}_2^3\left(·\right)$
${\mathbf{c}}_{10}\equiv {\mathbf{g}}_2^{\left(1\right)}$	66	99	92	${\mathbf{c}}_{88}\equiv {\mathbf{b}}_2^{\left(1\right)}$	51	2	35
${\mathbf{c}}_{86}\equiv {\mathbf{g}}_2^{\left(2\right)}$	64	40	96	${\mathbf{c}}_{18}\equiv {\mathbf{b}}_2^{\left(2\right)}$	52	28	1
${\mathbf{c}}_{70}\equiv {\mathbf{g}}_2^{\left(3\right)}$	99	76	57	${\mathbf{c}}_{100}\equiv {\mathbf{b}}_2^{\left(3\right)}$	1	23	34

and

Table 3. The sets of good and bad objects for stage 3.

Good Objects				Bad Objects
Object	${\mathit{f}}_3^1\left(·\right)$	${\mathit{f}}_3^2\left(·\right)$	${\mathit{f}}_3^3\left(·\right)$	Object	${\mathit{f}}_3^1\left(·\right)$	${\mathit{f}}_3^2\left(·\right)$	${\mathit{f}}_3^3\left(·\right)$
${\mathbf{c}}_{23}\equiv {\mathbf{g}}_3^{\left(1\right)}$	76	38	100	${\mathbf{c}}_{48}\equiv {\mathbf{b}}_3^{\left(1\right)}$	8	78	4
${\mathbf{c}}_{10}\equiv {\mathbf{g}}_3^{\left(2\right)}$	96	13	79	${\mathbf{c}}_{30}\equiv {\mathbf{b}}_3^{\left(2\right)}$	1	28	67
${\mathbf{c}}_{79}\equiv {\mathbf{g}}_3^{\left(3\right)}$	36	98	69	${\mathbf{c}}_6\equiv {\mathbf{b}}_3^{\left(3\right)}$	63	2	25

. In these tables, objects are denoted both according to the notation used above $\left({\mathbf{c}}_i\right)$ and according to the notation used below in Example 2 $\left({\mathbf{g}}_t^{\left(·\right)},{\mathbf{b}}_t^{\left(·\right)}\right)$.

5. Application in Resource Allocation in Spatial Development Planning Problem

5.1. Assumptions and Numerical Data

Three synthetic measures will be used, which play the roles of criteria in the Multistage Bipolar method:

• A measure of economic development (criterion 1);
• A measure of social development (criterion 2);
• A measure of environmental development (criterion 3).

We assume that each of them can take values on a scale from 0 to 100. We want to achieve the sustainable development goals set by the decision-maker as best as possible.

We will consider this issue as a multistage decision process, taking into account a time horizon consisting of short, medium and long periods. Our considerations will be based on the Multistage Bipolar method, in which we will use the stage “good” and “bad” objects generated in Example 1 by means of the interactive procedure proposed in this paper. These sets are shown for subsequent stages in Table 1, Table 2 and Table 3, respectively.

In Stage 1, the decision-maker places a particular emphasis on economic development; in Stage 2, on social development and in Stage 3, on environmental development. This is reflected in the adopted set of weights, presented in

Table 4. Criterion weights.

	Weights
Stage	Criterion 1	Criterion 2	Criterion 3
1	0.45	0.35	0.20
2	0.20	0.45	0.35
3	0.35	0.20	0.45

.

The achievable values of the synthetic measures considered depend on the amount of expenditure incurred. Four levels of expenditure, related to the orders considered, were estimated, a very low level, low level, medium level and high level, to which we assigned the numbers 0, 1, 2 and 3. The numerical values that we considered further are shown in

Table 5. Expenditures and values of synthetic measures.

		Values of Synthetic Measures
Level	Expenditures	Economic Development	Social Development	Environmental Development
0	Very low expenditure level	8	12	10
1	Low expenditure level	18	20	21
2	Medium expenditure level	43	42	45
3	High expenditure level	70	66	71

.

We assume that at the first stage, the total level of expenditures will be equal to 3. We can state 10 stage alternatives in stage 1: ${\mathbf{a}}_1^{\left(1\right)}$, …, ${\mathbf{a}}_1^{\left(10\right)}$. The stage alternative ${\mathbf{a}}_1^{\left(1\right)}$ has the form ${\mathbf{a}}_1^{\left(1\right)}=(3,0,0)$. It means that the level of expenditures for economic order is equal to 3 (high expenditure level), the level of expenditure for social development is equal to 0 (very low expenditure level) and the level of expenditures for environmental development is equal to 0 (very low expenditure level). From Table 5, we find that for this stage alternatives, the values of the synthetic measures are as follows:

$$f_1^1\left({\mathbf{a}}_1^{\left(1\right)}\right)=70, f_1^2\left({\mathbf{a}}_1^{\left(1\right)}\right)=12, f_1^3\left({\mathbf{a}}_1^{\left(1\right)}\right)=8.$$

In a similar way, we interpret the stage alternatives, ${\mathbf{a}}_1^{\left(2\right)}$, …, ${\mathbf{a}}_1^{\left(10\right)}$. All these stage alternatives are shown in

Table A4. Scenarios of development (stage alternatives) for stage 1.

	Expenditure Levels			$\mathbf{Values} \mathbf{of} \mathbf{Synthetic} \mathbf{Measures} {\mathit{f}}_1^k(\cdot )$
Stage Alternatives Stage 1	Economic Order	Social Order	Environmental Order	Economic Development (k = 1)	Social Development (k = 2)	Environmental Development (k = 3)
${\mathbf{a}}_1^{\left(1\right)}$	3	0	0	70	12	10
${\mathbf{a}}_1^{\left(2\right)}$	2	1	0	43	20	10
${\mathbf{a}}_1^{\left(3\right)}$	1	2	0	18	42	10
${\mathbf{a}}_1^{\left(4\right)}$	0	3	0	8	66	10
${\mathbf{a}}_1^{\left(5\right)}$	2	0	1	43	12	21
${\mathbf{a}}_1^{\left(6\right)}$	1	1	1	18	20	21
${\mathbf{a}}_1^{\left(7\right)}$	0	2	1	8	42	21
${\mathbf{a}}_1^{\left(8\right)}$	0	1	2	8	20	45
${\mathbf{a}}_1^{\left(9\right)}$	1	0	2	18	12	45
${\mathbf{a}}_1^{\left(10\right)}$	0	0	3	8	12	71

(see Appendix B).

We assume that, in the second stage, the total level of expenditures will be equal to 4. We can state 12 stage alternatives: ${\mathbf{a}}_2^{\left(1\right)}$, …, ${\mathbf{a}}_2^{\left(12\right)}$. The stage alternative ${\mathbf{a}}_2^{\left(1\right)}$ has the form ${\mathbf{a}}_2^{\left(1\right)}=(3, 1, 0)$. From Table 5 we find that, for this stage alternative, the values of the synthetic measures are as follows:

$$f_2^1\left({\mathbf{a}}_2^{\left(1\right)}\right)=70, f_2^2\left({\mathbf{a}}_2^{\left(1\right)}\right)=20, f_2^3\left({\mathbf{a}}_2^{\left(1\right)}\right)=8$$

In a similar way, we interpret the stage alternatives ${\mathbf{a}}_2^{\left(2\right)}$, …, ${\mathbf{a}}_2^{\left(12\right)}$. All these stage alternatives are shown in

Table A5. Scenarios of development (stage alternatives) for stage 2.

	Expenditure Levels			$\mathbf{Values} \mathbf{of} \mathbf{Synthetic} \mathbf{Measures} {\mathit{f}}_2^k(\cdot )$
Stage Alternatives Stage 2	Economic Order	Social Order	Environmental Order	Economic Development (k = 1)	Social Development (k = 2)	Environmental Development (k = 3)
${\mathbf{a}}_2^{\left(1\right)}$	3	1	0	70	20	10
${\mathbf{a}}_2^{\left(2\right)}$	2	2	0	43	42	10
${\mathbf{a}}_2^{\left(3\right)}$	1	3	0	18	66	10
${\mathbf{a}}_2^{\left(4\right)}$	3	0	1	70	12	21
${\mathbf{a}}_2^{\left(5\right)}$	2	1	1	43	20	21
${\mathbf{a}}_2^{\left(6\right)}$	1	2	1	18	42	21
${\mathbf{a}}_2^{\left(7\right)}$	0	3	1	8	66	21
${\mathbf{a}}_2^{\left(8\right)}$	2	0	2	43	12	45
${\mathbf{a}}_2^{\left(9\right)}$	1	1	2	18	20	45
${\mathbf{a}}_2^{\left(10\right)}$	0	2	2	8	42	45
${\mathbf{a}}_2^{\left(11\right)}$	1	0	3	18	12	71
${\mathbf{a}}_2^{\left(12\right)}$	0	1	3	8	20	71

(see Appendix B).

We assume that in the third stage, the total level of expenditures will be equal to 5. We can state 12 stage alternatives: ${\mathbf{a}}_3^{\left(1\right)}$, …, ${\mathbf{a}}_3^{\left(12\right)}$. The stage alternative ${\mathbf{a}}_3^{\left(1\right)}$ has the form ${\mathbf{a}}_3^{\left(1\right)}=(3, 2, 0)$. From Table 5 we find that for this stage alternative, the values of the synthetic measures are as follows:

$$f_3^1\left({\mathbf{a}}_3^{\left(1\right)}\right)=70, f_3^2\left({\mathbf{a}}_3^{\left(1\right)}\right)=20, f_3^3\left({\mathbf{a}}_3^{\left(1\right)}\right)=8$$

In a similar way, we interpret the stage alternatives ${\mathbf{a}}_3^{\left(2\right)}$, …, ${\mathbf{a}}_3^{\left(12\right)}$. All these stage alternatives are shown in

Table A6. Scenarios of development (stage alternatives) for stage 3.

	Expenditure Levels			$\mathbf{Values} \mathbf{of} \mathbf{Synthetic} \mathbf{Measures} {\mathit{f}}_3^k(\cdot )$
Stage Alternatives Stage 3	Economic Order	Social Order	Environmental Order	Economic Development (k = 1)	Social Development (k = 2)	Environmental Development (k = 3)
${\mathbf{a}}_3^{\left(1\right)}$	3	2	0	70	42	10
${\mathbf{a}}_3^{\left(2\right)}$	2	3	0	43	66	10
${\mathbf{a}}_3^{\left(3\right)}$	3	1	1	70	20	21
${\mathbf{a}}_3^{\left(4\right)}$	2	2	1	43	42	21
${\mathbf{a}}_3^{\left(5\right)}$	1	3	1	18	66	21
${\mathbf{a}}_3^{\left(6\right)}$	3	0	2	70	12	45
${\mathbf{a}}_3^{\left(7\right)}$	2	1	2	43	20	45
${\mathbf{a}}_3^{\left(8\right)}$	1	2	2	18	42	45
${\mathbf{a}}_3^{\left(9\right)}$	0	3	2	8	66	45
${\mathbf{a}}_3^{\left(10\right)}$	2	0	3	43	12	71
${\mathbf{a}}_3^{\left(11\right)}$	1	1	3	18	20	71
${\mathbf{a}}_3^{\left(12\right)}$	0	2	3	8	42	71

(see Appendix B).

In Example 2, each multistage alternative is defined as follows:

$$\mathbf{a}=\left({\mathbf{a}}_{1, }{\mathbf{a}}_{2, }{\mathbf{a}}_3\right)$$

(see Formula 1). As

$$card {\mathbf{A}}_{\mathrm{1}}=10, card {\mathbf{A}}_2=12, card {\mathbf{A}}_3=12,$$

we have a total of 10 × 12 × 12 = 1440 multistage alternatives, which we number from 1 to 1440. We arrange them in lexicographic order, shown in

Table A7. Multistage alternatives.

Number of Multistage Alternatives	Stage 1	Stage 2	Stage 3	Number of Multistage Alternatives	Stage 1	Stage 2	Stage3
${\mathbf{a}}^{\left(1\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	…	…	…	…
${\mathbf{a}}^{\left(2\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(2\right)}$	${\mathbf{a}}^{\left(1426\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(11\right)}$	${\mathbf{a}}_3^{\left(10\right)}$
${\mathbf{a}}^{\left(3\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(3\right)}$	${\mathbf{a}}^{\left(1427\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(11\right)}$	${\mathbf{a}}_3^{\left(11\right)}$
${\mathbf{a}}^{\left(4\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(4\right)}$	${\mathbf{a}}^{\left(1428\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(11\right)}$	${\mathbf{a}}_3^{\left(12\right)}$
${\mathbf{a}}^{\left(5\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(5\right)}$	${\mathbf{a}}^{\left(1429\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(1\right)}$
${\mathbf{a}}^{\left(6\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(6\right)}$	${\mathbf{a}}^{\left(1430\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(2\right)}$
${\mathbf{a}}^{\left(7\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(7\right)}$	${\mathbf{a}}^{\left(1431\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(3\right)}$
${\mathbf{a}}^{\left(8\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(8\right)}$	${\mathbf{a}}^{\left(1432\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(4\right)}$
${\mathbf{a}}^{\left(9\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(9\right)}$	${\mathbf{a}}^{\left(1433\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(5\right)}$
${\mathbf{a}}^{\left(10\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(10\right)}$	${\mathbf{a}}^{\left(1434\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(6\right)}$
${\mathbf{a}}^{\left(11\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(11\right)}$	${\mathbf{a}}^{\left(1435\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(7\right)}$
${\mathbf{a}}^{\left(12\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(1\right)}$	${\mathbf{a}}_3^{\left(12\right)}$	${\mathbf{a}}^{\left(1436\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(8\right)}$
${\mathbf{a}}^{\left(13\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	${\mathbf{a}}^{\left(1437\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(9\right)}$
${\mathbf{a}}^{\left(14\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	${\mathbf{a}}^{\left(1438\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(10\right)}$
${\mathbf{a}}^{\left(15\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	${\mathbf{a}}^{\left(1439\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(11\right)}$
…	…	…	…	${\mathbf{a}}^{\left(1440\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(12\right)}$	${\mathbf{a}}_3^{\left(12\right)}$

(see Appendix B).

For example, let us consider the multistage alternative ${\mathbf{a}}^{\left(15\right)}=\left[{\mathbf{a}}_1^{\left(1\right)},{\mathbf{a}}_2^{\left(2\right)},{\mathbf{a}}_3^{\left(1\right)}\right]$. This development scenario is characterized by the following:

• High expenditures in economic development in stage 1, medium expenditures in economic development in stage 2 and high expenditures in economic development in stage 3;
• Very low expenditures in social development in stage 1, medium expenditures in social development in stage 2 and medium expenditures in social development in stage 3;
• Very low expenditures in environmental development at stage 1, very low expenditures in environmental development in stage 2 and very low expenditures in environmental development in stage 3.

The decision-maker applied the interactive method described in the previous chapters of the paper and obtained (Example 1) three good objects and three bad objects at each stage. We have the following stage sets of good and bad objects:

$${\mathbf{G}}_1=\{{\mathbf{g}}_1^{\left(1\right)}, {\mathbf{g}}_1^{\left(2\right)},{\mathbf{g}}_1^{\left(3\right)}\} {\mathbf{B}}_1=\{{\mathbf{b}}_1^{\left(1\right)}, {\mathbf{b}}_1^{\left(2\right)},{\mathbf{b}}_1^{\left(3\right)}\}$$

$${\mathbf{G}}_2=\{{\mathbf{g}}_2^{\left(1\right)}, {\mathbf{g}}_2^{\left(2\right)},{\mathbf{g}}_2^{\left(3\right)}\} {\mathbf{B}}_2=\{{\mathbf{b}}_2^{\left(1\right)}, {\mathbf{b}}_2^{\left(2\right)},{\mathbf{b}}_2^{\left(3\right)}\}$$

$${\mathbf{G}}_3=\{{\mathbf{g}}_3^{\left(1\right)}, {\mathbf{g}}_3^{\left(2\right)},{\mathbf{g}}_3^{\left(3\right)}\} {\mathbf{B}}_3=\{{\mathbf{b}}_3^{\left(1\right)}, {\mathbf{b}}_3^{\left(2\right)},{\mathbf{b}}_3^{\left(3\right)}\}$$

The values of the stage criteria are presented in Table 1, Table 2 and Table 3, respectively.

5.2. Using the Multistage Bipolar Method to Select the Best Multistage Scenario

Step 1: We compare the stage alternative ${\mathbf{a}}_t$ with the stage reference objects ${\mathbf{r}}_t$, where ${\mathbf{r}}_t$ is taken as the objects ${\mathbf{b}}_t$ and ${\mathbf{g}}_t$, respectively. For each criterion k (k = 1, …, K), we calculate the differences:

$${\mathit{⸹}}_t^k\left({\mathbf{a}}_t,{\mathbf{r}}_t\right)= f_t^k\left({\mathbf{a}}_t\right)-f_t^k\left({\mathbf{r}}_t\right)$$

(12)

Next we calculate values:

$$S({\mathit{⸹}}_t^k({\mathbf{a}}_t,{\mathbf{r}}_t))=\{\begin{array}{cc}0& if {\mathit{⸹}}_t^k\left({\mathbf{a}}_t,{\mathbf{r}}_t\right)\le 0\\ 1& if {\mathit{⸹}}_t^{\left(k\right)}\left({\mathbf{a}}_t,{\mathbf{r}}_t\right)>0\end{array}$$

(13)

If $S({\mathit{⸹}}_t^k({\mathbf{a}}_t,{\mathbf{r}}_t))=1,$it means that the stage alternative ${\mathbf{a}}_t$ prevails over the stage reference object ${\mathbf{r}}_t$ with respect to the k-th criterion.

Comparing ${\mathbf{r}}_t$ with ${\mathbf{a}}_t$ we apply the following formula:

$${S(\mathit{⸹}}_t^k({\mathbf{r}}_t,{\mathbf{a}}_t))=1-S({\mathit{⸹}}_t^k({\mathbf{a}}_t,{\mathbf{r}}_t))$$

(14)

If ${S(\mathit{⸹}}_t^k({\mathbf{r}}_t,{\mathbf{a}}_t))=1,$it means that the stage reference object ${\mathbf{r}}_t$ prevails over the stage alternative ${\mathbf{a}}_t$ with respect to the k-th criterion.

Using stage weights from Table 4, we calculate the following:

$${\mathsf{\Psi }}_t\left({\mathbf{a}}_t,{\mathbf{r}}_t\right)=\sum _{k=1}^K{w}_t^k{S(\mathit{⸹}}_t^k({\mathbf{a}}_t,{\mathbf{r}}_t))$$

(15)

According to Equation (14), the following relationship occurs:

$${{\mathsf{\Psi }}_t\left({\mathbf{r}}_t,{\mathbf{a}}_t\right)=1-\mathsf{\Psi }}_t({\mathbf{a}}_t,{\mathbf{r}}_t)$$

(16)

The value of ${\mathsf{\Psi }}_t({\mathbf{a}}_t$,${\mathbf{r}}_t$) determines the degree of advantage of ${\mathbf{a}}_t$ over ${\mathbf{r}}_t$, while ${\mathsf{\Psi }}_t({\mathbf{r}}_t$,${\mathbf{a}}_t$) determines the degree of advantage of ${\mathbf{r}}_t$ over ${\mathbf{a}}_t$. The calculated values of the functions ${\mathsf{\Psi }}_t({\mathbf{a}}_t$,${\mathbf{b}}_t$) and ${\mathsf{\Psi }}_t({\mathbf{a}}_t$,${\mathbf{g}}_t$) for t = 1, 2, 3 are presented in

Table A8. Values ${\mathsf{\Psi }}_t({\mathbf{a}}_t$,${\mathbf{b}}_t$) and ${\mathsf{\Psi }}_t({\mathbf{a}}_t$,${\mathbf{g}}_t$) for t = 1, 2, 3.

Stage 1
$\left({\mathbf{a}}_1,{\mathbf{b}}_1\right)$	${\mathsf{\Psi }}_1\left(·\right)$	$\left({\mathbf{a}}_1,{\mathbf{g}}_1\right)$	${\mathsf{\Psi }}_1\left(·\right)$	$\left({\mathbf{a}}_1,{\mathbf{b}}_1\right)$	${\mathsf{\Psi }}_1\left(·\right)$	$\left({\mathbf{a}}_1,{\mathbf{g}}_1\right)$	${\mathsf{\Psi }}_1\left(·\right)$
$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.8	$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(1\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0.45	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(6\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(2\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(7\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(3\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(8\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(4\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(9\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{b}}_1^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{g}}_1^{\left(1\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{b}}_1^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{g}}_1^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(5\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{b}}_1^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_1^{\left(10\right)},{\mathbf{g}}_1^{\left(3\right)}\right)$	0
Stage 2
$\left({\mathbf{a}}_{\mathbf{2}},{\mathbf{b}}_{\mathbf{2}}\right)$	${\mathsf{\Psi }}_{\mathbf{2}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{2}},{\mathbf{g}}_{\mathbf{2}}\right)$	${\mathsf{\Psi }}_{\mathbf{2}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{2}},{\mathbf{b}}_{\mathbf{2}}\right)$	${\mathsf{\Psi }}_{\mathbf{2}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{2}},{\mathbf{g}}_{\mathbf{2}}\right)$	${\mathsf{\Psi }}_{\mathbf{2}}\left(·\right)$
$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.45
$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(1\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(7\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(2\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(8\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(3\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(9\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.45
$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(4\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(10\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(5\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(11\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0.35
$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{b}}_2^{\left(1\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{g}}_2^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0.45	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{b}}_2^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{g}}_2^{\left(2\right)}\right)$	0
$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.65	$\left({\mathbf{a}}_2^{\left(6\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{b}}_2^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_2^{\left(12\right)},{\mathbf{g}}_2^{\left(3\right)}\right)$	0.35
Stage 3
$\left({\mathbf{a}}_{\mathbf{3}},{\mathbf{b}}_{\mathbf{3}}\right)$	${\mathsf{\Psi }}_{\mathbf{3}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{3}},{\mathbf{g}}_{\mathbf{3}}\right)$	${\mathsf{\Psi }}_{\mathbf{3}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{3}},{\mathbf{b}}_{\mathbf{3}}\right)$	${\mathsf{\Psi }}_{\mathbf{3}}\left(·\right)$	$\left({\mathbf{a}}_{\mathbf{3}},{\mathbf{g}}_{\mathbf{3}}\right)$	${\mathsf{\Psi }}_{\mathbf{3}}\left(·\right)$
$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	1	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(1\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(7\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35
$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(2\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(8\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(3\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(9\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(4\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(10\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.8
$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0
$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(5\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(11\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.45
$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{b}}_3^{\left(1\right)}\right)$	0.2	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{g}}_3^{\left(1\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{b}}_3^{\left(2\right)}\right)$	0	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{g}}_3^{\left(2\right)}\right)$	0.2
$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(6\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.35	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{b}}_3^{\left(3\right)}\right)$	0.55	$\left({\mathbf{a}}_3^{\left(12\right)},{\mathbf{g}}_3^{\left(3\right)}\right)$	0.45

(see Appendix B).

Step 2: To compare the stage alternatives with the set of stage reference bad objects, we calculate the flow values:

• Positive:

$${\mathsf{\Phi }}_{{\mathbf{R}}_t}^{+}\left({\mathbf{a}}_t\right)={\displaystyle \frac{1}{r_t}}{\sum }_{{\mathbf{r}}_t\in {\mathbf{R}}_t}\mathsf{\Psi }({\mathbf{a}}_t,{\mathbf{r}}_t)$$

(17)

• Negative:

$${\mathsf{\Phi }}_{{\mathbf{R}}_t}^{-}\left({\mathbf{a}}_t\right)={\displaystyle \frac{1}{r_t}}{\sum }_{{\mathbf{r}}_t\in {\mathbf{R}}_t}\mathsf{\Psi }({\mathbf{r}}_t,{\mathbf{a}}_t)$$

(18)

• Net:

$${\mathsf{\Phi }}_{{\mathbf{R}}_t}\left({\mathbf{a}}_t\right)={\mathsf{\Phi }}_{{\mathbf{R}}_t}^{+}\left({\mathbf{a}}_t\right)-{\mathsf{\Phi }}_{{\mathbf{R}}_t}^{-}\left({\mathbf{a}}_t\right)$$

(19)

where $r_t$ denotes the cardinality of the corresponding reference set ${\mathbf{R}}_t$. The calculation results for net flows are presented in

Table A9. Values of ${\mathsf{\Phi }}_{{\mathbf{B}}_t}\left({\mathbf{a}}_t^{\left(i\right)}\right)$ and ${\mathsf{\Phi }}_{{\mathbf{G}}_t}\left({\mathbf{a}}_t^{\left(i\right)}\right)$.

Stage 1			Stage 2			Stage 3
${\mathbf{a}}_1$	${\Phi }_{{\mathbf{B}}_1}\left({\mathbf{a}}_1^{\left(i\right)}\right)$	${\Phi }_{{\mathbf{G}}_1}\left({\mathbf{a}}_1^{\left(i\right)}\right)$	${\mathbf{a}}_2$	${\Phi }_{{\mathbf{B}}_2}\left({\mathbf{a}}_1^{\left(i\right)}\right)$	${\Phi }_{{\mathbf{G}}_2}\left({\mathbf{a}}_2^{\left(i\right)}\right)$	${\mathbf{a}}_3$	${\Phi }_{{\mathbf{B}}_3}\left({\mathbf{a}}_3^{\left(i\right)}\right)$	${\Phi }_{{\mathbf{G}}_3}\left({\mathbf{a}}_3^{\left(i\right)}\right)$
${\mathbf{a}}_1^{\left(1\right)}$	0.27	−0.7	${\mathrm{a}}_2^{\left(1\right)}$	−0.3	−0.7	${\mathrm{a}}_3^{\left(1\right)}$	−0.0333	−0.5
${\mathrm{a}}_1^{\left(2\right)}$	−0	−1	${\mathrm{a}}_2^{\left(2\right)}$	0.03	−0.7	${\mathrm{a}}_3^{\left(2\right)}$	−0.2667	−0.5
${\mathrm{a}}_1^{\left(3\right)}$	−0	−1	${\mathrm{a}}_2^{\left(3\right)}$	0.03	−0.7	${\mathrm{a}}_3^{\left(3\right)}$	−0.1667	−0.6333
${\mathrm{a}}_1^{\left(4\right)}$	−0	−1	${\mathrm{a}}_2^{\left(4\right)}$	−0.3	−0.7	${\mathrm{a}}_3^{\left(4\right)}$	−0.2667	−0.5
${\mathrm{a}}_1^{\left(5\right)}$	0.1	−0.87	${\mathrm{a}}_2^{\left(5\right)}$	−0.6	−1	${\mathrm{a}}_3^{\left(5\right)}$	−0.2667	−0.7333
${\mathrm{a}}_1^{\left(6\right)}$	0.1	−0.87	${\mathrm{a}}_2^{\left(6\right)}$	0.03	−0.7	${\mathrm{a}}_3^{\left(6\right)}$	−0.1667	−0.7667
${\mathrm{a}}_1^{\left(7\right)}$	0.1	−0.87	${\mathrm{a}}_2^{\left(7\right)}$	0.03	−0.7	${\mathrm{a}}_3^{\left(7\right)}$	−0.4	−0.6333
${\mathrm{a}}_1^{\left(8\right)}$	0.23	−0.87	${\mathrm{a}}_2^{\left(8\right)}$	−0.6	−1	${\mathrm{a}}_3^{\left(8\right)}$	−0.2667	−0.7333
${\mathrm{a}}_1^{\left(9\right)}$	0.23	−0.87	${\mathrm{a}}_2^{\left(9\right)}$	−0.6	−1	${\mathrm{a}}_3^{\left(9\right)}$	−0.5	−0.7333
${\mathrm{a}}_1^{\left(10\right)}$	0.23	−0.73	${\mathrm{a}}_2^{\left(10\right)}$	0.03	−0.7	${\mathrm{a}}_3^{\left(10\right)}$	−0.4	−0.4667
			${\mathrm{a}}_2^{\left(11\right)}$	−0.6	−0.8	${\mathrm{a}}_3^{\left(11\right)}$	−0.4	−0.5667
			${\mathrm{a}}_2^{\left(12\right)}$	−0.6	−0.8	${\mathrm{a}}_3^{\left(12\right)}$	−0.5	−0.4333

(see Appendix B).

Step 3: We determine the position of multistage alternatives by calculating the following for each multistage alternative:

• The multistage failure avoidance degree:

$${\mathsf{\Phi }}_{\mathbf{B}}\left(\mathbf{a}\right)={\displaystyle \frac{1}{T}}\sum _{t=1}^T{\mathsf{\Phi }}_{{\mathbf{B}}_t}\left({\mathbf{a}}_t\right)$$

(20)

• The multistage success achievement degree:

$${\mathsf{\Phi }}_{\mathbf{G}}\left(\mathbf{a}\right)={\displaystyle \frac{1}{T}}\sum _{t=1}^T{\mathsf{\Phi }}_{{\mathbf{G}}_t}\left({\mathbf{a}}_t\right)$$

(21)

Step 4: We sort multistage alternatives ${\mathbf{a}}^{\left(i\right)}$, assigning them to predefined classes:

$${\mathbf{A}}^H=\left\{\mathbf{a}\in \mathbf{A}:{\Phi }_{\mathbf{B}}\left(\mathbf{a}\right)\ge 0,{\Phi }_{\mathbf{G}}\left(\mathbf{a}\right)\ge 0\right\}$$

(22)

$${\mathbf{A}}^M=\left\{\mathbf{a}\in \mathbf{A}:{\Phi }_{\mathbf{B}}\left(\mathbf{a}\right)\ge 0,{\Phi }_{\mathbf{G}}\left(\mathbf{a}\right)<0\right\},$$

(23)

$${\mathbf{A}}^L=\left\{\mathbf{a}\in \mathbf{A}:{\Phi }_{\mathbf{B}}\left(\mathbf{a}\right)<0,{\Phi }_{\mathbf{G}}\left(\mathbf{a}\right)<0\right\}.$$

(24)

Using the calculated values ${\mathsf{\Phi }}_{\mathbf{B}}\left({\mathbf{a}}^{\left(i\right)}\right)$, ${\mathsf{\Phi }}_{\mathbf{G}}\left({\mathbf{a}}^{\left(i\right)}\right)$, we state that set A^H is empty and all the multistage alternatives belong to classes A^M or A^L.

Step 5: Ranking multistage alternatives. To find the best multistage alternative, we calculate the values d(${\mathbf{a}}^{\left(i\right)}$):

$$d\left(\mathbf{a}\right)={\Phi }_{\mathbf{G}}\left(\mathbf{a}\right)+{\Phi }_{\mathbf{B}}\left(\mathbf{a}\right)$$

(25)

for consecutive multistage alternatives from class ${\mathbf{A}}^M.$The 20 best-rated multistage alternatives are shown in

Table 6. Ranking multistage alternatives.

No.	$\mathbf{a}$	${\mathbf{a}}_1$	${\mathbf{a}}_2$	${\mathbf{a}}_3$	${\mathsf{\Phi }}_{\mathbf{B}}\left(\mathbf{a}\right)$	${\mathsf{\Phi }}_{\mathit{G}}\left(\mathbf{a}\right)$	d(a)
1	${\mathbf{a}}^{\left(13\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6333	0.0889	−0.5444
2	${\mathbf{a}}^{\left(25\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(3\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6333	0.0889	−0.5444
3	${\mathbf{a}}^{\left(61\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(6\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6333	0.0889	−0.5444
4	${\mathbf{a}}^{\left(73\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(7\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6333	0.0889	−0.5444
5	${\mathbf{a}}^{\left(109\right)}$	${\mathbf{a}}_1^{\left(1\right)}$	${\mathbf{a}}_2^{\left(10\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6333	0.0889	−0.5444
6	${\mathbf{a}}^{\left(1021\right)}$	${\mathbf{a}}_1^{\left(8\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
7	${\mathbf{a}}^{\left(1033\right)}$	${\mathbf{a}}_1^{\left(8\right)}$	${\mathbf{a}}_2^{\left(3\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
8	${\mathbf{a}}^{\left(1069\right)}$	${\mathbf{a}}_1^{\left(8\right)}$	${\mathbf{a}}_2^{\left(6\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
9	${\mathbf{a}}^{\left(1081\right)}$	${\mathbf{a}}_1^{\left(8\right)}$	${\mathbf{a}}_2^{\left(7\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
10	${\mathbf{a}}^{\left(1117\right)}$	${\mathbf{a}}_1^{\left(8\right)}$	${\mathbf{a}}_2^{\left(10\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
11	${\mathbf{a}}^{\left(1165\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
12	${\mathbf{a}}^{\left(1177\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(3\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
13	${\mathbf{a}}^{\left(1213\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(6\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
14	${\mathbf{a}}^{\left(1225\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(7\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
15	${\mathbf{a}}^{\left(1261\right)}$	${\mathbf{a}}_1^{\left(9\right)}$	${\mathbf{a}}_2^{\left(10\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
16	${\mathbf{a}}^{\left(1309\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(2\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
17	${\mathbf{a}}^{\left(1321\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(3\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
18	${\mathbf{a}}^{1357}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(6\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
19	${\mathbf{a}}^{\left(1369\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(7\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111
20	${\mathbf{a}}^{\left(1405\right)}$	${\mathbf{a}}_1^{\left(10\right)}$	${\mathbf{a}}_2^{\left(10\right)}$	${\mathbf{a}}_3^{\left(1\right)}$	−0.6889	0.0778	−0.6111

.

5.3. Analysis of Numerical Results

When presenting the issue of planning the distribution of funds for sustainable development of the region, 1440 scenarios were considered. This number is the result of combining successive stage scenarios from earlier stages with all possible stage scenarios in later stages. The set of the best alternatives consists of five elements. They are the following multistage alternatives:

$${\mathbf{a}}^{\left(13\right)}= [{\mathbf{a}}_1^{\left(1\right)}, {\mathbf{a}}_2^{\left(2\right)}, {\mathbf{a}}_3^{\left(1\right)}]=[\left(3, 0, 0\right), \left(2, 2, 0\right), \left(3, 2, 0\right)],$$

$${\mathbf{a}}^{\left(25\right)}=[{\mathbf{a}}_1^{\left(1\right)},{\mathbf{a}}_2^{\left(3\right)}, {\mathbf{a}}_3^{\left(1\right)}]=\left[\left(3, 0, 0\right), \left(1, 3, 0\right), \left(3, 2, 0\right)\right],$$

$${\mathbf{a}}^{\left(61\right)}=[{\mathbf{a}}_1^{\left(1\right)},{\mathbf{a}}_2^{\left(6\right)}, {\mathbf{a}}_3^{\left(1\right)}]=[\left(3, 0, 0\right), \left(1, 2, 1\right), \left(3, 2, 0\right)],$$

$${\mathbf{a}}^{\left(73\right)}=[{\mathbf{a}}_1^{\left(1\right)},{\mathbf{a}}_2^{\left(7\right)},{\mathbf{a}}_3^{\left(1\right)}]=[\left(3, 0, 0\right), \left(0, 3, 1\right), \left(3, 2, 0\right)],$$

$${\mathbf{a}}^{\left(109\right)}=[{\mathbf{a}}_1^{\left(1\right)},{\mathbf{a}}_2^{\left(10\right)}, {\mathbf{a}}_3^{\left(1\right)}]=[\left(3, 0, 0\right), \left(0, 2, 2\right), \left(3, 2, 0\right)].$$

In all the best solutions, expenditures on economic development are preferred. The most balanced multistage alternative appears to be multistage alternative ${\mathbf{a}}^{\left(109\right)}$. This development scenario is characterized by the following:

• High expenditures in economic development in stage 1, very low expenditures in economic development in stage 2 and high expenditures in economic development in stage 3;
• Very low expenditures in social development in stage 1, medium expenditures in social development in stage 2 and medium expenditures in social development in stage 3;
• Very low expenditures in environmental development at stage 1, medium expenditure development in stage 2 and very low expenditures in environmental development in stage 3.

The calculations presented in this paper were performed using an Excel spreadsheet, a single set of weights and a single set of stage reference sets were utilized. It is expected that modifying the values of these weights and preferences regarding the development sequence would yield different development scenarios, which should be subjected to further analysis. It would be interesting to investigate the sensitivity of the solution to changing weights and sets of reference objects, which would require the development of specialized software.

6. Conclusions

This paper presents a synthesis of three streams related to multi-criteria decision-making: the reference point-based approach, the interactive approach and multistage decision processes. Particular attention is devoted to the interactive selection of objects forming the stage reference sets. The proposed procedure represents a first proposal for solving this problem and may be modified depending on decision-makers’ opinions.

The procedure is based on an interactive approach. This approach inherently results in both some advantages and limitations in its application. Its primary advantage is its simplicity.

In each iteration, the decision-maker is presented with a specific object, which they can either accept as “good” or “bad,” or indicate why it cannot be classified as such. In the latter case, the information provided by the decision-maker is subsequently used to determine further proposals. Consequently, the decision-maker does not have to provide all the information necessary to determine a solution at once, but can articulate it step by step. By monitoring the results, the decision-maker expands their knowledge regarding the essence of the problem being solved.

However, the use of this type of approach has its limitations. It may happen that the decision-maker’s expectations regarding the objects to be considered “good” or “bad” are too high, resulting in a failure to obtain the assumed number of reference objects. In such a case, it would be necessary to restart the procedure, which could, of course, lower the decision-maker’s confidence in the obtained results.

It should also be remembered that the fundamental assumption of the procedure is the availability of sets of candidates from which “good” or “bad” objects are selected. In the presented example, this set was generated randomly, based on knowledge of the value ranges within which the analyzed criteria fall. However, objects generated in this manner may be perceived by the decision-maker as differing too much from the alternatives they might encounter in reality. If the problem is repetitive—meaning it has been solved previously for a different data set—a better solution may be to utilize historical data.

The problem addressed constitutes a mathematical economics model, while simultaneously demonstrating that multi-criteria methods are widely applicable in management. When analyzing the specific nature of managerial decision-making problems, attention is drawn to their multidimensional character. Typically, they require evaluation based on multiple criteria of both a quantitative and qualitative nature.

This situation is also encountered in public management. In striving to effectively fulfill their tasks, public administration units must employ methods proven in the private sector. Of fundamental importance here is the expenditure of public funds in a manner that yields maximum benefits for citizens.