*Article* **Probabilistic Linguistic Preference Relation-Based Decision Framework for Multi-Attribute Group Decision Making**

#### **R. Krishankumar 1, K. S. Ravichandran 1, M. Ifjaz Ahmed 1, Samarjit Kar 2,\* and Sanjay K. Tyagi <sup>3</sup>**


Received: 26 November 2018; Accepted: 17 December 2018; Published: 20 December 2018

**Abstract:** With trending competition in decision-making process, linguistic decision-making is gaining attractive attention. Previous studies on linguistic decision-making have neglected the occurring probability (relative importance) of each linguistic term which causes unreasonable ranking of objects. Further, decision-makers' (DMs) often face difficulties in providing apt preference information for evaluation. Motivated by these challenges, in this paper, we set our proposal on probabilistic linguistic preference relation (PLPR)-based decision framework. The framework consists of two phases viz., (a) *missing value entry phase* and (b) *ranking phase*. In phase (a), the missing values of PLPR are filled using a newly proposed automatic procedure and consistency of PLPR is ensured using a consistency check and repair mechanism. Following this, in phase (b), objects are ranked using newly proposed analytic hierarchy process (AHP) method under PLPR context. The practicality of the proposal is validated by using two numerical examples viz., green supplier selection problem for healthcare and the automobile industry. Finally, the strength and weakness of the proposal are discussed by comparing with similar methods.

**Keywords:** analytic hierarchy process; consistency measure; group decision-making; probabilistic linguistic preference relation

#### **1. Introduction**

Decision-making is an inevitable aspect of human life which involves uncertainty and vagueness. The process of selecting a suitable object for the task brings cognitive thought processes into the picture, which is dynamic and competing in nature [1]. DMs often face difficulties in expressing their opinions in a sensible manner and to alleviate the issue to a certain extent; they adopt linguistic preference information [2]. Previous studies on linguistic decision process [2–4] have claimed that (a) linguistic preferences are simple and straightforward information which can be directly obtained from the DM, (b) also, these linguistic preferences mitigate the cost of inaccuracies to some extent. Motivated by these claims; scholars presented different decision-making framework under the linguistic context, of which, some are reviewed here. Zadeh [5] framed the genesis of a linguistic variable and applied the same for approximate reasoning. Later, Herrera et al. [2] fabricated the initial idea of the linguistic decision process and proposed a sequential decision framework in a linguistic environment. Following this, Herrera et al. [3,6] presented the consensus model for group decision-making under the linguistic context. Inspired by the power of linguistic theory, Xu [7] extended the geometric mean and ordered weighted geometric aggregation operator for a linguistic domain. Further, He et al. [8] put forward a new entropy measure for linguistic-based group decision-making.

Though linguistic decision-making is an attractive concept, DMs still face difficulties in rationally rating the objects. The main reason for this difficulty is the cognitive behavior of the human mind, which encourages pair-wise comparative analysis rather than a standalone rating [9]. Motivated by the power of pair-wise comparison and linguistic term set (LTS), Herrera et al. [10,11] proposed the linguistic preference relation (LPR) concept for group decision-making and investigated some choice functions for the same. Following this, Xu [12] put forward some deviation measures for decision-making process under LPR context. Recently, Molinera et al. [13] developed new fuzzy ontologies for linguistic preference information and applied the same for decision-making. Wang and Xu [14] put forward an interactive algorithm for filling the missing LPR values using consistency measures and repaired the consistency of the same using the repairing mechanism.

Inspired by the power of linguistic information and its substantial use in decision-making, Rodriguez [15] proposed the hesitant fuzzy linguistic term set (HFLTS) which is an extension to LTS under hesitant fuzzy environment [16]. The HFLTS allowed DMs to give different choices of preference for the same instance, which managed uncertainty to some extent. Motivated by the power of HFLTS, Zhu and Xu [17] put forward the hesitant fuzzy linguistic preference relation (HFLPR), which is an extension to preference relation under HFLTS context. They also investigated some consistency measures for the same. Following this, Wang and Xu [18] presented the concept of extended hesitant fuzzy preference relation and studied some consistency measures for the same. Wu [19] presented a consensus model based on possibility distribution for HFLPR and validated the applicability of same for decision-making process. Recently, Song and Hu [20] proposed a decision framework for handling incomplete HFLPR and applied the same for real-time group decision-making problem. Tuysuz and Simsek [21] extended the popular AHP method under HFLTS context and applied the same for assessing the performance of cargo factory.

Though the HFLPR is able to manage DMs' hesitation in preference information, the occurring probability (distribution assessment) of each linguistic term in the decision-making process is neglected. In many practical applications, all linguistic choices by the DM do not bear the same importance and hence, ignoring the occurring probability of each linguistic term is unreasonable and illogical. To circumvent this challenge, Zhang et al. [22] introduced the concept of linguistic distribution assessment (LDA) and associated symbolic proportion for each linguistic term. Later, Pang et al. [23] generalized the idea of LDA by allowing partial ignorance (∑*<sup>i</sup> pi* ≤ 1) in preference elicitation and termed it as probabilistic linguistic term set (PLTS) which is an extension to HFLTS with probability concept. Recently, Zhang et al. [24] put forward the concept of incomplete LDA which is similar to PLTS and used in for decision-making. Inspired by the superiority of PLTS in associating occurring probability to each linguistic term; Bai et al. [25] presented a new comparison method use area concept for PLTS. Later, Liao et al. [26] extended the programming model to PLTS for multi-attribute decision-making (MADM). Liu and Teng [27] extended the Muirhead mean aggregation operator to PLTS for group decision-making. Zhang et al. [28,29] put forward the probabilistic linguistic preference relation (PLPR) concept which is an extension to preference relation under PLTS context and some additive consistency and consensus reaching measures were also investigated. Recently, Xie et al. [30] proposed probabilistic uncertain preference relation and applied the same for virtual reality application. Recently, attracted by the power of PLPR, Wu and Liao [31] proposed gain-lost dominance score method under PLTS for consensus reaching. Xie et al. [32] extended AHP (analytic hierarchy process) method and applied the same for assessing the performance of a new area. Since the concept of PLPR just began, we gained motivation to throw some light towards this concept and set our research focus in this direction.

Based on the review conducted above, some genuine challenges/lacunas are identified which are presented in a nutshell below:

(1) Investigation of decision process using the preference relation proves to be effective than investigation using attribute driven methods [28]. The reason for this is evident from the ease of pair-wise comparison mechanism, which allows DMs to produce sensible preference information about each object with respect to a specific criterion. Also, the process of pair-wise comparison closely resembles with the practical decision process. Thus, motivated by the power of pair-wise comparison, we set our proposal in this context.


Motivated by these challenges and with the view of alleviating these challenges, in this paper, we propose a new scientific decision framework, which consists of two phases viz., (1) *missing value entry phase* and (2) *ranking phase*. Xu [33] clearly pointed out that, (i) DMs are often unwilling to reconstruct the evaluation matrix and (ii) also the chance for the manually reconstructed matrix to be consistent is very less. Thus, motivated by these claims,


The rest of the paper is constructed as Section 2 for preliminaries, Section 3 for calculation of missing values and ranking of objects. Section 4 presents a numerical example for demonstrating the practical use of the framework. Section 5 presents the comparative study and Section 6 gives the concluding remarks and future works.

#### **2. Preliminaries**

Let us review some basics of LTS and PLTS concepts.

**Definition 1** ([12])**.** *Consider a LTS S defined by* {*sα*|*α* ∈ [−*n*, *n*]} *with n being the limits of the term set and s*−*<sup>n</sup> and sn are the lower and upper bounds of the term set. The s<sup>α</sup> is a linguistic term set with the following characteristics:*

*(a) s<sup>α</sup> and s<sup>β</sup> are two linguistic term sets with s<sup>α</sup>* > *s<sup>β</sup> only if α* > *β*.

*(b) The negation of s<sup>α</sup> is denoted by neg*(*sα*) *and is given by neg*(*sα*) = *s*−*α. As a special case, neg*(*s*0) = *s*0.

**Definition 2** ([23])**.** *Consider a LTS S defined by* {*sα*|*α* ∈ [−*n*, *n*]}*, then the PLTS is defined by:*

$$L(p) = \left\{ L^t(p^t) \,|\, 1^t \in \mathcal{S}, 0 \le p^t \le 1, t = 1, 2, \dots, \#L(p), \sum\_{i=1}^{\#L(s)} p\_i^t \le 1 \right\} \tag{1}$$

*where L<sup>t</sup> is the tth linguistic term and p<sup>t</sup> is the associated occurring probability of the tth linguistic term.*

**Note 1:** The concept of PLTS [23] is a generalization to LDA [22] that allows partial ignorance (∑*<sup>i</sup> pi* ≤ 1) in preference elicitation and the concept of incomplete LDA [24] is similar to PLTS.

**Remark 1.** *For brevity of representation, we denote a probabilistic linguistic element (PLE) as rt* - *pt where r is the subscript of the linguistic term, p is the corresponding probability of the linguistic term and t is the number of instances*.

**Definition 3** ([29])**.** *The PLPR is a square matrix of the form R* = *Lt ij pt ij <sup>n</sup>*×*<sup>n</sup> with <sup>L</sup><sup>t</sup> ii* <sup>=</sup> {*s*0}*, <sup>p</sup><sup>t</sup> ji* = *<sup>p</sup><sup>t</sup> ij and L<sup>t</sup> ji* <sup>=</sup> *neg Lt ij* .

**Definition 4** ([23])**.** *Consider two PLEs, L*1(*p*) *and L*2(*p*) *as defined before. Then,*

$$L\_1(p) \oplus L\_2(p) = \left\{ r\_1^t + r\_2^t ; p\_1^t \times p\_2^t \right\} = \left\{ r\_3^t(p\_3^t) \right\} = L\_3^t(p) \tag{2}$$

$$
\lambda L\_1(p) = \left\{ r\_1^t \times \lambda; p\_1^t \right\} = L\_3^t(p) \tag{3}
$$

*where t* = 0, 1, . . . , #*L*(*p*).

**Remark 2.** *The operational laws defined in Definition 4 are valid only when the length of the PLEs is equal. If the length is unequal, we apply method from [23] to make the length of PLEs equal. Also from Equations (2) and (3), we observe that the linguistic term of PLE sometimes gets outside the boundary which can be transformed to PLTS within the boundary by using [28].*

#### **3. Proposed Decision Framework under Probabilistic Linguistic Preference Relation (PLPR) Context**

#### *3.1. Proposed Architecture of PLPR Based Decision Framework*

The architecture of the proposed scientific decision framework is presented in Figure 1 which is simple and straightforward to understand.

**Figure 1.** Architecture of proposed scientific decision framework.

#### *3.2. Proposed Automatic Procedure for Filling Missing Values and Consistency Check and Repair for PLPRs*

In this section, the procedure for finding the missing values of a PLPR is presented. Generally, DMs find pairwise comparison as an easier option for rating alternatives [28]. DMs rate the alternatives upon each criterion and sometimes they are unwilling or confused between alternatives' performance over a specific criterion and this forces them to ignore such rating. As a result, the decision matrix is now incomplete and further processing becomes difficult. To circumvent this issue, an automated procedure is proposed which automatically fits a value to the missing information. Zhang et al. [33] claimed that "(a) manual entry of missing values by some random information is unreasonable and causes potential loss of information and (b) returning of decision matrix to the DM for re-entry is also unreasonable and computationally ineffective". Motivated by such claims, in this paper, an automated procedure is presented under PLTS context for filling missing values.

The procedure for automated filling of missing value is given below:

**Step 1:** Consider a PLPR *R* = *Lt ij pt ij <sup>n</sup>*×*<sup>n</sup>* which has PLEs. Identify the instance which is missing. If *j* > *i* + 1, then the missing instance can be automatically estimated (follow steps below), else follow Equation (4).

$$R\_{ij} = \left\{ \frac{\sum\_{i=1}^{m} r\_{ij}}{m}, \frac{\sum\_{i=1}^{m} p\_{ij}}{m} \right\} \forall j \le i+1 \tag{4}$$

where *rij* is the subscript of the linguistic term, *pij* is the associated occurring probability of the linguistic term and *m* is the order of the matrix.

$$\begin{array}{l} \textbf{Step 2: When } j > i + 1, \text{ apply Equation (5) to automatically estimate the missing values.}\\ R\_{ij} = \min\left( \left( \oplus\_{k=1}^{j-i-1} \left\{ r\_{(i+k)(i+k+1)} \right\} \right), \left( \oplus\_{k=1}^{j-i-1} \left\{ r\_{(i+k)(i+k+1)} \right\} \right\} \oplus \boxplus\_{k=1}^{j-i-1} \left\{ \left( 1 - r\_{(i+k)(i+k+1)} \right) \right\} \right) \text{ and } \forall i \in \{1, 2, \dots, n-1\} \end{array}$$

$$\min \left( \left( \bigoplus\_{k=1}^{j-i-1} \left\{ p\_{(i+k)(i+k+1)} \right\} \right), \left( \bigoplus\_{k=1}^{j-i-1} \left\{ p\_{(i+k)(i+k+1)} \right\} \quad \bigoplus\_{k=1}^{j-i-1} \left\{ \left( 1 - p\_{(i+k)(i+k+1)} \right) \right\} \right) \right) \tag{5}$$

where ⊕ is an operator given in Definition 4.

**Note 2:** The result from Equation (5) is also a PLE and the values that go out of bounds when ⊕ operator is applied are transformed using Remark 2.

**Step 3:** Check the consistency of the matrix *R* = *L<sup>t</sup> ij pt ij <sup>n</sup>*×*<sup>n</sup>* by using Equations (6) and (7).

$$R\_{ij}^{\mathbb{Z}} = \left( \left\{ L\_{ij}^{\*t} \left( p\_{ij}^{\*t} \right) \right\} \right)\_{n \times n} \tag{6}$$

*Symmetry* **2019**, *11*, 2

where *L*∗*<sup>t</sup> ij* and *<sup>p</sup>*∗*<sup>t</sup> ij* can be calculated by using Equation (7).

$$L\_{ij}^{\*t} \begin{pmatrix} p\_{ij}^{\*t} \end{pmatrix} = \begin{cases} \frac{1}{m} \left( \bigoplus\_{\varepsilon=1}^{m} \left( L\_{i\varepsilon}(p) \oplus L\_{\varepsilon j}(p) \right) \right) \,\forall i \neq j\\ \{s\} \,\, otherwise \end{cases} \tag{7}$$

where ⊕ is an operator given in Definition 4.

Here linguistic terms are added as per Definition 4 and transformation procedure is applied to those terms that exceed the limits. However, the corresponding probability terms are calculated by using weighted geometry method to avoid unreasonable probability values. The personal opinion on each alternative is given by the DM with ∑*<sup>n</sup> <sup>i</sup>*=<sup>1</sup> *ω<sup>i</sup>* = 1.

**Step 4:** Calculate the distance between *Rij* and *R<sup>z</sup> ij* by using Equation (8) to determine the consistency index (*C I*).

$$CI(R) = d\left(R\_{ij}, R\_{ij}^z\right) = \sqrt{\frac{2}{m(m-1)} \sum\_{i=1}^{m} \sum\_{j=i+1}^{m} \left(\sum\_{t=1}^{\#L(p)} \left(p\_{ij}^t \times p\_{ij}^{t\*}\right) \left(\frac{r\_{ij}^t - r\_{ij}^{t\*}}{T}\right)\right)^2} \tag{8}$$

where *T* is the cardinality of the LTS, *r* is the subscript of the PLTS and *p* is the corresponding probability of the term set.

**Note 3:** The distance formula described in Equation (8) obeys the desirable distance properties viz., non-negative, non-degenerate, symmetric and transitive.

**Step 5:** The consistency values obtained from step 4 (*C I*(*R*)) are compared with the standard consistency value *C I* ,(*R*) (suggested as 0.05 by DMs). If *C I*(*R*) <sup>≤</sup> *C I* ,(*R*) then, *<sup>R</sup>* is acceptable; else *R* is unacceptable and automatic repairing must be done by following the steps below.

**Step 6:** Repair the inconsistent PLPR automatically by using Equation (9).

$$R\_{ij}^{z+1} = \frac{\left(L\_{ij}(p)\right)^{1-\tau\sigma} \oplus \left(L\_{ij}^\*(p)\right)^{\tau\sigma}}{\left\{\left(L\_{ij}(p)\right)^{1-\tau\sigma} \oplus \left(L\_{ij}^\*(p)\right)^{\tau\sigma}\right\} \oplus \left\{\left(1-L\_{ij}(p)\right)^{1-\tau\sigma} \oplus \left(1-L\_{ij}^\*(p)\right)^{\tau\sigma}\right\}}\tag{9}$$

where *Lij*(*p*) = *rij*- *pij*, *L*<sup>∗</sup> *ij*(*p*) = *r*∗ *ij p*∗ *ij*, *<sup>τ</sup>* and *<sup>σ</sup>* are parameters in the range [0,1].

Note that this repairing is an iterative process and until consistent matrix is obtained, we apply the procedure.

**Step 7:** Repeat the steps 5 and 6 iteratively till a PLPR of acceptable consistency is obtained.

#### *3.3. Proposed Analytic Hierarchy Process (AHP) Method under PLPR Context*

Analytic hierarchy process (AHP) is a classical ranking method that is based on the pairwise comparison concept [34]. This ranking method works with preference relations and weight of each alternative is determined. Based on the weight values, alternative are ranked and the suitable object is selected for the process. Recently, Emrouznejad and Marra [35] conducted a comprehensive review on AHP method and identified its diverse applicability in MCDM and the interesting variants of AHP. Clearly from the review, extension of AHP to PLTS context is a new idea for exploration and the work of Xie et al. [32] framed the genesis for the same. Some lacunas are discussed in Section 1 which motivates the proposed extension of AHP under PLPR context.

Now, we present the procedure for ranking objects using the proposed extension to AHP under PLPR context.

**Step 1:** Define the problem under multi-attributes decision-making context and determine the number of objects, attributes and DMs. Use PLEs as preference information.

**Step 2:** Suppose, *m* objects and *n* attributes are considered, *n* PLPRs of order (*m* × *m*) is formed. Following this, a PLPR of order (*n* × *n*) is formed for the attributes.

**Step 3:** Check the consistency of all PLPRs using the procedure presented in Section 3.2 and repair the inconsistent PLPR. Apply Equation (2) to the PLPR of order (*n* × *n*). This forms a weight vector for the attributes which is probabilistic linguistic in nature.

**Step 4:** Following step 3, we aggregate the PLEs from (*m* × *m*) matrices using Equation (2) to form a decision matrix with PLTS information of order (*m* × *n*) where *m* is the number of alternatives and *n* is the number of attributes.

**Step 5:** The attribute weights and decision matrix are taken from steps 3 and 4 respectively and Equation (2) is applied to obtain a vector of order (*m* × 1) for each of the *m* alternatives.

**Step 6:** The vector obtained from step 6 contains PLTS information which is used for the final ranking by applying Equation (10).

$$\varphi\_i = \sum\_{k=1}^{\#L(p)} \left( r\_i^k \times p\_i^k \right) \tag{10}$$

where *ri* is the subscript of the *i*th object and *pi* is the probability of the corresponding *i*th object.

Thus, the object which has large *ϕ<sup>i</sup>* value is ranked first and so on.

#### **4. Numerical Example**

#### *4.1. Green Supplier Selection for Healthcare Center*

Indian healthcare industries are gaining high interest in recent times because of its diverse spectrum of high-tech equipment, highly skilled professionals, eco-friendly infrastructure etc. On April 2015, IBEF (Indian brand equity foundation) conducted a survey and identified that Indian healthcare industries are a big asset for the nation with an outreach of USD 280 billion by 2020. The report also showed that India is ranked third in the global healthcare sector. With the motive of igniting the spirit, GoI (government of India) started many interesting and innovative initiatives (www.ibef.org) like "*signing of MoA (memorandum of agreement) with WHO (world health organization) for promoting public health in India, signing MoU (memorandum of understanding) with medical agencies of BRICS to facilitate healthy medical products*". A study by Healthcare Design magazine showed that "*each year, expenditure on energy usage by healthcare is USD 8 billion*" which drives them to place a concrete carbon footprint. To better reduce the CO2 emission and energy usage, healthcare must tune their thoughts towards green technologies and selection of equipment suppliers who follow green standards ISO 14000 and 14001 actively.

With this train of thought, we consider a healthcare center in Tirchy that wants to expand its service and hospitality for the betterment of the people in and around the region and also reduce its contribution in carbon footprint by adopting green technology. To do so, the management decides to renovate certain policies of the hospital which include proper and hygienic service to patients, proper and effective resource management, purchase of equipment from green suppliers and intense and sensible care at critical times. Surfing through the previous reports, the management finds an urgent need to make a reasonable decision with regards to the purchase of surgical equipment for the health center. An expert committee of three members viz., chief doctor (*E*1), senior stock manager (*E*2) and chief technical officer (*E*3) is formed and suitable supplier is chosen using a systematic scientific approach. Initially, seven green suppliers are chosen for the process and out of these seven suppliers four green suppliers who actively follow ISO 14000 and 14001 standards are selected based on the pre-screening test. Now, the committee decides four attributes for evaluation of four green suppliers. The committee plans to do the pairwise comparison and used PLTS information for rating. The details of these four attributes are given below:


Let us now consider the following procedure for evaluation:

**Step 1:** Construct four PLPRs of order (4 × 4) with PLTS information. Each criterion is taken and the DMs form pairwise comparison matrices with each supplier over a specific criterion.

The missing values in Table 1 are determined using Equation (5) and it is shown in Table 2. Clearly, the missing values which are calculated is also a PLE.


**Table 1.** Probabilistic Linguistic Preference Relation (PLPR) matrices for each criterion.


**Table 2.** PLPR matrices after finding the missing values.

**Step 2:** Construct one PLPR matrix of order (4 × 4) to determine the weights of the attributes. The Equation (2) is used to determine the weight of each criterion. The weight values are probabilistic linguistic in nature.

From Table 3 we obtain the weight value (relative importance) for each criterion. By applying Equation (2) we get the weight values as PLEs and it is given by *C*<sup>1</sup> = {2,(0.32), 1,(0.37), 2,(0.41)}, *C*<sup>2</sup> = {1,(0.49), 0,(0.37), 2,(0.38)}, *C*<sup>3</sup> = {1,(0.30), 1,(0.32), 2,(0.38)} and *C*<sup>4</sup> = {1,(0.35), 2,(0.45), 0,(0.44)}.

**Step 3:** Check the consistency of all PLPRs and repair those PLPRs that are inconsistent in nature. Follow the procedure from Section 3.2 for automatic repairing of inconsistent PLPR.

All the above four PLPR matrices are checked for consistency by using the Equations (6) and (7). The child PLPR matrices - *Rz i* are initially formed from all four parent PLPR matrices (*Ri*) and the distance between each of these PLPRs is calculated. These distance values are shown in Table 4. Just for

an example, let us consider the child matrix corresponding to *C*<sup>1</sup> and the distance between the parent and child matrices are calculated and it is given in Table 4.

*Rz* <sup>1</sup> = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.15), 0,(0.14), −2,(0.2) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.17), 2,(0.12), −2,(0.12) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.17), −1,(0.15), −2,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.15), 0,(0.14), 2,(0.2) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.16), 2,(0.15), −2,(0.16) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.16), −1,(0.18), −2,(0.2) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.17), −2,(0.12), 2,(0.12) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.16), −2,(0.15), 2,(0.16) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.18), −2,(0.16), −2,(0.12) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.17), 1,(0.15), 2,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.16), 1,(0.18), 2,(0.2) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.18), 2,(0.16), 2,(0.12) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

**Table 3.** Attributes weight estimation matrix.


**Table 4.** Calculation of distance values.


These values are compared against the standard value *C I* ,(*R*) (0.05). Since the distance values of the first three PLPRs are greater than 0.05, it is inconsistent and so we apply Equation (9) to repair these matrices. Just for an example, consider the child matrix of *C*<sup>1</sup> which becomes consistent in the second iteration with the distance value of 0.0136 which is less than the threshold 0.05. Further, child matrix of *C*<sup>2</sup> and *C*<sup>3</sup> are also inconsistent which are repaired using Equation (9) and the distance value is given by 0.018 (second iteration) and 0.019 (first iteration) which is less than the threshold value 0.05.


**Step 4:** Apply the proposed ranking method from Section 3.3 over the consistent PLPRs and obtain a suitable supplier for the process. The four PLPR matrices of order (*m* × *m*) and attributes weight matrix of order (*n* × 1) are aggregated using the ⊕ operator defined in Definition 4. The resultant matrix is given in Table 5.


**Table 5.** Decision matrix with probabilistic linguistic term set (PLTS) information.

Table 5 is formed after applying Equation (2) over the attributes-alternative pair. When Equation (2) is applied, a vector of order (1 × *n*) is obtained for all *m* suppliers and finally, a decision matrix of order (*m* × *n*) with PLTS information is shown in Table 5. By using the procedure given in Section 3.3 on Table 5, we obtain final rank values as shown in Table 6.

From Table 6, we observe that the ranking order is given by *S*<sup>1</sup> > *S*<sup>4</sup> > *S*<sup>2</sup> > *S*<sup>3</sup> and *S*<sup>1</sup> is chosen as a suitable supplier for the healthcare. Further, suppliers *S*4, *S*<sup>2</sup> and *S*<sup>3</sup> are for backup plans. When method from [21] is applied, the ranking order becomes *S*<sup>4</sup> > *S*<sup>1</sup> > *S*<sup>2</sup> > *S*<sup>3</sup> which is different from the ranking order obtained by the proposed framework. This is evident from the fact that the method discussed in [21] does not contain occurring probability values.

**Step 5:** Compare the strength and weakness of the proposal with state of the art methods. Readers are encouraged to refer Section 5 for the same.


**Table 6.** Final rank values.

#### *4.2. Green Supplier Selection for Automobile Industry in India*

Automobile industries in India are booming at a faster pace providing economic growth and global market improvement. These industries drive avenues of employment to approximately 13 million people in India. As per the 2013–14 annual report on automobiles, a grand total of 21,500,165 vehicles were produced which eventually boomed the revenue for India. Despite the attractive advantages, the pollution caused by these industries is huge which affect the living beings and the environment as a whole. A study found that by 2020, almost half of the cars in India will use diesel and roughly 620,000 people will die due to respiratory issues (https://community.data.gov.in/automobiles-andpollution-in-india/). This alarming analysis motivates automobile industries to choose green suppliers for purchasing their raw materials. Green suppliers actively monitor their system and practices to ensure limited emission of environmental pollutants. These suppliers strongly follow the ISO 14000 and 14001 standards pertaining to the adoption of green practices and technologies.

Motivated by this background, in this paper, we plan to provide a systematic framework for suitable selection of green supplier from the set of suppliers for leading automobile industry in India (name anonymous). Let *E* = (*e*1,*e*2,*e*3) be a set of three DMs who constitute the expert committee. *G* = (*g*1, *g*2, *g*3, *g*4) and *C* = (*c*1, *c*2, *c*3, *c*4) be the set of green suppliers and the corresponding evaluation attributes respectively. Since the attributes used in Section 4.1 adhere to the green standards, we adopt the same in this example also. Initally, eight green suppliers were chosen for the process and based on pre-screening and Delphi method, four green suppliers are finalized for evaluation. These suppliers actively obey ISO 14,000 and 14,001 standards and they are evaluated under four attributes adapted from Section 4.1. Following steps are presented for the systematic selection of green supplier:

**Step 1.** Form the PLPRs supplier wise for each criterion. This produces four matrices of order 4 × 4 that correspond to one preference relation for each criterion.

**Step 2:** Fill the missing values by using the proposed procedure given in Section 3.2. The missing values are represented by "X" in Table 7 and these values are filled systematically using procedure proposed in Section 3.2 and the values are PLEs (refer Table 8).


**Table 7.** PLPR information supplier to supplier for each attribute.


**Table 8.** Filling of missing values in PLPRs.

**Step 3:** Determine the consistency of each PLPR and repair the inconsistent PLPR iteratively using the proposed procedure given in Section 3.2. The *d* - *R*1, *R<sup>z</sup>* 1 is 0.13 which is inconsistent and it is made consistent in two iterations with *d* - *R*1, *R<sup>z</sup>* 1 as 0.013. Further, *d*(*R*2, *R<sup>z</sup>* <sup>2</sup>) is 0.091 which is inconsistent and it is made consistent in two iterations with *d*(*R*2, *R<sup>z</sup>* <sup>2</sup>) as 0.03. The *d* - *R*3, *R<sup>z</sup>* 3 and *d* - *R*4, *R<sup>z</sup>* 4 are 0.14 and 0.11 respectively which is inconsistent and it is made consistent with in a single iteration with *d* - *R*3, *R<sup>z</sup>* 3 as 0.021 and 0.025 respectively.

*<sup>R</sup>z*+<sup>1</sup> <sup>1</sup> <sup>=</sup> ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.15), 2,(0.12), −2,(0.17) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.16), 2,(0.13), −2,(0.13) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.15), −1,(0.13), 0,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.15), −2,(0.12), 2,(0.17) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.16), 0,(0.14), 0,(0.16) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.16), −2,(0.14), 2,(0.19) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.16), −2,(0.13), 2,(0.13) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.16), 0,(0.14), 0,(0.16) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 2,(0.17), 0,(0.15), 1,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.15), 1,(0.13), 0,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.16), 2,(0.14), −2,(0.19) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ −2,(0.17), 0,(0.15), −1,(0.15) ⎫ ⎪⎬ ⎪⎭ ⎧ ⎪⎨ ⎪⎩ 0,(1), 0,(1), 0,(1) ⎫ ⎪⎬ ⎪⎭ ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

Just as an example, the consistent PLPR *R*(2) <sup>1</sup> after second iteration is shown above.

**Step 4:** From step 3, we obtain consistent PLPRs which are used for prioritizing green suppliers and selection of a suitable green supplier for the automobile industry. The extended AHP under PLPR context (from Section 3.3) is used for prioritization of green suppliers.

Table 9 shows the decision matrix which is formed by applying Equation (2) over Table 8. The elements of Table 9 are PLEs and the order of the matrix is 4 × 4.


**Table 9.** Decision matrix with PLTS information.

The green suppliers are prioritized by applying Equation (2) on Table 9. Table 9 depicts the PLPR values after step 5 of Section 3.3. We again apply Equation (2) on Table 9 to obtain a vector (order 4 × 1) of PLEs corresponding to each green supplier. The green suppliers are prioritized using the vector and it is given by: *S*<sup>1</sup> = {2,(0.57), 2,(0.58), 2,(0.58)}; *S*<sup>2</sup> = {2,(0.53), 2,(0.54), 2,(0.55)}; *S*<sup>3</sup> = {1,(0.59), 1,(0.59), 0,(0.60)} and *S*<sup>1</sup> = {1,(0.52), 0,(0.53), 0,(0.54)}.

By applying Equation (10) on this vector, we obtain the ranking order as *S*<sup>1</sup> *S*<sup>2</sup> *S*<sup>4</sup> *S*3.

**Step 5:** Compare the superiority and weakness of the proposed framework with other methods (refer Section 5 for details).

#### **5. Comparative Analysis: PLPR Based Decision Framework vs. Others**

In this section, we make a comparative analysis of the proposed decision framework with [32] and [21]. The method [32] presents an extension to AHP method under PLTS context and method [21] extends AHP method to HFLTS context. In order to maintain homogeneity in the process of comparison, the proposed decision framework is compared with [32] and [21]. Table 10 shows the analysis of these methods under the theoretic and numeric perspectives. The theoretic factors are chosen based on intuition and the numeric factors are chosen from [36].

The strengths of the proposed decision framework are:


(5) Also, from the time complexity analysis, we can observe that proposed decision framework and method [27] has three crucial operations viz., (a) filling missing values, (b) check & repair of inconsistent and (c) ranking of objects with *m* objects and *n* attributes. Operation (a) takes *O* - *m*2 time complexity, operation (b) takes *O* - *m*2 time complexity and operation (c) takes *O* - *m*2(*n* + 1) . So, the complexity of the proposed decision framework is *O* - 3*m*<sup>2</sup> + *nm*<sup>2</sup> ≈ *O* - *m*2 . In contrary, the complexity of [27] (by similar analysis) is *<sup>O</sup>*(*m*<sup>3</sup> + *<sup>m</sup>*<sup>2</sup> + *nm*2) ≈ *<sup>O</sup>* - *m*3 which is evidently complex than the proposed decision framework.

Some weaknesses of the proposed framework are:



**Table 10.** Investigation of features: Proposed vs. Others.

#### **6. Conclusions**

This paper presents a new scientific decision framework under the PLPR context for rational decision-making under critical situations. The missing values are sensibly filled by using a systematic approach. Also, the consistency of the PLPR is determined and inconsistent PLPRs are repaired using the proposed method. Finally, the AHP method is extended to PLPR for selecting a suitable object from the set of objects. The practicality of the proposed decision framework is demonstrated by solving equipment supplier selection problem for a healthcare center. Also, the strengths and weaknesses of the proposal are realized by comparison with other methods under both theoretic and numeric perspectives.

Some managerial implications are presented in a nutshell below:


As a part of the future scope, we plan the following research directions: (i) to present new methods for ranking under pair-wise comparison ideas; (ii) to enhance the consistency of the PLPRs under both additive and multiplicative context; (iii) to develop methods for consensus reaching by gaining motivation from [38,39] and strategic weight calculation inspired by [40,41].

**Author Contributions:** The individual contribution and responsibilities of the authors were as follows: R.K., K.S.R. and M.I.A. designed the research model, collected, pre-processed, and analyzed the data and the obtained results, and worked on the development of the paper. S.K. and S.K.T. provided good advice throughout the research by giving suggestions on model design, methodology, and inferences, and refined the manuscript. All the authors have read and approved the final manuscript.

**Funding:** This research was funded by University Grants Commission (UGC), India and Department of Science & Technology (DST), India under grant number F./2015-17/RGNF-2015-17-TAM-83 and SR/FST/ETI-349/2013.

**Acknowledgments:** Authors thank the editors and the anonymous reviewers for their insightful comments which improved the quality of the paper.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **A New Methodology for Improving Service Quality Measurement: Delphi-FUCOM-SERVQUAL Model**

#### **Olegas Prentkovskis 1,\*, Živko Erceg 2, Željko Stevi´c 2, Ilija Tanackov 3, Marko Vasiljevi´c <sup>2</sup> and Mladen Gavranovi´c <sup>2</sup>**


Received: 25 November 2018; Accepted: 12 December 2018; Published: 16 December 2018

**Abstract:** The daily requirements and needs imposed on the executors of logistics services imply the need for a higher level of quality. In this, the proper execution of all sustainability processes and activities plays an important role. In this paper, a new methodology for improving the measurement of the quality of the service consisting of three phases has been developed. The first phase is the application of the Delphi method to determine the quality dimension ranking. After that, in the second phase, using the FUCOM (full consistency method), we determined the weight coefficients of the quality dimensions. The third phase represents determining the level of quality using the SERVQUAL (service quality) model, or the difference between the established gaps. The new methodology considers the assessment of the quality dimensions of a large number of participants (customers), on the one hand, and experts' assessments on the other hand. The methodology was verified through the research carried out in an express post company. After processing and analyzing the collected data, the Cronbach alpha coefficient for each dimension of the SERVQUAL model for determining the reliability of the response was calculated. To determine the validity of the results and the developed methodology, an extensive statistical analysis (ANOVA, Duncan, Signum, and chi square tests) was carried out. The integration of certain methods and models into the new methodology has demonstrated greater objectivity and more precise results in determining the level of quality of sustainability processes and activities.

**Keywords:** quality; sustainability processes; Delphi; FUCOM (full consistency method); SERVQUAL (service quality); new methodology

#### **1. Introduction**

According to Nowotarski [1], it can be said that quality is directly connected with meeting requirements, expectations, and needs of customers. By applying different tools and techniques, it is possible to manage a quality level in one way. Measuring the quality of all processes that make a coherent whole can greatly affect the full quality of service in all areas. Whether a service will be reused also depends on adequate quality, especially nowadays, when production processes are of approximate and high quality. In such conditions, proper and perfect execution of logistics services can have a crucial impact on its reuse. It is important to strive constantly for higher goals and their achievements. It requires also an adequate methodology that can help improve the quality measurement of logistics services.

The research domain is the logistics of express post, including all the activities and processes carried out within it, from the aspect of logistics service quality. The activities included into the domain of research are the activities of informing customers of express post services until the end that implies a logistics service provided. The survey was conducted on a sample of 70 respondents, permanent customers of services of the express post company, as well as customers who used services on a one-time basis. Introducing the types of express post services to customers leads to the creation of a certain degree of expectation, which may differ from the perception of the service provided. The subject of the research is to determine the quality of the logistics service of express post based on a new developed methodology. The motivation for execution of this research can be explained through two main reasons. The first reason represents a lack of universal methodology for service quality assessment that considers the nature of input parameters, needs, and requests of customers' ability of companies and other uncertainties. The second reason is the possibility for improving the efficiency of a company that is the object of research by developing a new methodology, which can be useful for strategic management and planning. This paper has several goals. The first one relates to the development of a new methodology that treats input and output parameters with precision and provides results that are more objective. The first goal is achieved throughout three different phases, which, when integrated, create the developed model. The advantages of the Delphi method are used first, whereby a total of 70 customers provide weighted dimension values, based on which, a ranking is made. Thereafter, the FUCOM (full consistency method) for determining the weight dimension values is applied, allowing consistent evaluations by the experts involved in this process. The second goal is to enrich the methodology for improving service quality measurement by applying the new developed model. This provides an adequate methodology for future research in this area. In addition, the third goal of the paper is to determine the difference between expectations and perceptions of the formed dimensions of the modified SERVQUAL (service quality) model and the possibility of identifying and improving critical factors of the logistics service in an express post company, which is the object of research.

After introductory considerations where the significance of research and goals are presented, the paper is structured throughout six more sections. Section 2 provides a review of the application of the SERVQUAL model in various areas for measuring the quality of different processes. Section 3 presents the new developed methodology that implies the integration of three different methods to provide the most accurate outputs. There is a flow chart of the study with an explanation of all phases and steps. Section 4 is a case study where the input parameters are defined, quality measurements are presented, the initial dimension ranking is provided, and the weighted values of all five dimensions are calculated. Section 5 presents the results of the research using the developed methodology, while Section 5.3 provides a comprehensive statistical analysis that establishes the regularities and conditions of expectation and perception processes. Section 6 is a conclusion, with an emphasis on the scientific contribution of this research and guidelines for future research.

#### **2. Literature Review**

A model that is often used to measure the quality of service is the SERVQUAL model. Motivated by the need to measure the contribution of the SERVQUAL model, Wang et al. [2] conducted a study, which proved that the SERVQUAL model was one of the major research topics for academic researchers in the period from 1998 to 2013 and that the model contributed significantly to the research on service quality.

#### *2.1. Quality Measurement in Logistics and Transport*

According to Kersten and Koch [3], in the past decades, the scope of logistics services has broadened from the provision of isolated services, such as transport and warehousing, to the management and handling of the flow of goods for entire companies. In such conditions of the market, service quality has a large influence on company efficiency. One of the most applied models for service quality is the SERVQUAL model. This model was applied in the field of passenger traffic [4], where the stated hypotheses were disproven because of a negative gap, and the SERVQUAL model pointed to critical business functions and the possibility of their improvement. In [5], the SERVQUAL model was based on 10 logistics service attributes for estimating performances in the field of refrigerated transport. The proof of how much the SERVQUAL model is used in all areas was shown by Roslan et al. [6], where the model measured the quality of service of logistics centers in Iskandar, Malaysia. For the same purpose, in research [7], authors developed a new hybrid MCDM (multi-criteria decision-making) model, consisting of an analytic hierarchy process (AHP), decision-making trial, and evaluation laboratory (DEMATEL), and analytic network process (ANP) methods. A combined approach integrating gap analysis, quality function deployment (QFD), and AHP for improving logistics service quality was applied in [8]. In research [9], an extension of the three-column format SERVQUAL instrument was extended for evaluation of passenger rail service quality. Three new transport dimensions (comfort, connection, and convenience) were added to the original five SERVQUAL dimensions.

For the evaluation of service quality in logistics and other fields, the Kano model [10–12], QFD method [13,14], six sigma [15,16] etc. can be applied, or, for example, a new developed Agro-Logistic Analysis and Design Instrument (ALADIN) model, which involves logistics, sustainability, and food quality analysis [17].

#### *2.2. Quality Measurement in Other Fields*

In their paper, Cho et al. [18] explored ways to improve services in service centers of electronics companies. They introduced and modified the SERVQUAL model to understand customers' demands for all service centers. According to Paryani et al. [19], the SERVQUAL model is also a very useful tool for identifying customers' demands. The evidence of how much the SERVQUAL model is present in studies is also shown in [20], where the authors used the model to assess patients' satisfaction by providing services at Sunyani Regional Hospital in Ghana; Behdio ˘glu et al. [21], who evaluated the quality of services at Yoncalı Physiotherapy and Rehabilitation Hospital in Kutahya, Turkey; Singh and Prasher [22], who measured the quality of services in hospitals from the Punjab state of India; as well as Khan et al. [23], who also measured the quality of services in hospitals. Using the SERVQUAL model, Chou et al. [24] have proved that the quality of service largely depends on the subjective assessment of service customers. To rank life insurance companies and assess the quality of services provided, Saeedpoor et al. [25] also used the SERVQUAL model. Additionally, the SERVQUAL model was used to measure the impact of technology on the quality of banking services and to measure the level of customer satisfaction [26]. Using the SERVQUAL model, Long [27] and Apornak [28] have shown that there is a significant link between technology used in providing services and the quality of services. The SERVQUAL model has also been used in a number of studies to rate the quality of banking services provided [29–33]. Wang et al. [34] also used five dimensions of the SERVQUAL model (tangibles, reliability, responsiveness, assurance, and empathy) to measure the service quality of an e-learning system. Moreover, those five dimensions were used by Yang and Zhu [35] to highlight the quality of community-based service provided by university-affiliated stadiums, as well as Luo et al. [36] while measuring satisfaction of outward-bound tourists.

#### *2.3. Integrated MCDM-SERVQUAL Model for Quality Measurement*

To measure the perception of service quality, Altuntas et al. [37] used the SERVQUAL model and two of the most known methods of MCDM method-based scales. By applying MCDM methods, it is possible to choose appropriate strategies, rationalize certain logistics and other processes, and make appropriate decisions that affect the operations of companies or their subsystems, as proved by the following research [38–51]. These methods can be easily integrated into other approaches, such as integration with SWOT (strengths, weaknesses, opportunities, and threats) analysis [42] or with the SERVQUAL model, as is the case in this paper. Rezaei et al. [52] integrated the SERVQUAL model with the best worst method, while Xuehua [53] applied a combined fuzzy AHP-SERVQUAL (analytic hierarchy process for service quality) model for evaluation of express service quality. The model was based on 14 indicators divided into five standard dimensions.

#### **3. New Methodology: DELPHI-FUCOM-SERVQUAL Model**

#### *3.1. The Proposed Methodology*

The developed methodology (Figure 1) for improving service quality measurement consists of four phases, with 18 steps in total. The first phase refers to the collection and preparation of data, which consists of six steps. First, it is necessary to form a SERVQUAL questionnaire on which the results of the research depend to a significant extent. It is necessary to consider the interdependence of certain elements of the questionnaire, which may influence the reliability of subsequent results. In this research, two important elements are taken into consideration when forming the questionnaire, the satisfaction of both the scientific and professional aspects.

Accordingly, scientists were consulted and the opinions of the management of the express post company were taken. A classic SQ (SERVQUAL) questionnaire consisting of 22 expectation questions and the same number of questions for perceptions was devised. The first contribution of this methodology is the modification of the SQ questionnaire for a specific case and the formation of a total of 25 elements for expectations and perceptions. It is recommended that this number is 20–30, depending on a specific situation. Subsequently, in the second step, the questionnaire was sent to customers to carry out their assessment in the fourth step, while the team of experts for evaluating the main dimensions of the SQ questionnaire was formed in the third step. Then, in the fifth step, hypotheses were defined, the number of which may vary depending on the area of application and a specific problem. It is possible to form hypotheses for each SQ dimension or for the overall SQ gap. In the last sixth step of the first phase, the data were processed and prepared for the next phase. The second phase implies the integration of different approaches into a new methodology consisting of nine steps. It is necessary to apply the Delphi method in the first step to allow customers to express their preferences regarding the main dimensions, i.e., their significance. After the results were obtained using the Delphi method, a ranking of all five dimensions was performed, so that a team of experts could determine their preferences. In the second step, the FUCOM for obtaining the weight values of SQ dimensions was applied. As it is group decision-making, all steps of this method should be implemented in the third step for each expert individually. In the fifth step, the averaging of the values obtained in the previous step to gain the final weight values of dimensions was performed. The sixth step determines the mean value of customers' responses for all dimensions regarding expectations, while, in the seventh step, the same was performed for perceptions. In the eighth step, the mean values obtained in the previous two steps were multiplied by the weight values obtained by the FUCOM. In the final step of this phase, the difference between perceptions and expectations was determined by taking into consideration the previously obtained values. The third phase implies the determination of the model reliability, which is defined by two steps: The calculation of the Cronbach alpha coefficient for all SQ dimensions and the performance of statistical analysis. The choice of adequate statistical tests is conditioned by the allocations of customers' responses, so it is impossible to define a universal one for application in this phase. Finally, the application of an adequate statistical test, and confirmation or rejection of previously set hypotheses was performed.

**Figure 1.** New methodology for improving service quality measurement.

In this paper, a new Delphi-FUCOM-SERVQUAL methodology has been developed to improve the process of service quality measurement. The advantages of the new methodology developed are reflected in that it provides precise treatment of input and output parameters, obtaining results that are more objective. Firstly, the advantages of Delphi method were used, whereby a total of 70 customers provided weight values of dimensions and based on which their ranking was made. Thereafter, the FUCOM for determining the weight values of dimensions was applied, allowing consistent evaluations of the experts involved in this process to determine finally the difference between perceptions and expectations of the modified SERVQUAL model. Mentioned advantages make this method better than other similar approaches because of the way data is handled. The developed methodology can be applied without any restrictions in various research fields. In addition, it is possible to determine the quality and efficiency of the companies which are the objects of research based on the satisfaction of its customers, but it also enables further application and re-application of this methodology. This methodology can be very helpful for strategic management of the company to improve their efficiency. This methodology enusres more precise treatment of input parameters and achieves better results than traditional quality measurement methods.

#### *3.2. Delphi Method*

The Delphi method does the study of and gives projections of uncertain or possible future situations for which we are unable to perform objective statistical legalities, to form a model, or apply a formal method. These phenomena are very difficult to quantify because they are mainly qualitative in their nature, i.e., there are not enough statistical data regarding them that could be used as the basis for our studies. The Delphi method is one of the basic forecasting methods, the most famous and most widely used expert judgment method. Methods of experts' assessments represent a significant improvement of the classical ways of obtaining the forecast by joint consultation of an expert group for a certain studied phenomenon. In other words, this is a methodologically organized use of experts' knowledge to predict future states and phenomena. A typical group in one Delphi session ranges from a few to 30 experts. Each interviewed expert, a participant in the method, relies on knowledge, experience, and his/her own opinion. The goal of the Delphi method is to exploit the collective, group thinking of experts about a certain field. The goal is to reach a consensus on an event by group thinking. This is a method of indirect collective testing, but with a return link. It consists of eight steps:


#### *3.3. Full Consistency Method (FUCOM)*

The FUCOM was developed by Pamuˇcar, Stevi´c, and Sremac, [54] for the determination of weights of criteria. It represents a new method that, according to the authors, represents a better method than AHP (analytical hierarchy process) and BWM (best worst method). For now, it has been applied in research by Nuni´c (2018). It consists of the three following steps.

*Step 1*. In the first step, the criteria from the predefined set of the evaluation criteria, *C* = {*C*1, *C*2,..., *Cn*}, are ranked. The ranking is performed according to the significance of the criteria, i.e., starting from the criterion that is expected to have the highest weight coefficient to the criterion of the least significance. Thus, the criteria ranked according to the expected values of the weight coefficients are obtained:

$$\mathcal{C}\_{j(1)} > \mathcal{C}\_{j(2)} > \dots > \mathcal{C}\_{j(k)} \tag{1}$$

where *k* represents the rank of the observed criterion. If there is a judgment of the existence of two or more criteria with the same significance, the sign of equality is placed instead of ">" between these criteria in expression (1).

*Step 2*. In the second step, a comparison of the ranked criteria is carried out and the comparative priority (*ϕk*/(*k*+1), *k* = 1, 2, ... , *n*, where *k* represents the rank of the criteria) of the evaluation criteria is determined. The comparative priority of the evaluation criteria (*ϕk*/(*k*+1)) is an advantage of the criterion of *Cj*(*k*) rank compared to the criterion of *Cj*(*k*+1) rank. Thus, the vectors of the comparative priorities of the evaluation criteria are obtained, as in expression (2):

$$\Phi = \left( \varphi\_{1/2}, \varphi\_{2/3^\prime}, \dots, \varphi\_{k/(k+1)} \right) \tag{2}$$

where *ϕk*/(*k*+1) represents the significance (priority) of the criterion of *Cj*(*k*) rank compared to the criterion of *Cj*(*k*) rank.

*Step 3.* In the third step, the final values of the weight coefficients of the evaluation criteria (*w*1, *w*2,..., *wn*) *<sup>T</sup>* are calculated. The final values of the weight coefficients should satisfy the two conditions:

(1) That the ratio of the weight coefficients is equal to the comparative priority among the observed criteria (*ϕk*/(*k*+1)) defined in *Step 2*, i.e., that the following condition is met:

$$\frac{w\_k}{w\_{k+1}} = \varphi\_{k/(k+1)}\tag{3}$$

(2) In addition to condition (3), the final values of the weight coefficients should satisfy the condition of mathematical transitivity:

$$\frac{w\_k}{w\_{k+2}} = \varrho\_{k/(k+1)} \odot \varrho\_{(k+1)/(k+2)}\tag{4}$$

Full consistency, i.e., minimum DFC (deviation from full consistency) (*χ*) is satisfied only if transitivity is fully respected. Based on the defined settings, the final model for determining the final values of the weight coefficients of the evaluation criteria can be defined:

$$\begin{array}{l} \min \chi\\ \text{s.t.} \\ \left| \frac{w\_{j(k)}}{w\_{j(k+1)}} - \boldsymbol{\varrho}\_{k/(k+1)} \right| \leq \chi\_{\prime} \; \forall j\\ \left| \frac{w\_{j(k)}}{w\_{j(k+2)}} - \boldsymbol{\varrho}\_{k/(k+1)} \right| \; \: \left| \boldsymbol{\varrho}\_{(k+1)/(k+2)} \right| \leq \chi\_{\prime} \; \forall j\\ \sum\_{j=1}^{N} w\_{j} = 1, \; \forall j\\ w\_{j} \geq 0, \; \forall j \end{array} \tag{5}$$

By solving the model (5), the final values of the evaluation criteria (*w*1, *w*2,..., *wn*) *<sup>T</sup>* and the degree of DFC (*χ*) are generated.

#### *3.4. SERVQUAL Model*

The model was developed in 1985 [55] and purified and improved in 1988 [56] and 1994 [57]. In current practice, it has become one of the most distinguished models in the area of service quality. It is expressed by the "perception minus expectation" algorithm.

The SERVQUAL model includes five basic quality dimensions: Tangibles, reliability, responsiveness, assurance, and empathy. Each of these dimensions is described by its attributes. The SERVQUAL model quality function is expressed by Equation (6):

$$SQ\_i \ = \Sigma \mathcal{W}\_{\dot{\!\!\!\/]}} \left( P\_{\dot{\!\!\/]}} - E\_{\dot{\!\!\/]} \right) \tag{6}$$

where: *SQi*—perceived dimension quality; *Wj—*attribute importance factor; *Pij—*perception of dimension *i* in relation to attribute *j*; *Eij—*expected level of attribute; and *j*, which is a normative of dimension *i*.

Five SERVQUAL dimensions (reliability, responsiveness, assurance, empathy, and tangibles) concisely represent an essential criterion used by customers when assessing the quality of services. The value of the dimensions in a classic SERVQUAL model is determined based on a questionnaire that contains 44 quality characteristics, 22 of which refer to expectations (*E*) and 22 to perceptions (*P*).

In this paper, as already mentioned, a modification of the SERVQUAL model has been carried out, which contains a total of 50 quality characteristics arranged equally for expectations and perceptions.

#### **4. Case Study: Measuring the Quality of Logistics Service in a Company of Express Post**

In this paper, the quality of logistics service was determined by applying the developed Delphi-FUCOM-SERVQUAL model. The aim of the research from the aspect of the company for which it was conducted was to determine the current level of logistics service quality and to improve it. For the survey of customers, a "Google forms" online application was used. The questionnaire consisted of 25 questions, including five dimensions: Reliability, assurance, tangibles, empathy, and responsiveness. Prior to filling in the questionnaire, respondents provided information, such as: Customer's status, gender, age, and employment status. The survey was conducted using the questionnaire in which a Likert scale was applied, including points from one to five. At the end of the questionnaire, the customer determined the values of weight coefficients depending on which dimension was most important to them. For every individual, each dimension that determines the quality level of service was different importance.

Regarding the status of respondents, 42 out of 70 customers were natural persons, while the remaining 28 were legal entities, i.e., 60% of respondents were natural persons, and 40% of respondents were legal entities. Division by gender shows that customers of both genders were approximately of the same percentage. Then, the highest number of respondents were aged 35–50, i.e., 25 respondents of 70. The percentage of 30% was taken by those aged 24–35, namely 21 respondents. Out of 70 respondents, the highest percentage of 68% belongs to the employed customers of the company's express post services. The target group are young entrepreneurial people with a frequent need for express post services. Table 1 shows the questions included into the questionnaire from the aspect of customer expectations.


**Table 1.** A questionnaire form from the aspect of customer expectations.

Table 1 presents all the questions that were used to test the degree of customer satisfaction. The questions are related to customer expectations about the services provided by the express post company. The questions are divided into five basic dimensions, i.e., the questions from one to six relate to the dimension of reliability, from seven to 10 to the dimension of assurance. The questions from 11

to 16 relate to the dimension of tangibles, from 17 to 21, to the dimension of empathy, and from 22 to 25 to the dimension of responsiveness. In this part of the questionnaire, questions are written in the future tense as they relate to customer expectations for the quality of logistics service. The form of questions for both aspects, expectations and perceptions, is the same, but questions in terms of perceptions are set in the past tense. Perception questions define the real customer perception of the quality of the service provided.

Based on all the above, a hypothesis of the research was set: *There is no significant difference between expectations and perceptions of the SERVQUAL model in providing logistics services*. In addition to the main hypothesis in the paper, some regularity of certain questions and attitudes of the customers has been established.

The dimension of reliability is mainly related to the timely delivery of a service that directly affects the quality of express post-delivery. The questions from the order number seven to 10 relate to the dimension of assurance. Within this dimension, it can be seen the degree of quality that refers to the trust and confidence of customers regarding the services of the express post company. The dimension of tangibles includes the questions that relate exclusively to couriers, delivery vehicles of the company, and the cost of the service. The results of the tangible dimension significantly provide information about the company where the research was conducted. This dimension also carries useful information on the real degree of quality of logistics service. Particular attention should be paid to each customer. Throughout the dimension of empathy, we can see how much the company really focuses on customers, their needs, and their problems. By understanding customers and anticipating their needs, the company can strive for an extremely high quality of service. Within the dimension of responsiveness, there are questions solely related to both daily and extraordinary situations. These are questions related to all necessary information and customers' requests, which can be obtained by employees in the company.

#### *4.1. Determining Dimension Ranks by Supplying the Delphi Method*

At the end of the questionnaire, the percentage of the dimensions most important for each customer were determined. The total sum of the assessed dimensions should be 100%. While assessing, customers considered which dimension was personally the most important factor affecting the quality of the logistics service. Table 2 shows the rank of SQ dimensions, used as a basis to create prerequisites for applying the FUCOM.

Table 2 shows the ranks obtained by the customers' responses. The method used to obtain these ranks is as follows: At the end of the questionnaire, all respondents determined the percentage for each dimension. After that, the sum of all values for one dimension was divided by 7000. The coefficient values for each dimension were obtained in the same way. Table 3 shows the percentages of dimensions for each dimension stated.

From Table 3, we can see that the sum of all percentage values is 7000. The procedure to obtain the weight coefficients is as follows: The sum of the percentage values of one dimension was divided by the sum of percentage values for all dimensions. The following example shows how to calculate the value of the weight coefficient for the dimension of assurance (*wj—*weight coefficient).

The weight coefficient value for the dimension of assurance is 0.2629:

$$w\_{j} = \frac{\text{sum of } percentage\text{ values for the dimension of ascurnac}}{\text{sum of } percentage\text{ values for all dimensions}}$$


**Table 2.** The ranks of dimensions by applying the Delphi method.


**Table 3.** Percentage values of five dimensions by 70 respondents.

#### *4.2. Determining the Weight Values of Dimensions Applying the FUCOM*

*Step 1*. In the first step, decision-makers need to rank criteria (dimensions). Compared to the original FUCOM, where the experts themselves perform the ranking, in this paper, the same was performed using the Delphi method based on the responses of 70 customers of logistics service. The dimensions ranking is as follows: *D*<sup>1</sup> > *D*<sup>2</sup> > *D*<sup>5</sup> > *D*<sup>3</sup> > *D*4.

*Step 2*. In the second step (Step 2b), the decision-maker performed the pairwise comparison of the ranked dimensions from Step 1. The comparison was made with respect to the first-ranked *D*1 dimension. In this step, it a team of five experts was formed who assessed previously ranked dimensions. The experts carried out the assessment based on the scale [1, 9]. Thus, the priorities of the dimensions (*Cj*(*k*) ) by the first decision-maker for all the criteria ranked in Step 1 were obtained (Table 4). Based on the obtained priorities of the dimensions, the comparative priorities of the dimensions were calculated: *ϕC*1/*C*<sup>2</sup> = 1.2/1 = 1.200, *ϕC*2/*C*<sup>5</sup> = 1.5/1.2 = 1.250, *ϕC*5/*C*<sup>3</sup> = 2.7/1.5 = 1.800, and *ϕC*3/*C*<sup>4</sup> = 3.2/2.7 = 1.185.


*Step 3*. The final values of the weight coefficients should meet the following two conditions:

(1) The final values of the weight coefficients should meet condition (3), i.e., that *<sup>w</sup>*<sup>1</sup> *<sup>w</sup>*<sup>2</sup> <sup>=</sup> 1.2, *<sup>w</sup>*<sup>2</sup> *<sup>w</sup>*<sup>5</sup> <sup>=</sup> 1.250, *<sup>w</sup>*<sup>5</sup> *<sup>w</sup>*<sup>3</sup> <sup>=</sup> 1.800 and *<sup>w</sup>*<sup>3</sup> *<sup>w</sup>*<sup>4</sup> = 1.185.

(2) In addition to condition (3), the final values of the weight coefficients should meet the condition of mathematical transitivity, i.e., that *<sup>w</sup>*<sup>1</sup> *<sup>w</sup>*<sup>5</sup> <sup>=</sup> 1.2 <sup>×</sup> 1.25 <sup>=</sup> 1.500, *<sup>w</sup>*<sup>2</sup> *<sup>w</sup>*<sup>3</sup> = 1.25 × 1.8 = 2.250, and *<sup>w</sup>*<sup>5</sup> *<sup>w</sup>*<sup>4</sup> = 1.8 × 1.185 = 2.133. By applying expression (5), the final model for determining the weight coefficients can be defined as:

$$\begin{array}{c} \min \chi\\ \text{s.t.} \begin{cases} \left| \frac{w\_1}{w\_2} - 1.200 \right| \le \chi\_{\prime} \left| \frac{w\_2}{w\_3} - 1.250 \right| \le \chi\_{\prime} \left| \frac{w\_3}{w\_3} - 1.800 \right| \le \chi\_{\prime} \left| \frac{w\_3}{w\_4} - 1.185 \right| \le \chi\_{\prime}\\ \left| \frac{w\_1}{w\_3} - 1.500 \right| \le \chi\_{\prime} \left| \frac{w\_2}{w\_3} - 2.250 \right| \le \chi\_{\prime} \left| \frac{w\_3}{w\_4} - 2.133 \right| \le \chi\_{\prime} \\ \sum\_{j=1}^{5} w\_j = 1, \ w\_j \ge 0, \forall j \end{cases} \\ \text{s.t.} \begin{cases} \left| \frac{w\_1}{w\_2} - 1.500 \right| \le \chi\_{\prime} \\ \left| \frac{w\_1}{w\_3} - 1.500 \right| \le \chi\_{\prime} \end{cases} \end{array}$$

By solving this model, the final values of the weight coefficients (0.315, 0.263, 0.210, 0.113, 0.099) *T* and *DFC* of the results *χ* = 0.000 were obtained. Weight coefficient values are shown in the ranked order of dimensions from the first step. The individual values of weight coefficients for all dimensions were obtained in the same way. Table 5 shows dimension ratings according to all criteria and values of weight coefficients using the previously demonstrated steps. The final values of weight coefficients of the dimension of reliability (*D*<sup>1</sup> = 0.291), assurance (*D*<sup>2</sup> = 0.259), tangibles (*D*<sup>3</sup> = 0.130), empathy (*D*<sup>4</sup> = 0.109), and responsiveness (*D*<sup>5</sup> = 0.207) were calculated using the geometric mean.


**Table 5.** Priorities of dimensions by five experts and obtained weights of dimensions.

#### *4.3. The Frequency of Responses*

The frequency of the occurrence of a response is called the frequency of responses. As mentioned earlier, when filling out a questionnaire, customers used a Likert scale, or more precisely for each question, they assigned a point from one to five: 1—completely disagree; 2—partially disagree; 3—have no opinion; 4—partially agree; 5—completely agree. According to the frequency of responses, customers had extremely high expectations because they only responded 14 times with a rating of 1 and 769 times with the highest rating. Compared to the frequency of responses in terms of expectations, a significant difference can be noticed for rating 1, but also a difference for the highest rating. Based on the response frequency, it can be assumed that there will be significant differences between customer expectations and perceptions of the service provided. There were 33 responses with a rating of 1, which further implies that a certain number of customers are dissatisfied with the service provided. In addition, while perceiving the service provided, customers mostly gave a rating of 5, and then a rating of 4.

Figure 2 shows a graph of customers' responses in terms of expectations and perceptions. Regarding expectations, only one customer assigned the lowest rating to *Q*4, while we had more responses with the lowest rating referring to perceptions. *Q*<sup>3</sup> did not record any of the lowest ratings regarding either expectations or perceptions. From the aspect of perceptions, *Q*<sup>1</sup> recorded the highest number of answers with the highest rating, namely 35, compared to the expectations where 30 customers responded with a rating of 5. For *Q*2, customers expressed great satisfaction, where in terms of expectations, 26 customers responded with a rating of 5, while 34 customers responded with the same rating for the same question regarding perceptions. The lowest rating for *Q*<sup>5</sup> was given by two customers, while no response with a rating of 1 was given for expectations. The number of customers for the same question with the highest rating from the aspect of perception was 33, while 31 customers marked 5 regarding expectations for the question. Customers also expressed satisfaction with *Q*<sup>6</sup> with a rating of 4, i.e., regarding expectations, 18 customers marked 4, while in response to perceptions, 25 customers responded by that rating. Generally, it can be noticed that the quality of the service provided is very high for this dimension.

After the dimension of reliability, high satisfaction was expressed for the dimension of assurance (Figure 3). No significant difference in customers' responses regarding expectations and perceptions was noticed for *Q*7. For each question of that dimension, the response with the lowest rating was recorded. *Q*<sup>8</sup> recorded 35 responses with a rating of 5 when perceived by customers, while the same rating was assigned to expectations by 30 customers. Three customers responded by rating 1 for *Q*9 regarding perception. For the same question, there is a difference in rating 5, where the highest rating was given by 33 customers regarding the perception, and when the expectation was recorded, the rating was recorded by 29 customers. *Q*10, the last question in the dimension of assurance, had the highest number of responses, with a rating of 5. Namely, customer satisfaction can be seen by the number of customers' responses, with a rating of 2 and 5. Regarding expectations, 12 customers responded with a rating of 2, while five customers less responded with the same rating for perceptions. The highest mark, rating 5, was selected by 31 customers for expectations while regarding perceptions, 37 customers responded to *Q*<sup>10</sup> with a rating of 5.

The results of the dimension of tangibles (Figure 4) are specific because of customers' responses to *Q*11. Generally, the *Q*<sup>11</sup> results did not significantly affect the overall customer satisfaction. Concerning expectations, three customers selected a rating of 1, while 8 customers responded with the same rating for perceptions. Additionally, a rating of 2 was given by six customers, while 14 customers responded with a rating of 2 for perceptions. The customer dissatisfaction for *Q*<sup>11</sup> can be noticed by the number of customers' responses with a rating of 4 where, in reference to expectations, 28 customers responded with that rating, while after the service provided, 18 customers responded with a rating of 4. The total satisfaction of the customers for assessing the tangibles was influenced by the results of *Q*12. For question *Q*12, after the service was provided, 38 customers responded with a rating of 5, while for the same question, when responding to expectations, 29 customers answered with a rating of 5. For question *Q*13, it is also possible to notice the difference in customers' responses for the highest rating. With regard to expectations, 29 customers selected a rating of 5, while the same rating after the perception of the service was selected by 39 customers. Rating 4 for *Q*<sup>14</sup> was chosen by the same number of customers, namely 26. After the service was provided, 34 customers answered with a rating of 5 for that question, while 32 customers responded with the highest rating regarding perceptions. The great satisfaction of customers concerning the dimension of tangibles was expressed for *Q*15. Before the service was provided, a rating of 5 was selected by 26 customers, and after the service

was perceived, 39 customers answered with the highest rating. For *Q*16, there was also a significant difference expressed by rating 4 and 5. For that question, before the service was provided, 23 customers answered with a rating of 4, and 27 customers after its realization. Rating 5 was given by three customers more after the service was provided.

**Figure 2.** Graph of customers' responses regarding the dimension of reliability (**left**-expectations and **right**- perceptions).

**Figure 3.** Graph of customers' responses regarding the dimension of assurance (**left**-expectations and **right**-perceptions).

**Figure 4.** Graph of customers' responses regarding the dimension of tangibles (**left**-expectations and **right**-perceptions).

From Figure 5, it can be seen that there is very little positive difference in terms of customer perception. For customer expectations, one response with a rating of 1 was noted for *Q*21. Customers expressed satisfaction for *Q*17, where 35 customers responded with a rating of 5 after the service was provided, and 30 customers responded with the same rating before its realization. Concerning *Q*18, a large number of customers (34) responded with a rating of 5, while 39 customers answered with the same rating after the service was provided. Question *Q*<sup>19</sup> was the only question that customers did not answer with a rating of 1 after the service was perceived. In addition, customers had high expectations for *Q*19, i.e., 33 customers answered with a rating of 5, while 38 customers answered with the same rating after the service was provided. In reference to *Q*20, 32 customers responded with a rating of 4 in terms of perceptions, while 25 responses were recorded with the same rating regarding expectations. For the same question, the diagram shows a much larger number of responses with a rating of 3 from the aspect of customer expectations, where 11 customers responded with that rating, and after the realization, only four customers responded with a rating of 3. Concerning question *Q*21, customers had very high expectations, with 44 customers responding with a rating of 5. A slight decrease in satisfaction could be noticed after the service was provided, where 44 respondents answered *Q*<sup>21</sup> with a rating of 5.

In Figure 6, in terms of customer expectations, it can be noticed that there were no responses with a rating of 1. Final survey results indicated that there was no difference between the customer expectations and perceptions of the quality of the service provided. Concerning *Q*22, 41 customers responded with a rating of 5 for perceptions, while regarding expectations, 33 customers answered with a rating of 5 for the same question. For *Q*23, customers did not generally express satisfaction, where 37 customers responded with a rating of 5 regarding expectations, and 34 customers responded with the highest rating after the service was provided. The number of customers' responses to *Q*<sup>23</sup> with a rating of 4 was the same, i.e., 24 responses for both aspects of the SERVQUAL model. Question *Q*<sup>24</sup> showed a small positive difference in customer satisfaction, i.e., 35 respondents answered with a rating of 5 prior to the service being provided, and after its realization, 38 customers responded with the highest rating. In the diagram, the biggest positive difference can be identified for question *Q*25. The number of customers who answered with a rating of 5 for that question regarding expectations was 28, and after the service was provided, 38 customers responded with a rating of 5. The positive difference regarding the last question, *Q*25, had a significant impact on the ultimate result of the dimension of responsiveness.

**Figure 5.** Graph of customers' responses regarding the dimension of empathy (**left**-expectations and **right**-perceptions).

**Figure 6.** Graph of customers' responses regarding the dimension of responsiveness (**left**-expectations and **right**-perceptions).

#### **5. Research Results**

#### *5.1. The Results of Dimensions in Terms of Customer Expectations*

Table 6 shows the results of dimensions with expectations. Dimensions are presented in terms of expectations and their average value, standard deviation, weight coefficients, and the value of the Cronbach alpha coefficient.


**Table 6.** The results of dimensions with customer expectations.

The Cronbach alpha test is considered positive only if coefficients above 0.7 are obtained. Certain sources state that a reliable value of the Cronbach alpha test is 0.5. From the table of percentage values, it can be seen that customers have the highest expectations regarding the reliability dimension, and the least expectations regarding the dimension of empathy. The average value for the dimension of responsiveness was 4.282, which is the highest average value. For the dimension of reliability, there

is the smallest average value and it was 4.029. The standard deviation for all dimensions was 0.923 and the average value of the Cronbach alpha test for all dimensions was 0.876. The weight coefficient values for both expectations and perceptions were the same.

#### *5.2. Results of Dimensions in Terms of Customer Perceptions*

Table 7 shows the results of dimensions regarding customer perceptions.

From the previous two tables, it can be seen that the value of the Cronbach alpha test was far above 0.7, which means that the dimensions are reliable. The highest quality perceptions were for the dimension of empathy, 4.360, and then for the dimension of responsiveness, 4.282. The lowest perceptions were related to the reliability dimension and were 4.176. It can be seen that the values for the dimension of responsiveness were the same for both expectations and perceptions, which means that there were no significant changes in relation to the quality of the service provided. In addition, it can be seen that the values for the dimension of responsiveness were the least from both aspects.

Table 8 shows the results obtained by using the developed Delphi-FUCOM-SERVQUAL methodology. Table 8 shows the difference between customer perceptions and expectations. Generally, customers are satisfied with the quality of the logistics service of the express post company. For all dimensions except for the dimension of responsiveness, the result is positive. It can be noticed that the greatest satisfaction of customers was expressed for the dimension of reliability. According to customers' percentage rating, the dimension of reliability is the most important of all the five dimensions for customers. The results of the dimension of responsiveness remains the same, as the expectations of customers are equal to their perceptions. The questions of the reliability dimension included a part of the logistics service where the company can create the biggest improvements.

**Table 7.** The results of dimensions regarding customer perceptions.



Table 9 outlines the questions of responsiveness from the aspect of customer perceptions. According to the results, after the responsiveness dimension, the dimension of tangibles with +0.064 also has opportunity for improvement. From the results obtained, it can be seen that *Q*<sup>11</sup> has a significant impact on the quality of this dimension. Namely, three customers from the aspect of expectations gave the lowest rating for this question, and after the service was provided, eight customers gave the lowest rating for that question. Concerning expectations, for the same question, six customers selected a rating of 2, or "partially disagree", while regarding perceptions, 14 customers responded with "partially disagree".

According to the analysis, customers expressed the highest satisfaction for the dimension of reliability, with +0.0392. The dimension of reliability is focused on delivery quality and delivery time. For all six questions of the reliability dimension, customers gave higher ratings than the ratings in terms of customer expectations.

**Table 9.** Statements for the dimension of responsiveness in terms of customer perceptions.


#### *5.3. Statistical Analysis*

For the set of expectations and perceptions, *n* ∈ *N* = 70 (Table 10), so that the parameter of binomial distribution for *n* ∈ [1, 5] can fully substitute mathematical expectation.



The mean value of the binomial distribution parameter for expectations was *pE* = 0.8307, and the mean value of the binomial distribution parameter for perceptions was *pp* = 0.8477. Between these values, there was a high significant correlation of mean values of *p* = 0.9303.

Although the distributions of expectations and perceptions are the same and in most cases, they have nonparametric correlation (not in one case out of 25, *E*15/*P*15), it should be noticed that the coefficients of liner correlations are, on average, small and of a normal distribution *N* (0.2562; 0.0233), with the significance threshold of *p* = 0.5708. This means that there is a large fluctuation between expectations and perceptions, i.e., there is a large number of respondents who had high expectations, but were disappointed with perceptions and vice versa.

The ANOVA test significantly confirmed that there are 17 out of 25 such cases. The Duncan test of post-hoc ANOVA was used to determine the mean values of perceptions for the factor of expectation estimates for all variables, where the variance analysis showed significant differences.

In Table 11, the values of the mean estimates for perceptions for given estimates of expectation where the ANOVA analysis has identified significant differences are given.

Based on the established mean values of binomial distribution parameters for expectations, *pE* = 0.8307 and *pP* = 0.8477 and for *N* = 70, we can estimate the mean number of respondents who, according to binomial distribution, assigned the ratings *n* ∈ [1, 5]. The expected average number of respondents who selected one of the ratings for expectations and perceptions is given in Table 12.



**Table 12.** Calculation of the average number of respondents based on the binomial distribution parameters, *pE* and *pP*, the number of respondents, *N* = 70, and ratings, *n* ∈ [1, 5].


#### *Further as follows:*

The respondents with the expected rating, *E*(*n*) = 1, provided an average perception estimate of 2.0916, so we can conclude that the increase in the values of estimates was not significant, with *p* = 0.1616 (the minimum number of respondents adopted for one side test difference between two means was two for expectations and perceptions).

Respondents with the expected rating, *E*(*n*) = 2, provided an average perception estimate of 4.2329, so we can conclude that the increase in values of estimates was significant, with *p* = 0.0176 (the minimum number of respondents adopted for two side test differences between two means was two for expectations and perceptions). Respondents with the expected rating, *E*(*n*) = 3, provided an average perception estimate of 3.7761, so we can conclude that the increase in values of estimates was significant, with *p* = 0.0002 (the number of respondents adopted for two side test differences between two means was eight for expectations and seven for perceptions). Respondents with the excepted rating, *E*(*n*) = 4, provided an average perception estimate of 4.0590, so we can conclude that the increase in values of estimates was not significant with *p* = 0.0970 (the number of respondents adopted for two side test differences between two means was 27 for expectations and 26 for perceptions).

Respondents with the expected rating, *E*(*n*) = 5, provided an average perception estimate of 4.4621, so we can conclude that the decrease in values of estimates was significant, with *p* = 0.0000 (the number of respondents adopted for one side test difference between two means was 33 for expectations and 36 for perceptions)

#### *To conclude:*

Respondents who had low expectations of 2 or 3 significantly identified a perception increase to 4.0590 or 3.7761, respectively, but respondents with high expectations of 5, significantly reduced their perceptions to 4.4621.

Respondents who had expectations of 4 significantly maintained the same level of 4.0590. Considering the above, there is a stable rating of 4 for expectations, which can be adopted as the company's final assessment.

Regarding the system of expectation and perception assessment:


The impact of expectations (*E*) as a factor on reliability, assurance, tangibles, empathy, and responsiveness is given in the Table 13. The variance analysis identified one significant case of the impact of expectations on reliability (*E*03), four on empathy (*E*05, *E*08, *E*13, *E*15, and *E*18), and two on responsiveness (*E*<sup>03</sup> and *E*20). The expectations of assurance and tangibles had no impact.


**Table 13.** The variance analysis of influencing factors of expectations on reliability, assurance, tangibles, empathy, and responsiveness.

Duncan's test of post-hoc ANOVA revealed the values that led to the emphasis of factors as follows (Table 14):


For a significant influence of expectations, it is necessary to have at least three significant differences (*p*(2) = 0.0745 > 0.05, no significant influence, *p*(3) = 0.0370 < 0.05 influence was significant) for one of the dimensions (reliability, assurance, tangibles, empathy, and responsiveness). With the significance threshold of *p*(5) = 0.0092, we confirm the significant influence of expectations on empathy.

The impact of perceptions (*P*) as a factor on reliability, assurance, tangibles, empathy, and responsiveness is given in Table 15. The variance analysis determined one significant case of the impact of expectation on assurance (*P*18), two on tangibles (*P*09, *P*21), and three on empathy (*P*08, *P*10, and *P*23). Perceptions had no influence on reliability and responsiveness.

**Table 14.** Calculation of the attribute mean values for a given rating of significant expectation influence.




Duncan's test of post-hoc ANOVA revealed the values that led to the emphasis of factors as follows (Table 16):


**Table 16.** Calculation of the attribute mean values for given ratings of significant perception influence.


Regarding expectations, for a significant influence of perceptions, it is necessary to have at least three significant differences for one of the dimensions (reliability, assurance, tangibles, empathy, and responsiveness), which was only recorded for empathy. With the significance threshold, *p*(3) = 0.0370, we confirm the significant influence of perceptions on empathy.

A particularly specific case is the empathy function as a variable depending on expectation *E*<sup>08</sup> and perception *P*08, which at the same time, had a significant impact on empathy. From the graph, it is evident that respondents who had low expectations (1 or 2) and identified great perceptions (4 or 5) had excessively high empathy (15 to 20), which was likely to be generated as a reactive compensation to the determined difference between perceptions and expectations (Figure 7).

**Figure 7.** Empathy function as a variable depending on expectation *E*<sup>08</sup> and perception *P*08.

Here, it is necessary to recall that a significant difference between expectation *E*<sup>08</sup> and perception *P*<sup>08</sup> was determined in the distribution changes, and it is also necessary to notice that the difference of binomial parameters (which is analogous to mathematical expectation) had the largest increase (+0.0629), particularly in the difference between the binomial parameters for *P*<sup>08</sup> and *E*08.

Between the impact of expectations on reliability (1 of 25) and the impact of perceptions on reliability (0 of 25), there was no significant difference, *p* = 0.3145 > 0.05.

Between the impact of expectations on assurance (0 of 25) and the impact of perceptions on assurance (1 of 25), there was no significant difference, *p* = 0.3145 > 0.05.

Between the impact of expectations on tangibles (0 of 25) and the impact of perceptions on tangibles (2 of 25), there was no significant difference, *p* = 0.1525 > 0.05.

Between the impact of expectations on empathy (5 of 25) and the impact of perceptions on empathy (3 of 25), there was no significant difference, *p* = 0.5755 > 0.05 (expectations and perceptions had a significant impact on empathy, but there were no differences between their significant impacts).

Between the impact of expectations on responsiveness (2 of 25) and the impact of perceptions on responsiveness (0 of 25), there was no significant difference *p* = 0.1525 > 0.05.

The others (*R*, *A*, *T*, *R*) had no significant impact, and there was no significant difference between them, too.

#### **6. Conclusions**

By applying appropriate scientific tools and techniques, it is possible to make improvements from a professional aspect in different areas, one of which is certainly quality management. In this paper, therefore, a new Delphi-FUCOM-SERVQUAL methodology was developed to improve the process of service quality measurement. The company where the case study was conducted provides express post services, so it can be said that this paper has a twofold contribution. The first contribution relates to a scientific aspect that implies the development of an integrated methodology to improve a quality measurement process that can be applied without any restrictions in various areas. The advantages of the developed methodology are reflected in the fact that it enables precision treatment of input and output parameters and provides results that are more objective. In addition, from a professional aspect of the study, it is possible to determine the quality and efficiency of the company based on the satisfaction of its customers, but it also enables further application and re-application of this methodology. This methodology can be very helpful for strategic management of the company to improve their efficiency. Considering all the relevant factors, it is possible to conclude that this paper contributes to the overall literature, enriching it in a certain way, as it provides future researchers with a new methodology that more precisely treats input parameters and achieves better results than traditional quality measurement methods.

All contributions and conclusions were confirmed throughout a comprehensive and detailed statistical analysis in which even the regularity of interaction between certain questions was established. The Cronbach alpha coefficient showed the reliability of the formed questionnaire, while ANOVA showed that there was a large fluctuation between expectations and perceptions, i.e., there was a large number of respondents who had high expectations and were disappointed with perceptions, and vice versa. Considering it at the general level, the research conducted on the system of estimating expectations and perceptions shows that: *There were no significant quantitative differences between expectations and perceptions,* which means that the hypothesis set in the paper was confirmed. Most of the estimates were significantly binomially distributed with approximately the same parameter, as confirmed by the Signum test in 24 out of 25 estimates. From the aspect of qualitative differences, there was significance in assessing expectations and perceptions, which was contained in the fluctuation towards a stable rating of "4". These differences support the objectivity of the respondents and the concept of assessment, the correctness of the questions asked, etc., and realistically evaluate the company with a rating of 4. Future research related to this paper may imply the improvement of the proposed methodology by defining a universal linguistics scale for expressing customer satisfaction. In addition, depending on specific cases, it is possible to modify the structure of dimensions within the SQ questionnaire.

**Author Contributions:** Each author has participated and contributed sufficiently to take public responsibility for appropriate portions of the content.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **A Novel Approach for Green Supplier Selection under a q-Rung Orthopair Fuzzy Environment**

#### **Rui Wang <sup>1</sup> and Yanlai Li 1,2,\***


Received: 9 November 2018; Accepted: 26 November 2018; Published: 1 December 2018

**Abstract:** With environmental issues becoming increasingly important worldwide, plenty of enterprises have applied the green supply chain management (GSCM) mode to achieve economic benefits while ensuring environmental sustainable development. As an important part of GSCM, green supplier selection has been researched in many literatures, which is regarded as a multiple criteria group decision making (MCGDM) problem. However, these existing approaches present several shortcomings, including determining the weights of decision makers subjectively, ignoring the consensus level of decision makers, and that the complexity and uncertainty of evaluation information cannot be adequately expressed. To overcome these drawbacks, a new method for green supplier selection based on the q-rung orthopair fuzzy set is proposed, in which the evaluation information of decision makers is represented by the q-rung orthopair fuzzy numbers. Combined with an iteration-based consensus model and the q-rung orthopair fuzzy power weighted average (q-ROFPWA) operator, an evaluation matrix that is accepted by decision makers or an enterprise is obtained. Then, a comprehensive weighting method can be developed to compute the weights of criteria, which is composed of the subjective weighting method and a deviation maximization model. Finally, the TODIM (TOmada de Decisao Interativa e Multicritevio) method, based on the prospect theory, can be extended into the q-rung orthopair fuzzy environment to obtain the ranking result. A numerical example of green supplier selection in an electric automobile company was implemented to illustrate the practicability and advantages of the proposed approach.

**Keywords:** green supplier selection; q-rung orthopair fuzzy set; consensus-reaching process; the q-ROFPWA operator; TODIM method

#### **1. Introduction**

During the past decades, environmental issues have been receiving more and more attention; certain enterprises, especially in the developing countries, have made great efforts in the fields of sustainable development and pollution prevention to face the environmental pressures [1]. These environmental pressures are rooted in two aspects, namely, through government or consumer [2]. The governments have promulgated a series of environmental laws and regulations to restrict the behavior of enterprises; consumers may take the environmental impact of different enterprises into account when making their choices. Therefore, more and more enterprises apply the novel environmental management mode of green supply chain management (GSCM) to reduce the pollution during the operation processes of supply chains [3–6]. GSCM involves many aspects of a supply chain, namely, product design, supplier selection, production, packaging, transportation, marketing, and recycling [7,8]. Among the different segments, green suppliers are the initial link of a supply chain and affect the efficiency and environmental performance of the supply chain; thus, the green supplier

selection plays a key role in GSCM [9–11]. To solve the complex green supplier selection problems in practice, many scholars have proposed different green supplier selection approaches [11].

Essentially, green supplier selection can be regarded as a multiple criteria group decision making (MCGDM) problem where decision makers evaluate several potential green suppliers with respect to some criteria to determine the best alternative [8,12,13]. In practice, the evaluation information may be uncertain and incomplete. The fuzzy set (FS) and its generalized forms have been widely used in the current literature to solve this problem [14–16]. Q-rung orthopair fuzzy set (q-ROFS), which was developed by Yager [17], can express the membership, non-membership, and indeterminacy membership degrees of decision makers, simultaneously. Scholars have introduced the q-ROFS to many practical fields, such as investment, enterprise resource planning system selection, and so on [18–20]. Therefore, to deal with the increasing complexity of green supplier selection, decision makers can express a wider range of evaluation information by using the q-ROFS to evaluate the potential green suppliers.

During the process of green supplier selection, decision makers may differentiate from the research fields and practical experiences; thus, the evaluation information of different decision makers will vary widely. However, under the premise of cooperation between decision makers, the ranking result with a relatively high consensus level is desirable [21,22]. In real life, unanimity is difficult or impossible to achieve; the concept of soft consensus was proposed to solve these MCGDM problems. Furthermore, the consensus model has been applied in many practical areas [23–25]. To the best of our knowledge, little attention has been paid to the green supplier selection approaches that include the consensus-reaching process. Therefore, we developed an iteration-based consensus model under the q-rung orthopair fuzzy (q-ROF) environment, which can offer suggestions for decision makers on how to revise their non-consensus evaluation information in each iteration round. Consequently, the consensus model is used during the green supplier selection process to obtain a more accurate ranking result.

The individual acceptable consensus evaluation matrix of each decision maker can be obtained by the consensus model; thus, the next issue is how to aggregate this evaluation information to determine a collective evaluation matrix. Due to the different backgrounds of decision makers in practice, the weights of them will always be difficult to determine simply. Most existing green supplier selection approaches determine the weights of decision makers using the subjective weighting methods or assume that the decision makers are equivalent important, which is inconsistent with the actual situations and may lead to an inaccurate ranking result. To address this problem, Yager [26] proposed the power average (PA) operator, in which the weights of aggregated arguments are determined by the support degrees of them objectively. Since then, the PA operator has been investigated by many scholars to propose its generalized forms under different fuzzy environments; the decision maker weights can be determined by considering the subjective and objective factors, simultaneously. In this paper, the q-rung orthopair fuzzy power weighted average (q-ROFPWA) operator, which was proposed by Liu et al. [27], is utilized during green supplier selection to complete the information fusion effectively.

Since the collective evaluation matrix of potential green suppliers was determined, we needed to obtain the ranking index of each green supplier. Because the evaluation behavior of decision makers is bounded rational, the attitude towards gain and loss of decision makers should be considered while determining the final ranking of green suppliers [8]. Nevertheless, most existing green supplier selection approaches ignored this bounded rationality behavior of decision makers. Inspired by the literature [8], we introduced the TODIM (TOmada de Decisao Interativa e Multicritevio) method to deal with these situations. Gomes and Lima [28] developed this TODIM method, in which the bounded rationality is considered according to the prospect theory [29]. The utility function is introduced to compute the dominance degree of each alternative over all the alternatives; then, the global values of alternatives can be obtained to determine the best alternative. Therefore, in this paper, the q-rung orthopair fuzzy TODIM (q-ROF-TODIM) method was put forward to determine the ranking result of green suppliers.

According to the discussion above, this paper proposes an improved green supplier selection approach based on q-ROFS and TODIM method. The main contributions of this study are presented as in the following. (1) The q-ROFS was used to express the evaluation information of decision makers, which can deal with the uncertainty and complexity of evaluation information in practice effectively. (2) The non-consensus evaluation information could be improved by an efficient iteration-based consensus model to obtain a ranking result that was accepted by decision makers or enterprise. (3) Considering the objective and subjective factors of the decision maker weights, the q-ROFPWA operator was introduced to aggregate the individual evaluation information. (4) The TODIM method under q-ROF environment was constructed to obtain the ranking that reflects the bounded rationality of decision makers. To achieve this, the rest of this paper is presented as follows. The related literature is reviewed in Section 2. The definition, operations, comparison method, distance measure, and aggregation operator of q-ROFS are introduced in Section 3. Section 4 proposes a novel approach for green supplier selection. Section 5 applies a numerical example to show the feasibility and validity of the proposed approach. Some conclusions are summarized in Section 6.

#### **2. Literature Review**

#### *2.1. Green Supplier Selection Approaches*

As the MCGDM problems become more and more complicated, many novel approaches based on MCGDM methods or soft computing were investigated [30–32]. Similarly, due to the features of green supplier selection, many scholars have researched the green supplier selection method by regarding it as a complex MCGDM problem; thus, a series of MCGDM methods under fuzzy environments have been applied into the research of green supplier selection. For example, Lee et al. [33] developed a fuzzy analytic hierarchy process (AHP) approach for green supplier selection in a high-tech industry. Both Chen et al. [34] and Yazdani [35] constructed an integrated fuzzy multiple criteria decision making approach to obtain the best green supplier, which is composed of fuzzy AHP and technique for order performance by similarity to ideal solution (TOPSIS) methods. Combined with AHP and entropy, elimination and choice expressing the reality III (ELECTRE III) methods, Tsui and Wen [36] proposed an approach for selecting a green supplier, and several improvement suggestions were presented to raise the performance of suppliers. Kannan et al. [9] determined the best green supplier for an engineering plastic material manufacturer in Singapore by using a fuzzy axiomatic design method. Dobos and Vörösmarty [37] evaluated green suppliers with respect to composite indicators based on the data envelopment analysis (DEA) method. Hashemi et al. [38] determined the ranking of green suppliers in GSCM by a comprehensive method that consisted of the analytic network process (ANP) and grey relational analysis (GRA) methods. Kuo et al. [39] integrated the artificial neural network (ANN), ANP, and DEA methods to choose suppliers by considering the environmental regulations. Kuo et al. [40] utilized the decision-making trial and evaluation laboratory (DEMATEL)-based ANP method to investigate the relationships between the criteria and compute the weights of criteria, and then selected the green suppliers combined with the VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje) method. To discuss the applications of fuzzy green supplier selection approaches, Banaeian et al. [14] evaluated the green suppliers in the agri-food industry by using the TOPSIS, VIKOR, and GRA methods, respectively. Both Qin et al. [8] and Sang and Liu [41] expressed the uncertainty of evaluation information using an interval type-2 fuzzy set, then utilized the TODIM method to obtain the ranking of green suppliers. Govindan et al. [42] put forward a green supplier selection method based on the revised Simos procedure and preference ranking organization method for enrichment evaluation (PROMETHEE) method. Quan et al. [43] investigated the green supplier selection with a large-scale group of decision makers and developed an integrated method

combined with ant colony algorithms and multi-objective optimizations by ratio analysis plus the full multiplicative form (MULTIMOORA) method.

#### *2.2. Q-ROFS*

In practice, the related qualitative and quantitative data of green suppliers are always incomplete and complex; thus, crisp numbers cannot express the uncertainty of evaluation information given by decision makers. To solve this problem, Zadeh [44] developed the FS theory to represent the evaluation information; the generalized fuzzy numbers, including triangular fuzzy numbers and type-2 fuzzy numbers, were widely used in approaches for green supplier selection [8,14,33,41]. However, the FS ignores the non-membership degree of evaluation information. For instance, a business manager evaluates an investment before investing; they might think the probability of profit is 0.6, and the probability of loss is 0.3. Obviously, the FS cannot represent the aforementioned evaluation information. Therefore, Atanassov [45] applied the non-membership degree to improve the FS, and proposed the intuitionistic fuzzy set (IFS). Consequently, the evaluation information of the business manager can be expressed by an IFS, i.e., the membership and non-membership degrees are 0.6 and 0.3, respectively. Afterwards, IFS has been applied into green supplier selection [16,46]. Yager [47] proposed a generalized form of IFS called the Pythagorean fuzzy set (PFS), in which the sum of squares of membership and non-membership degrees is less than 1. Furthermore, to provide decision makers with a more relaxed evaluation environment, Yager [17] put forward the q-ROFS theory to express more potential evaluation information of decision makers. Then, the generalized form of q-ROFS, i.e., the q-rung picture linguistic set was proposed [48]. The q-ROFS theory can be regarded as a generalized form of IFS [45] and PFS [47], and the space of acceptable orthopairs increased with the increasing rung q as shown in Figure 1. Combined with the refusal membership degree, Cuong [49] developed the picture fuzzy set; subsequently, the similarity measures of the generalized picture fuzzy sets that including spherical fuzzy sets and T-spherical fuzzy sets were investigated [50]. However, picture fuzzy set is more applicable to model phenomena like voting.

**Figure 1.** Geometric space range of the intuitionistic fuzzy set (IFS), Pythagorean fuzzy set (PFS), and q-rung orthopair fuzzy set (q-ROFS).

#### *2.3. Consensus Model*

During the green supplier selection process, decision makers may come from different research fields of GSCM and have varying degrees of domain experience. Therefore, the non-consensus evaluation information, which is far from group opinions, will be inevitably revealed; however, a ranking result with a low consensus level may be obtained, which is not desirable. In recent years, the consensus model of MCGDM problems has been a hot topic. The existing consensus models can be divided into two categories; one is the iteration-based consensus model. For example, Herrera-Viedma et al. [51] defined the consensus and proximity measures of different preference structures to construct an iteration-based consensus support system. Herrera-Viedma et al. [52] investigated the consistency and consensus of incomplete fuzzy preference relations; a feedback mechanism was put forward to revise the consistency and consensus levels, simultaneously. With respect to intuitionistic fuzzy preference relations, Chu et al. [53] developed two iteration-based algorithms to improve the consistency and consensus levels, respectively. Wu et al. [54] put forward an iteration-based consensus model to revise the incomplete linguistic information, in which the trust degree was used to complete the decision matrices and adjust the weights of decision makers. Wu and Xu [55] developed a consensus measure of hesitant fuzzy linguistic information to complete the consensus-reaching process. Another kind of consensus model is the optimization-based consensus model. For instance, Dong et al. [56] constructed an optimization programming model for minimizing the number of adjusted simple terms to complete the consensus-reaching process under hesitant linguistic environment. Gong et al. [57] constructed two consensus models according to the minimum cost of consensus and maximum return to achieve a relatively high consensus level. Gong et al. [58] put forward a consensus model for optimizing the economic efficiency. Based on the multiplicative consistency, Xu et al. [59] and Zhang and Pedrycz [60] proposed goal programming models to improve the consistency and consensus levels of intuitionistic fuzzy preference and intuitionistic multiplicative preference relations, respectively. For green supplier selection issues, Zhu and Li [12] introduced a consensus model to put forward a novel green supplier selection approach; nevertheless, the consensus model can only provide suggestions to one of the decision makers for revising the non-consensus evaluation information in each iteration round, an thus, it can take a lot of time to achieve consensus in a complex environment.

#### *2.4. The PA Operator*

Based on the individual evaluation information, the collective information of each green supplier with respect to the criteria can be obtained by aggregation tools. Considering the relationships between the input information, Yager [26] developed the PA operator to aggregate the individual information. According to the PA operator and generalized weighted average operator, Zhou et al. [61] proposed the generalized power weighted average (GPWA) operator, in which the weight vectors were obtained by the subjective weight values and support degrees between different aggregated arguments. Xu [62] extended the PA operator into the intuitionistic fuzzy (IF) and interval-valued intuitionistic fuzzy environment; then, the generalized power weighted average operators were defined. Wan [63] put forward the MCGDM method by using a trapezoidal intuitionistic fuzzy power weighted average (TIFPWA) operator. Furthermore, Liu and Liu [64] investigated the generalized form of a TIFPWA operator. He et al. [65] discussed the properties of the interval-valued intuitionistic fuzzy power weighted average (IVIFPWA) operator and developed a novel MCGDM approach based on the IVIFPWA operator. According to the Frank operational laws, Zhang et al. [66] proposed a new form of PA operator. Wei and Lu [67] developed the Pythagorean fuzzy power weighted average (PFPWA) operator. To aggregate the q-rung orthopair fuzzy numbers (q-ROFNs), Liu et al. [27] extended the PA operator to the q-ROFPWA operator. Furthermore, the generalized PA operators have been applied to many practical areas to solve MCGDM problems [62,68–71].

#### **3. Preliminaries**

To make this paper as self-contained as possible, this section introduces the definition, operational laws, comparison method, Minkowski distance, and aggregation operator of q-ROFS, that will be utilized in the subsequent research.

#### *3.1. q-ROFS*

Based on the IFS and PFS, Yager [17] proposed a more general form, i.e., q-ROFS, and developed the operations of q-ROFS.

**Definition 1 [17].** *Let X be a non-empty and finite set, A q-ROFS Q on X* is defined by:

$$\mathcal{Q} = \{ \langle x, (\mu\_{\mathcal{Q}}(x), v\_{\mathcal{Q}}(x)) \rangle | x \in \mathcal{X} \}, \tag{1}$$

*where the functions μ<sup>Q</sup>* : *X* → [0, 1] *and vQ* : *X* → [0, 1] *represent the membership and non-membership degrees of x* ∈ *X to Q, respectively, and they satisfy the condition of* - *μQ*(*x*) *<sup>q</sup>* + - *vQ*(*x*) *<sup>q</sup>* <sup>≤</sup> 1, *<sup>q</sup>* <sup>≥</sup> <sup>1</sup>*. Furthermore, function πQ*(*x*) = *<sup>q</sup>* A- *μQ*(*x*) *<sup>q</sup>* + - *vQ*(*x*) *<sup>q</sup>* <sup>−</sup> - *μQ*(*x*) *q*- *vQ*(*x*) *<sup>q</sup> indicates the indeterminacy membership degree. For convenience, we call a* = (*μ*, *v*) a q-ROFN.

**Definition 2 [17].** *Let a* = (*μ*, *v*)*, a*<sup>1</sup> = (*μ*1, *v*1)*, and a*<sup>2</sup> = (*μ*2, *v*2) *be three q-ROFNs, λ* > 0*, and a<sup>c</sup> is the complementary set of a, then:*

$$a^\varepsilon = (\upsilon, \mu);$$

$$(2)$$

$$a\_1 \oplus a\_2 = \left( \sqrt[q]{(\mu\_1)^q + (\mu\_2)^q - (\mu\_1)^q (\mu\_2)^q}, \upsilon\_1 \upsilon\_2 \right);\tag{3}$$

$$a\_1 \odot a\_2 = \left(\mu\_1 \mu\_2, \sqrt[q]{(v\_1)^q + (v\_2)^q - (v\_1)^q (v\_2)^q}\right);\tag{4}$$

$$
\lambda a = \left( \sqrt[q]{1 - \left(1 - \mu^q\right)^{\lambda}}, v^{\lambda} \right); \tag{5}
$$

$$a^{\lambda} = \left(\mu^{\lambda}, \sqrt[\ell]{1 - \left(1 - \upsilon^{q}\right)^{\lambda}}\right). \tag{6}$$

**Example 1.** *Suppose that a*<sup>1</sup> = (0.6500, 0.8298) *and a*<sup>2</sup> = (0.5000, 0.7500) *are two q-ROFNs, q* = 3 *and λ* = 2*, then:*


Liu et al. [18] and Wei et al. [19] investigated the score and accuracy functions of q-ROFS, then, the comparison method of q-ROFNs was put forward.

**Definition 3 [18,19].** *Let a* = (*μ*, *v*) *be a q-ROFN, the score and accuracy functions of a are respectively given by:*

$$s(a) = (1 + \mu^q - v^q)/2;\tag{7}$$

$$h(a) = \mu^q + \upsilon^q. \tag{8}$$

**Definition 4 [19].** *Let a*<sup>1</sup> = (*μ*1, *v*1) *and a*<sup>2</sup> = (*μ*2, *v*2) *be two q-ROFNs, then:*


*a*. *h*(*a*1) < *h*(*a*2)*, then a*<sup>1</sup> < *a*2*; b*. *h*(*a*1) = *h*(*a*2)*, then a*<sup>1</sup> = *a*2.

Later, Du [72] developed the Minkowski distance measure of q-ROFNs.

**Definition 5 [72].** *Let a*<sup>1</sup> = (*μ*1, *v*1) *and a*<sup>2</sup> = (*μ*2, *v*2) *be two q-ROFNs, then the Minkowski distance between them is defined by:*

$$d(a\_1, a\_2) = \left(\frac{1}{2}|\mu\_1 - \mu\_2|^p + \frac{1}{2}|\upsilon\_1 - \upsilon\_2|^p\right)^{1/p}.\tag{9}$$

**Example 2.** *Suppose that a*<sup>1</sup> = (0.6500, 0.8298) *and a*<sup>2</sup> = (0.5000, 0.7500) *are two q-ROFNs, q* = 3*; according to Definition 3, we have s*(*a*1) = *s*(*a*2) = 0.3516*, h*(*a*1) = 0.8460*, and h*(*a*2) = 0.5469*, then a*<sup>1</sup> > *a*2*. In addition, the Minkowski distance between them can be computed as d*(*a*1, *a*2) = 0.1248.

#### *3.2. The q-ROFPWA Operator*

Considering the relationship between the aggregated values, Yager [26] proposed the PA operator to fuse the information.

**Definition 6 [26].** *Let ai*(*i* = 1, 2, . . . , *n*) *be a collection of evaluation values, the PA operator is a mapping* <sup>Ω</sup>*<sup>n</sup>* → <sup>Ω</sup> *as:*

$$PA(a\_1, a\_2, \dots, a\_n) = \sum\_{i=1}^n \frac{(1 + T(a\_i))a\_i}{\sum\_{j=1}^n (1 + T(a\_j))}.\tag{10}$$

*where T*(*ai*) = ∑*<sup>n</sup> <sup>j</sup>*=1,*j*=*<sup>i</sup> Sup*- *ai*, *aj and Sup*- *ai*, *aj is the support degree for ai from aj that satisfies the conditions as follows: (1) Sup*- *ai*, *aj* ∈ [0, 1]; *(2) Sup*- *ai*, *aj* = *Sup*- *aj*, *ai* ; *(3) If ai* − *aj* > |*as* − *at*|*, then Sup*- *ai*, *aj* ≤ *Sup*(*as*, *at*).

The PA operator can reflect the relationship between the aggregated values during the information fusion; however, it can only aggregate a crisp number. Therefore, Liu et al. [27] extended the PA operator into the q-ROF environment to propose the q-ROFPWA operator.

**Definition 7 [27].** *Let ai* = (*μi*, *vi*)(*i* = 1, 2, . . . , *n*) *be a collection of q-ROFNs; the q-ROFPWA operator is a mapping* <sup>Ω</sup>*<sup>n</sup>* → <sup>Ω</sup> *as:*

$$a\_l - \text{ROFPWM}(a\_1, a\_2, \dots, a\_n) = \overset{n}{\underset{i=1}{\overset{n}{\sum}}} \frac{w\_i (1 + T(a\_i)) a\_i}{\sum\_{j=1}^n \left(w\_j \left(1 + T\left(a\_j\right)\right)\right)}.\tag{11}$$

*where w* = (*w*1, *w*2,..., *wn*) *<sup>T</sup> is the weight vector of the q-ROFNs ai, <sup>T</sup>*(*ai*) = <sup>∑</sup>*<sup>n</sup> j*=1,*j*=*i* - *wjSup*- *ai*, *aj , and Sup*- *ai*, *aj* = 1 − *d* - *ai*, *aj is the support degree for ai from aj, in which d* - *ai*, *aj is the Minkowski distance between ai and aj in this study.*

Combined with the operations of q-ROFNs, we can obtain the following result.

**Theorem 1 [27].** *Let ai*(*i* = 1, 2, . . . , *n*) *be a collection of q-ROFNs; their aggregated value by using the q-ROFPWA operator is also a q-ROFN, and:*

$$q - ROFPWA(a\_1, a\_2, \dots, a\_n) = \left( \sqrt[n]{1 - \prod\_{i=1}^n \left(1 - \mu\_i^q\right)^{w\_i(1 + T(a\_i)) / \sum\_{j=1}^n \left(w\_j(1 + T(a\_j))\right)}}, \prod\_{i=1}^n \left(w\_i\right)^{w\_i(1 + T(a\_i)) / \sum\_{j=1}^n \left(w\_j(1 + T(a\_j))\right)} \right). \tag{12}$$

#### **4. Green Supplier Selection Method under q-ROF Environment**

In this section, we defined the q-ROF consensus measures on three levels, namely, criteria, alternative, and evaluation matrix levels to construct the consensus model. Then, the q-ROFPWA operator was investigated to fuse the q-ROF evaluation information. Finally, combined with the comprehensive weighting method and q-ROF-TODIM method, a novel green supplier selection

approach under q-ROF environment was developed. The flowchart of the proposed approach is presented in Figure 2.

**Figure 2.** The flowchart of the proposed approach. Q-ROF: q-rung orthopair fuzzy. Q-ROFPWA: q-rung orthopair fuzzy power weighted average. Q-ROF-TODIM: q-rung orthopair fuzzy TOmada de Decisao Interativa e Multicritevio

#### *4.1. Obtain the Normalized Evaluation Matrices of Decision Makers*

For a green supplier selection problem, suppose that a group of decision makers *Dk*(*k* = 1, 2, . . . , *l*) is assembled to evaluate the green suppliers for an enterprise, in which decision makers may come from different backgrounds of GSCM. Then, the normalized q-ROF evaluation matrices of decision makers can be obtained by the steps as follows:

**Step 1.1:** After the primary evaluation of the green supplier selection problem, decision makers can identify the potential green supplier *Ai*(*i* = 1, 2, . . . , *m*) and a collection of criteria *Cj*(*j* = 1, 2, . . . , *n*).

**Step 1.2:** Combined with the q-ROFS, the evaluation information of green suppliers can be expressed by q-ROF evaluation matrix *F<sup>k</sup>* = *ak ij m*×*n* , where *a<sup>k</sup> ij* = *μk ij*, *vk ij* indicates the q-ROF evaluation information of green supplier *Ai* concerning criteria *Cj* given by decision maker *Dk*. Moreover, decision makers also evaluate the weights of criteria using q-ROFNs; subsequently, the q-ROF evaluation matrix *W<sup>k</sup>* = *ak j* <sup>1</sup>×*<sup>n</sup>* was obtained, where *<sup>a</sup><sup>k</sup> <sup>j</sup>* = *μk <sup>j</sup>* , *<sup>v</sup><sup>k</sup> j* represents the importance degree of criteria *Cj* given by decision maker *Dk*.

**Step 1.3:** Generally, the criteria of green supplier selection can be divided into two types, namely, cost type and benefit type; thus, we should transform the information with respect to cost type criteria into the information with respect to benefit type criteria to determine the normalized q-ROF evaluation matrix *Q<sup>k</sup>* = *ak ij <sup>m</sup>*×*<sup>n</sup>* as:

$$a\_{ij}^k = \left(\mu\_{ij}^k, \upsilon\_{ij}^k\right) = \begin{cases} \quad \hat{a}\_{ij}^k & \text{if } \mathbb{C}\_j \text{ is the benefit type criteria;}\\ \quad \left(\hat{a}\_{ij}^k\right)^c & \text{if } \mathbb{C}\_j \text{ is the cost type criteria.} \end{cases} \tag{13}$$

#### *4.2. Consensus-Reaching Process*

Most research focused on the consensus model with preference relations that were obtained by pairwise comparison; Wu and Xu [55] proposed an iteration-based consensus model to solve the MCGDM problems based on a hesitant fuzzy linguistic set. Motivated by the literature, we develop the similarity matrix between different q-ROF evaluation matrices.

**Definition 8.** *Suppose that decision maker Dk*(*k* = 1, 2, . . . , *l*)*evaluated the alternative Ai*(*i* = 1, 2, . . . , *m*) *concerning the criteria Cj*(*j* = 1, 2, . . . , *n*) *using q-ROFNs. For each pair of decision makers,* - *Dk*, *Dp* (*<sup>k</sup>* <sup>=</sup> 1, 2, . . . , *<sup>l</sup>* <sup>−</sup> 1; *<sup>p</sup>* <sup>=</sup> *<sup>k</sup>* <sup>+</sup> 1, *<sup>k</sup>* <sup>+</sup> 2, . . . , *<sup>l</sup>*)*, the similarity matrix SMkp between the q-ROF evaluation matrices Qk* = *ak ij <sup>m</sup>*×*<sup>n</sup> and Qp* <sup>=</sup> *a p ij <sup>m</sup>*×*<sup>n</sup> is defined by:*

$$\mathcal{SM}^{kp} = \left( s m\_{i\bar{j}}^{kp} \right)\_{m \times n} = \left( 1 - d \left( a\_{i\bar{j}}^k, a\_{i\bar{j}}^p \right) \right)\_{m \times n} \tag{14}$$

*where d ak ij*, *a p ij is the Minkowski distance between the q-ROF evaluation information a<sup>k</sup> ij and a p ij. Furthermore, the consensus matrix CM is determined as:*

$$\mathbf{CM} = \left( c m\_{\mathbf{i}\bar{\jmath}} \right)\_{m \times n} = \left( \psi \left( s m\_{\mathbf{i}\bar{\jmath}}^{kp} \right) \right)\_{m \times n'} \tag{15}$$

*where ψ is the arithmetic average operator.*

The three consensus measures on criteria, alternative, and evaluation matrix levels could then bedefined according to the consensus matrix *CM*, which will be used to complete the consensus-reaching process.

**Definition 9.** *Criteria level: the consensus measure ccij for alternative Ai with respect to criteria Cj can be represented by the element of consensus matrix CM as:*

$$
\mathfrak{cc}\_{i\bar{j}} = \mathfrak{c}m\_{i\bar{j}}.\tag{16}
$$

*Alternative level: the consensus measure cai on alternative Ai can be obtained by:*

$$ca\_i = \frac{\sum\_{j=1}^{n} cc\_{ij}}{n}.\tag{17}$$

*Evaluation matrix level: the consensus measure ce on the evaluation matrix, i.e., the global consensus measure, can be defined by:*

$$cc = \min\_{i} \{ca\_{i}\}.\tag{18}$$

Once the q-ROF consensus measures on three levels in Definition 9 were computed, we could check whether the consensus was achieved by comparing the consensus measure *ce* with the predefined ideal consensus threshold *ε* ∈ (0, 1]. If *ce* ≥ *ε*, the consensus was reached; thus, the normalized q-ROF evaluation matrix *Q<sup>k</sup>* was the acceptable consensus evaluation matrix. Otherwise, several identification and direction rules could be obtained according to the aforementioned three consensus measures; identification rules were utilized to determine the non-consensus evaluation information set that contributed less to reach a high consensus level for each iteration round, and direction rules could guide decision makers to revise the non-consensus evaluation information in this round. An iteration-based consensus model under q-ROF environment was constructed to reach consensus as follows.

**Input:** The original individual evaluation matrix *Qk*, the ideal consensus threshold *ε*, and the maximum permission iterative number of times *r*max.

**Output:** The revised individual q-ROF evaluation matrix *Qk* and the global consensus measure *ce*. **Step 2.1:** Let the initial iterative number be *r* = 1, and the individual evaluation matrix in the first round be *Qk* **<sup>1</sup>** = *ak ij*,1 *<sup>m</sup>*×*<sup>n</sup>* <sup>=</sup> *ak ij* .

*m*×*n* **Step 2.2:** Calculate the similarity matrix *SMkp*(*<sup>k</sup>* <sup>=</sup> 1, 2, . . . , *<sup>l</sup>* <sup>−</sup> 1; *<sup>p</sup>* <sup>=</sup> *<sup>k</sup>* <sup>+</sup> 1, *<sup>k</sup>* <sup>+</sup> 2, . . . , *<sup>l</sup>*) and aggregate them to obtain the consensus matrix *CM*; thus, the consensus measures *ccij*, *cai*, and *ce* in round *r* are computed. If *ce* ≥ *ε* or *r* > *r*max, then proceed to Step 1.5; otherwise, proceed to the next step.

**Step 2.3:** Obtain the identification rules as in the following:

(1) Identification rule 1. The non-consensus alternative set *IRA* = {*Ai*|*cai* < *ε*, *i* = 1, 2, . . . , *m*} identifies the rows of the evaluation matrices that should be revised.

(2) Identification rule 2. The non-consensus criteria set *IRCi* = *Cj Ai* ∈ *IRA* ∧ *ccij* < *ε*, *j* = 1, 2, . . . , *n* identifies the columns that should be revised for the rows distinguished in the non-consensus alternative set *IRA*.

(3) Identification rule 3. The non-consensus decision maker set *IRDij* = *Dp Ai* ∈ *IRA* ∧ *Cj* ∈ *IRCi* ∧ *d* (*p*) *ij* <sup>=</sup> max*<sup>k</sup> d* (*k*) *ij* identifies the decision makers that should revise the evaluation information at the position (*i*, *j*) in evaluation matrices, where *d* (*p*) *ij* is the distance between the similarity measures of *Dp* and other decision makers, i.e., *d* (*p*) *ij* <sup>=</sup> <sup>∑</sup>*<sup>l</sup> k*=1,*k*=*p* <sup>1</sup> <sup>−</sup> *smkp ij* .

Subsequently, combined with the identification rules 1~3, the non-consensus evaluation information set *IR* that should be revised in round *r* can be determined as:

$$IR = \left\{ \left( p\_\prime \left( i\_\prime j \right) \right) \middle| D\_p \in IR \\ \left. D\_{\hat{l}j} \wedge A\_{\hat{l}} \in IR \, A \wedge \mathbb{C}\_{\hat{j}} \in IR \mathbb{C}\_{\hat{l}} \right\}. \tag{19}$$

**Step 2.4:** Aggregate the individual evaluation matrix *Qk <sup>r</sup>* using the q-ROFAA operator that is reduced by the q-ROFWA operator [18], then, the collective evaluation matrix *Qr* = - *aij*,*<sup>r</sup> <sup>m</sup>*×*<sup>n</sup>* can be obtained as:

$$a\_{\overline{i}|J} = q - \text{ROFAA}\left(a\_{\overline{i}j,r'}^1 a\_{\overline{i}|J}^2, \dots, a\_{\overline{i}|J}^l\right) = \left(\sqrt[q]{1 - \prod\_{k=1}^l \left(1 - \left(\mu\_{\overline{i}|J}^k\right)^q\right)^{1/l}}, \prod\_{k=1}^l \left(v\_{\overline{i}|J}^k\right)^{1/l}\right). \tag{20}$$

*Symmetry* **2018**, *10*, 687

Both the collective evaluation information *aij*,*<sup>r</sup>* and the non-consensus evaluation information set *IR* show that the direction rules, which suggest decision makers how to change their non-consensus evaluation information as in the following:

(1) If *aij*,*<sup>r</sup>* > *a<sup>k</sup> ij*,*r*, then the decision maker *Dk* should decrease the evaluation on alternative *Ai* concerning criteria *Cj* when *Cj* is the benefit type criteria, and the decision maker *Dk* should increase the evaluation on alternative *Ai* concerning criteria *Cj* when *Cj* is the cost type criteria.

(2) If *aij*,*<sup>r</sup>* < *a<sup>k</sup> ij*,*r*, the decision maker *Dk* should increase the evaluation on alternative *Ai* concerning criteria *Cj* when *Cj* is the benefit type criteria, and the decision maker *Dk* should decrease the evaluation on alternative *Ai* concerning criteria *Cj* when *Cj* is the cost type criteria.

Then, the revised individual q-ROF evaluation matrix *Qk <sup>r</sup>***+1** can be obtained. Set *r* = *r* + 1 and proceed to **Step 1.2**.

$$\text{Step 2.5: Let } \overline{\mathbb{Q}}^k = \mathbb{Q}\_r^k = \left(\overline{\mathbb{a}}\_{ij}^k\right)\_{m \times n} = \left(\overline{\mathbb{a}}\_{ij}^k, \overline{\mathbb{v}}\_{ij}^k\right)\_{m \times n}. \text{Output } \overline{\mathbb{Q}}^k \text{ and } c\boldsymbol{\epsilon} \text{ in this round.}$$

#### *4.3. Aggregation of Individual Acceptable Consensus Evaluation Matrices*

According to the individual acceptable consensus evaluation matrices, we can use the q-ROFPWA operator to fuse them; then, the weights of decision makers can be determined by both the subjective weights and support degrees between individual evaluation information. Thus, the collective evaluation matrix *Q* = - *aij <sup>m</sup>*×*<sup>n</sup>* is obtained by the steps as below.

**Step 3.1:** Compute the support degree:

$$\operatorname{Supp}\left(\overline{a}\_{ij}^k, \overline{a}\_{ij}^p\right) = 1 - d\left(\overline{a}\_{ij}^k, \overline{a}\_{ij}^p\right), k, p = 1, 2, \dots, l,\tag{21}$$

where *d ak ij*, *a p ij* is the Minkowski distance between the evaluation information *a<sup>k</sup> ij* and *a p ij*.

**Step 3.2:** Combined with the subjective weight vector of decision makers *w* = (*w*1, *w*2,..., *wl*) *T* that is provided by the enterprise, the weighted support degree of *a<sup>k</sup> ij* can be calculated as:

$$T\left(\overline{a}\_{ij}^k\right) = \sum\_{p=1, p\neq k}^l w\_p \text{Sup}\left(\overline{a}\_{ij}^k, \overline{a}\_{ij}^p\right),\tag{22}$$

Then, the weights associated with *a<sup>k</sup> ij* can be determined as:

$$\mathcal{I}\_{ij}^{k} = \frac{w\_k \left(1 + T\left(\overline{\pi}\_{ij}^k\right)\right)}{\sum\_{k=1}^l \left(w\_k \left(1 + T\left(\overline{\pi}\_{ij}^k\right)\right)\right)}, \mathcal{I}\_{ij}^k \ge 0, \sum\_{k=1}^l \mathcal{I}\_{ij}^k = 1. \tag{23}$$

**Step 3.3:** Use the q-ROFPWA operator to fuse the evaluation matrix *<sup>Q</sup><sup>k</sup>* to obtain the collective evaluation matrix *Q* as:

$$a\_{ij} = q - \text{ROFPWA}\left(\overline{a}\_{ij}^1, \overline{a}\_{ij}^2, \dots, \overline{a}\_{ij}^l\right) = \left(\sqrt[q]{1 - \prod\_{k=1}^l \left(1 - \left(\overline{\mu}\_{ij}^k\right)^q\right)^{\frac{p^k}{q\_{ij}}} \prod\_{i=1}^n \left(\overline{v}\_{ij}^k\right)^{\frac{p^k}{q\_{ij}}}}\right) = \left(\mu\_{ij}, v\_{ij}\right). \tag{24}$$

#### *4.4. Determine the Weights of Criteria*

In practice, it is sometimes unreasonable to determine the criteria weights only considering the views of decision makers. To investigate both the subjective and objective factors, we constructed a comprehensive weighting method that consists of a subjective weighting method and a deviation maximization model to calculate the weights of criteria as follows.

**Step 4.1:** Combined with the evaluation matrix *Wk* and the similar steps in Sections 4.3 and 4.4, we can obtain the collective evaluation matrix *W* = - *aj* <sup>1</sup>×*n*. The larger the score value of *aj*, which

means the criteria *Cj* is more important, the higher the weight of criteria *Cj* and vice versa. Then, the subjective weight vector of criteria *λ<sup>S</sup>* = - *λS* <sup>1</sup> , *<sup>λ</sup><sup>S</sup>* <sup>2</sup> ,..., *<sup>λ</sup><sup>S</sup> n <sup>T</sup>* can be determined as:

$$
\lambda\_j^S = \frac{s\left(a\_j\right)}{\sum\_{j=1}^n s\left(a\_j\right)}.\tag{25}
$$

where *s* - *aj* is the score value of *aj*.

**Step 4.2:** Let ∑*<sup>m</sup> <sup>h</sup>*=1,*h*=*<sup>i</sup> <sup>d</sup> aij*, *ahj λ<sup>O</sup> <sup>j</sup>* be the deviation between the collective evaluation information on green supplier *Ai* and other green suppliers concerning *Cj*, where *d aij*, *ahj* is the Minkowski distance between *aij* and *ahj*; then, the total deviation is obtained as ∑*<sup>n</sup> <sup>j</sup>*=<sup>1</sup> <sup>∑</sup>*<sup>m</sup> <sup>i</sup>*=<sup>1</sup> <sup>∑</sup>*<sup>m</sup> <sup>h</sup>*=1,*h*=*<sup>i</sup> <sup>d</sup> aij*, *ahj λ<sup>O</sup> j* . According to the information theory, if all green suppliers have similar evaluation information concerning one of criteria, a small weight value should be assigned to the criteria as it contributes less to differentiate green suppliers [73]. Subsequently, a deviation maximization model can be developed as:

$$\begin{aligned} \max & \sum\_{j=1}^{n} \sum\_{i=1}^{m} \sum\_{h=1, h \neq i}^{m} d \left( a\_{ij}, a\_{hj} \right) \lambda\_j^O \\ & \text{s.t.} \sum\_{j=1}^{n} \left( \lambda\_j^O \right)^2 = 1, \lambda\_j^O \ge 0. \end{aligned} \tag{26}$$

Solve the model according to the Lagrange function:

$$L\left(\boldsymbol{\lambda}^{O},\boldsymbol{\wp}\right) = \sum\_{j=1}^{n} \sum\_{i=1}^{m} \sum\_{h=1,h\neq i}^{m} d\left(a\_{ij}, a\_{hj}\right) \boldsymbol{\lambda}\_{j}^{O} + \frac{\wp}{2} \left(\sum\_{j=1}^{n} \left(\boldsymbol{\lambda}\_{j}^{O}\right)^{2} - 1\right). \tag{27}$$

where ℘ is the Lagrange multiplier. Differentiate Equation (27) concerning *λ<sup>O</sup> <sup>j</sup>* and ℘, and let these partial derivatives be equal to zero:

$$\begin{cases} \begin{array}{c} \frac{\partial L\left(\lambda^{\bullet}, \wp\right)}{\partial \lambda^{\bullet}\_{j}} = \sum\_{i=1}^{m} \sum\_{h=1, h \neq i}^{m} d\left(a\_{ij}, a\_{hj}\right) + \wp \lambda^{\bullet}\_{j} = 0;\\ \frac{\partial L\left(\lambda^{\bullet}, \wp\right)}{\partial \wp} = \sum\_{j=1}^{n} \left(\lambda^{\bullet}\_{j}\right)^{2} - 1 = 0. \end{array} \end{cases} \tag{28}$$

By solving Equation (28), the normalized weights of criteria, i.e., objective weight vector of criteria *λ<sup>O</sup>* = - *λ<sup>O</sup>* <sup>1</sup> , *<sup>λ</sup><sup>O</sup>* <sup>2</sup> ,..., *<sup>λ</sup><sup>O</sup> n <sup>T</sup>* can be obtained:

$$\Lambda\_j^O = \frac{\sum\_{i=1}^m \sum\_{h=1, h \neq i}^m d\left(a\_{ij}, a\_{hj}\right)}{\sum\_{j=1}^m \sum\_{i=1}^m \sum\_{h=1, h \neq i}^m d\left(a\_{ij}, a\_{hj}\right)}.\tag{29}$$

**Step 4.3:** Determine the comprehensive weight vector of criteria *λ* = (*λ*1, *λ*2,..., *λn*) *<sup>T</sup>* as:

$$
\lambda\_j = \varrho \lambda\_j^S + (1 - \varrho) \lambda\_j^O. \tag{30}
$$

where *ϕ* ∈ [0, 1] is the importance coefficient of subjective weights, and 1 − *ϕ* is the importance coefficient of objective weights.

#### *4.5. Rank the Green Suppliers Using the TODIM Method under q-ROF Environment*

Based on the collective q-ROF evaluation matrix *Q* and weight vector of criteria *λ*, we construct the q-ROF-TODIM method that can deal with the multiple criteria decision making problems with q-ROFS to obtain the ranking indices of green suppliers and determine the best green supplier.

**Step 5.1:** Compute the relative weight *λjr* of criteria *Cj* with respect to the reference criteria *Cr* as:

$$
\lambda\_{\rm jr} = \lambda\_{\rm j} / \lambda\_{\rm r\_{\rm r}} \tag{31}
$$

where *<sup>λ</sup><sup>j</sup>* is the weight of criteria *Cj* and *<sup>λ</sup><sup>r</sup>* <sup>=</sup> max*<sup>j</sup> λj* is the weight of reference criteria *Cr*.

**Step 5.2:** Calculate the dominance degree of green supplier *Ai* over each green supplier *Ah*(*h* = 1, 2, . . . , *m*) by the following equation:

$$\delta(A\_{\rm i}, A\_{\rm h}) = \sum\_{j=1}^{n} \phi\_{\rm j}(A\_{\rm i}, A\_{\rm h})\_{\prime} \tag{32}$$

where:

$$\phi\_{\vec{f}}(A\_{i\cdot}, A\_{h}) = \begin{cases} \sqrt{\lambda\_{jr} d \left(a\_{i\vec{j}\cdot} a\_{h\vec{j}}\right) / \sum\_{j=1}^{n} \lambda\_{jr}} & \text{if } a\_{i\vec{j}\cdot} > a\_{h\vec{j}\cdot} \\ 0 & \text{if } a\_{i\vec{j}} = a\_{h\vec{j}}; \\\ -\frac{1}{\mathfrak{F}} \sqrt{\left(\sum\_{j=1}^{n} \lambda\_{jr}\right) d \left(a\_{i\vec{j}\cdot} a\_{h\vec{j}}\right) / \lambda\_{jr}} & \text{if } a\_{i\vec{j}} < a\_{h\vec{j}}. \end{cases} \tag{33}$$

The parameter *θ* > 0 indicates the attenuation factor of the losses, and *d aij*, *ahj* is the Minkowski distance between *aij* and *ahj*.

**Step 5.3:** Compute the global value of green supplier *Ai* by:

$$\Phi(A\_{\bar{i}}) = \frac{\sum\_{h=1}^{m} \delta(A\_{\bar{i}}, A\_{h}) - \min\_{\bar{i}} \{ \sum\_{h=1}^{m} \delta(A\_{\bar{i}}, A\_{h}) \}}{\max\_{\bar{i}} \{ \sum\_{h=1}^{m} \delta(A\_{\bar{i}}, A\_{h}) \} - \min\_{\bar{i}} \{ \sum\_{h=1}^{m} \delta(A\_{\bar{i}}, A\_{h}) \}}. \tag{34}$$

**Step 5.4:** Determine the ranking of potential green suppliers based on their global values; the larger the global value Φ(*Ai*), the higher the ranking of green supplier *Ai*.

#### **5. Numerical Example**

In this section, a numerical example in the literature [16] was applied to show the feasibility and advantages of the proposed approach. An electric automobile enterprise plans to purchase a key component of a manufacturing procedure from the green suppliers market; the ranking of green suppliers can be determined by the following steps in the next subsection.

#### *5.1. Implementation*

**Step 1:** Obtain the normalized evaluation matrices of decision makers.

**Step 1.1:** After a preliminary evaluation, four potential green suppliers *Ai*(*i* = 1, 2, 3, 4) are determined by a group of decision makers *Dk*(*k* = 1, 2, 3). Decision makers evaluate the four green suppliers concerning six criteria *Cj*(*j* = 1, 2, 3, 4, 5, 6), namely, environmental costs (*C*1), remanufacturing activity (*C*2), energy assumption (*C*3), reverse logistics program (*C*4), hazardous waste management (*C*5), and environmental certification (*C*6), where *C*<sup>1</sup> and *C*<sup>3</sup> are the cost type criteria, and the others are the benefit type criteria.

**Step 1.2:** According to the relationships between the linguistic terms and interval-valued Pythagorean fuzzy numbers [74], we can construct the transformation between the linguistic terms and the corresponding q-ROFNs (*q* = 3) as shown in Table 1. Then, decision makers use the linguistic terms to assess the green suppliers as shown in Table 2; thus, the q-ROF evaluation matrix *F<sup>k</sup>* = *ak ij* 4×6 is obtained. It is noteworthy that we adopt the subjective weights of criteria obtained in the literature [16], i.e., *λ<sup>S</sup>* = (0.180, 0.090, 0.130, 0.130, 0.310, 0.160) *T*.


**Table 1.** Linguistic terms and the corresponding q-rung orthopair fuzzy numbers (q-ROFNs).


**Table 2.** Evaluation information of decision makers.

**Step 1.3:** After the normalization step according to the different types of criteria, the normalized q-ROF evaluation matrix *Qk* = *ak ij* 4×6 can be obtained by using Equation (13).

**Step 2:** Consensus-reaching process (*ε* = 0.85,*r*max = 5).

**Step 2.1:** Let the initial iterative number be *r* = 1, and *Q<sup>k</sup>* <sup>1</sup> = *ak ij*,1 <sup>4</sup>×<sup>6</sup> <sup>=</sup> *ak ij* 4×6

**Step 2.2:** Calculate the similarity matrix *SMkp* between the q-ROF evaluation matrices *Qk* <sup>1</sup> and *Qp* <sup>1</sup> as

.


Then, aggregate them to obtain the consensus matrix *CM* in round one:


Thus, we can calculate the consensus measures *ccij*, *cai*, and *ce* based on Equations (16)~(18); the global consensus measure in round one *ce* = 0.7889. It can be seen that *ce* < *ε* after which we can proceed to the next step.

**Step 2.3:** Obtain the identification rules as in the following:

(1) Identification rule 1. The non-consensus alternative set: *IRA* = {*Ai*|*cai* < 0.85} = {*A*1, *A*3, *A*4}.

(2) Identification rule 2. The non-consensus criteria set:

$$IR\mathbb{C}\_1 = \left\{\mathbb{C}\_j \middle| A\_1 \in IRA \land cc\_{1j} < 0.85\right\} = \left\{\mathbb{C}\_1, \mathbb{C}\_2\right\};$$

$$IR\mathbb{C}\_3 = \left\{\mathbb{C}\_j \middle| A\_3 \in IRA \land cc\_{3j} < 0.85\right\} = \left\{\mathbb{C}\_5\right\};$$

$$IR\mathbb{C}\_4 = \left\{\mathbb{C}\_j \middle| A\_4 \in IRA \land cc\_{4j} < 0.85\right\} = \left\{\mathbb{C}\_2, \mathbb{C}\_6\right\}.$$

(3) Identification rule 3. Combined with the distances between the similarity measures of decision maker *Dp* and the other decision makers at the positions {(1, 1),(1, 2),(3, 5),(4, 2),(4, 6)} in evaluation matrix *Q<sup>k</sup>* 1, we can obtain the non-consensus decision maker set:

$$IRD\_{11} = \left\{D\_p \middle| A\_1 \in IRA \land \mathbb{C}\_1 \in IRC\_1 \land d\_{11}^{(p)} = \max\_k \left\{d\_{11}^{(k)}\right\} \right\} = \{D\_3\};$$

$$IRD\_{12} = \left\{D\_p \middle| A\_1 \in IRA \land \mathbb{C}\_2 \in IRC\_1 \land d\_{12}^{(p)} = \max\_k \left\{d\_{12}^{(k)}\right\} \right\} = \{D\_2\};$$

$$IRD\_{35} = \left\{D\_p \middle| A\_3 \in IRA \land \mathbb{C}\_5 \in IRC\_3 \land d\_{35}^{(p)} = \max\_k \left\{d\_{35}^{(k)}\right\} \right\} = \{D\_1\};$$

$$IRD\_{42} = \left\{D\_p \middle| A\_4 \in IRA \land \mathbb{C}\_2 \in IRC\_4 \land d\_{42}^{(p)} = \max\_k \left\{d\_{42}^{(k)}\right\} \right\} = \{D\_1\};$$

$$IRD\_{46} = \left\{D\_p \middle| A\_4 \in IRA \land \mathbb{C}\_6 \in IRC\_4 \land d\_{46}^{(p)} = \max\_k \left\{d\_{46}^{(k)}\right\} \right\} = \{D\_3\}.$$

Finally, based on the identification rules 1~3, the non-consensus evaluation information set *IR* that should be revised in round one can be determined as:

$$IR = \left\{ (p\_r(i\_r)) \middle| D\_{\overline{r}} \in IR \\ D\_{\overline{\eta}} \wedge A\_l \in IR \\ A \wedge C\_{\overline{\eta}} \in IR \\ C\_i \right\} \\ = \left\{ (\mathfrak{J}, (1, 1)), (2, (1, 2)), (1, (3, 5)), (1, (4, 2)), (3, (4, 6)) \right\}.$$

**Step 2.4:** Aggregate the individual evaluation matrix *Qk* <sup>1</sup> in round one using the q-ROFAA operator to obtain collective evaluation matrix *Q*<sup>1</sup> = - *aij*,1 4×6 ; then, the direction rules can be put forward to revise the non-consensus evaluation information in set *IR* as shown in Table 3. Set *r* = 2 and proceed to Step 1.2.


**Table 3.** Direction rules in round one.

Then, combined with the similar **Steps 2.2~2.4**, we can obtain the global consensus measure in round four *ce* = 0.8556 > *ε*, which means that a high consensus level between decision makers has been achieved; the individual acceptable consensus q-ROF evaluation matrix *<sup>Q</sup><sup>k</sup>* are determined as shown in Table 4.


**Table 4.** Individual acceptable consensus q-rung orthopair fuzzy (q-ROF) evaluation matrices.

**Step 3:** Aggregation of individual acceptable consensus evaluation matrices.

**Steps 3.1~3.2:** Suppose that the subjective weight values of decision makers are equal, i.e., *w* = (1/3, 1/3, 1/3) *<sup>T</sup>*; we can use Equations (21)~(23) to calculate the weighted support degree of *ak ij* as:

$$\begin{aligned} T^1 &= \begin{pmatrix} 0.5333 & 0.5333 & 0.5667 & 0.5667 & 0.6333 & 0.6333 \\ 0.5667 & 0.5667 & 0.6333 & 0.6000 & 0.6333 & 0.5667 \\ 0.6333 & 0.6000 & 0.6333 & 0.6333 & 0.2000 & 0.6333 \\ 0.5667 & 0.4333 & 0.6000 & 0.6333 & 0.6333 & 0.5667 \end{pmatrix}; \\ T^2 &= \begin{pmatrix} 0.5667 & 0.5000 & 0.5667 & 0.5667 & 0.6333 & 0.6000 \\ 0.6000 & 0.4667 & 0.6000 & 0.6333 & 0.6333 & 0.6333 \\ 0.6333 & 0.5667 & 0.6333 & 0.6333 & 0.4333 & 0.6333 \\ 0.6000 & 0.5333 & 0.6000 & 0.6333 & 0.6333 & 0.5667 \end{pmatrix}; \\ T^3 &= \begin{pmatrix} 0.5000 & 0.5667 & 0.6000 & 0.6000 & 0.6000 & 0.6333 \\ 0.5667 & 0.5667 & 0.6333 & 0.6333 & 0.6000 & 0.5667 \\ 0.6000 & 0.5667 & 0.6000 & 0.6000 & 0.4333 & 0.6000 \\ 0.5667 & 0.5000 & 0.5333 & 0.6000 & 0.6000 & 0.4667 \end{pmatrix}. \end{aligned}$$

Then, the weights associated with *a<sup>k</sup> ij* can be determined as:

$$\begin{aligned} \mathfrak{E}^1 &= \begin{pmatrix} 0.3333 & 0.3333 & 0.3310 & 0.3310 & 0.3356 & 0.3356 \\ 0.3310 & 0.3406 & 0.3356 & 0.3288 & 0.3356 & 0.3310 \\ 0.3356 & 0.3380 & 0.3356 & 0.3356 & 0.2951 & 0.3356 \\ 0.3310 & 0.3209 & 0.3380 & 0.3356 & 0.3356 & 0.3406 \end{pmatrix}; \\\\ \mathfrak{E}^2 &= \begin{pmatrix} 0.3406 & 0.3261 & 0.3310 & 0.3310 & 0.3356 & 0.3288 \\ 0.3380 & 0.3188 & 0.3288 & 0.3356 & 0.3356 & 0.3380 \\ 0.3356 & 0.3310 & 0.3356 & 0.3356 & 0.3308 & 0.3356 \\ 0.3380 & 0.3333 & 0.3380 & 0.3356 & 0.3356 & 0.3356 \end{pmatrix}; \\\\ \mathfrak{E}^3 &= \begin{pmatrix} 0.3261 & 0.3406 & 0.3380 & 0.3380 & 0.3288 & 0.3356 \\ 0.3310 & 0.3406 & 0.3356 & 0.3356 & 0.3288 & 0.3310 \\ 0.3288 & 0.3310 & 0.3288 & 0.3288 & 0.3308 & 0.3288 \\ 0.3310 & 0.3261 & 0.3299 & 0.3288 & 0.3288 & 0.3188 \end{pmatrix}. \end{aligned}$$

**Step 3.3:** Use the q-ROFPWA operator to fuse the evaluation matrix *<sup>Q</sup><sup>k</sup>* to obtain the collective evaluation matrix *Q* as shown in Table 5.



**Step 4:** Determine the weights of the criteria.

**Step 4.1:** Because the subjective weights of criteria were determined in the literature [16], we adopt the subjective weight vector of criteria as *λ<sup>S</sup>* = (0.180, 0.090, 0.130, 0.130, 0.310, 0.160) *T*.

**Step 4.2:** Based on the collective evaluation matrix *Q*, we can construct the programming model, i.e., Equation (26); then, the objective weight vector of criteria can be determined as *λ<sup>O</sup>* = (0.201, 0.160, 0.150, 0.182, 0.151, 0.156) *T*.

**Step 4.3:** Set the importance coefficient of subjective weights to *ϕ* = 0.5; we can obtain the comprehensive weights of criteria as *λ* = (0.191, 0.125, 0.140, 0.156, 0.230, 0.158) *T*.

**Step 5:** Rank the green suppliers using the TODIM method under a q-ROF environment (*θ* = 1).

**Step 5.1:** Utilize Equation (31) to compute the relative weight *λjr* of criteria *Cj* concerning the reference criteria *Cr* as:

$$
\lambda\_{1r} = 0.8304, \lambda\_{2r} = 0.5435, \lambda\_{3r} = 0.6087, \lambda\_{4r} = 0.6783, \lambda\_{5r} = 1.0000, \lambda\_{6r} = 0.6870.
$$

**Step 5.2:** Compute the dominance degree of green supplier *Ai* over each green supplier:

$$
\delta = \begin{pmatrix}
0 & -1.5220 & -4.0821 & -4.2886 \\
\end{pmatrix}.
$$

**Step 5.3:** Compute the global value of green supplier *Ai* by Equation (34):

$$
\Phi(A\_1) = 0.3028, \Phi(A\_2) = 0, \Phi(A\_3) = 0.9653, \Phi(A\_4) = 1.
$$

**Step 5.4:** Based on the global values of green suppliers, the ranking of potential green suppliers can be determined as *A*<sup>4</sup> > *A*<sup>3</sup> > *A*<sup>1</sup> > *A*2. The green supplier *A*<sup>4</sup> is the best choice for the electric automobile company.

#### *5.2. Comparison and Sensitivity Analysis*

To investigate the influence of the consensus-reaching process on the ranking result and further verify the effectiveness of the proposed approach, we compared the ranking result of the green suppliers in Section 5.1 with the results that were obtained by the proposed approach without the consensus-reaching process, the green supplier selection approach based on the intuitionistic fuzzy TOPSIS (IF-TOPSIS) method [16], and the green supplier selection approach based on the fuzzy TODIM method [75]. The ranking results of the three green supplier selection approaches are shown in Figure 3; the detailed computation procedures of the proposed approach without consensus-reaching process, IF-TOPSIS method, and fuzzy TODIM method are presented in Appendices A–C, respectively.

The inconsistent ranking results between the proposed approach and the proposed approach without consensus-reaching process, i.e., the different ranking orders of green suppliers *A*<sup>3</sup> and *A*4, can be explained by ignoring the consensus level of q-ROF evaluation information of decision makers. For instance, the linguistic term of decision maker *D*<sup>1</sup> for green supplier *A*<sup>3</sup> with respect to criteria *C*<sup>5</sup> was extremely low (EL); by contrast, the linguistic evaluation information of decision makers *D*<sup>2</sup> and *D*<sup>3</sup> for green supplier *A*<sup>3</sup> with respect to criteria *C*<sup>5</sup> were both extremely high (EH). Similarly,

the linguistic terms of decision makers *D*<sup>1</sup> and *D*<sup>2</sup> for green supplier *A*<sup>4</sup> concerning criteria *C*<sup>6</sup> were both high (H); however, the linguistic evaluation information of decision maker *D*<sup>3</sup> for green supplier *A*<sup>4</sup> concerning criteria *C*<sup>6</sup> was EL. The aforementioned non-consensus evaluation information led to a change in the ranking orders of green suppliers *A*<sup>3</sup> and *A*<sup>4</sup> without a consensus-reaching process. In the procedures of the proposed approach, an iteration-based consensus model under a q-ROF environment was utilized to revise this non-consensus evaluation information until an acceptable consensus level between decision makers was achieved. Thus, we can obtain the ranking of green suppliers that was accepted by decision makers or enterprise; furthermore, the possible extreme evaluation information of individual decision maker was also revised to avoid affecting the accuracy of ranking result.

**Figure 3.** Ranking results of different approaches.

A notable difference existed between the rankings of the proposed method and IF-TOPSIS method. With the exception of ignoring the consensus problem between decision makers, the inconsistent result was caused by several other reasons. First, the evaluation information was represented by the q-ROFS in the proposed method, which is a generalized form of IFS that is used in the IF-TOPSIS method. Different basic data of green suppliers will lead to different result by aggregation tools. Second, combined with the q-ROFPWA operator, the decision maker weights were obtained by the subjective weights and support degrees between the evaluation information in the proposed approach; nevertheless, the determination of decision maker weights was omitted in the IF-TOPSIS method. Third, instead of the TOPSIS method, we utilized the q-ROF-TODIM method to determine the ranking of green suppliers. The q-ROF-TODIM method can consider the bounded rationality behavior of decision makers, which cannot be achieved by the TOPSIS method; consequently, the ranking result of green suppliers may differ.

From Figure 3, we can see that the ranking orders of green suppliers *A*<sup>3</sup> and *A*<sup>4</sup> were different between the rankings obtained by the proposed approach and fuzzy TODIM method. The main reason for this result is that the consensus-reaching process was omitted in the fuzzy TODIM method; the non-consensus evaluation information of decision makers made green supplier *A*<sup>3</sup> rank first in the ranking results determined by the proposed approach without a consensus-reaching process, IF-TOPSIS method, and fuzzy TODIM method. Moreover, the fuzzy TODIM method utilizes the triangular fuzzy numbers to express the evaluation information of decision makers, in which the non-membership and indeterminacy membership levels were ignored. The weights of decision makers in fuzzy TODIM method were assumed to be equal, which was inconsistent with the actual situation.

Furthermore, a sensitivity analysis was implemented by changing the weights of criteria as shown in Table 6. The rankings under different situations of the proposed approach, IF-TOPSIS method, and fuzzy TODIM method are illustrated in Figures 4–6, respectively. Example 0 showed the weights of criteria that were determined by the proposed method, and Examples 1~7 showed the other possible weight values. From Table 6 and Figure 4, we can see that when the weight values of criteria *C*<sup>4</sup> and *C*<sup>5</sup> were relatively large, the best green supplier changed from *A*<sup>4</sup> to *A*3, which means that the criteria weights play a crucial role in determining the ranking of green suppliers. Therefore, we should select the appropriate weighting method in practice. The comprehensive weighting approach in the proposed method considered the subjective and objective factors to obtain the more accurate weights of criteria. Once decision makers were confident for the evaluation information of criteria weights, the coefficient *ϕ* could be assigned a large value; otherwise, the coefficient *ϕ* could be assigned a small value. On the other hand, in addition to Examples 5 and 6, the rankings remain the same as *A*<sup>4</sup> > *A*<sup>3</sup> > *A*<sup>1</sup> > *A*<sup>2</sup> under other situations; the proposed method is proven to be relatively insensitive to the weights of criteria.


**Table 6.** Different weights of criteria in the sensitivity analysis.

**Figure 4.** Ranking results of the proposed approach with different weights of criteria.

**Figure 5.** Ranking results of intuitionistic fuzzy (IF)-technique for order performance by similarity to ideal solution (TOPSIS) method with different weights of criteria.

**Figure 6.** Ranking results of fuzzy TOmada de Decisao Interativa e Multicritevio (TODIM) method with different weights of criteria.

The Spearman's rank correlation coefficient is a powerful tool for measuring the similarity between rankings obtained by MCGDM methods [76]. To investigate the robustness of different green supplier selection approaches, combined with the rankings in Figures 4–6, we can calculate the Spearman's rank correlation coefficients between the ranking of Example 0 and the rankings of other possible weights of criteria, respectively. Thus, the average of these Spearman's rank correlation coefficients can be utilized to measure the robustness of each green supplier selection approach, which are presented in Table 7. The larger the average of Spearman's rank correlation coefficients, which means that the smaller the rankings change with different criteria weights, the stronger the robustness of this green supplier selection approach and vice versa. From Table 7, we can see that the robustness levels of all three green supplier selection approaches were relatively high, and the robustness of the proposed method and IF-TOPSIS was is slightly stronger than that of fuzzy TODIM method.


**Table 7.** Average of Spearman's rank correlation coefficients of different approaches.

Based on the analysis above, the advantages of determining the best green supplier by using the proposed approach can be summarized as follows.


The proposed approach also presents several limitations. With respect to the complicated green supplier selection issues, in which the number of evaluation criteria is relatively large; the interactions or dependencies between the criteria will inevitably exist. These situations cannot be solved combined with the proposed green supplier selection approach. Furthermore, decision makers may have difficulty determining the accurate value of a membership degree or linguistic term in real life. The proposed approach cannot deal with the issue of allowing decision makers to provide several possible values of different membership degrees or linguistic terms, which will be the focus of future research.

#### **6. Conclusions**

To deal with the complexity of green supplier selection problems in practice, this paper proposed a novel approach for green supplier selection under q-ROF environment. The q-ROFNs were utilized to express the evaluation information of decision makers; the uncertainty and incompleteness of the evaluation information were effectively addressed. Combined with the consensus measures on three levels, a q-ROF consensus model was developed to revise the non-consensus evaluation information of decision makers to improve the accuracy of the ranking results. To aggregate the q-ROF evaluation information of decision makers, the q-ROFPWA operator that considers both subjective and objective factors of decision maker weights was applied. Furthermore, a comprehensive weighting method was constructed to determine the weights of criteria, which consisted of the subjective weighting method and a deviation maximization model. Finally, the TODIM method under an q-ROF environment was proposed to obtain a ranking of potential green suppliers. An example of a green supplier selection problem in an electric automobile company was used to demonstrate the feasibility of the proposed method; subsequently, the effectiveness of the proposed method was illustrated by the sensitivity analysis and comparative analysis. In the case of increasingly complex green supplier selection issues, the proposed approach can deal with several aspects effectively, such as providing a relaxed evaluation environment for decision makers, promoting a relatively high consensus level between decision makers, and determining the weights of decision makers comprehensively. Thus, this paper provides a more reasonable and effective approach for enterprises to choose green suppliers in practice.

In future research, we will introduce the Choquet integral or Bonferroni mean operator to aggregate the evaluation information, which takes into account the relationships between the criteria. Furthermore, we can extend the proposed method into the q-rung orthopair hesitant fuzzy environment, in which decision makers have difficulty in determining the accurate membership and non-membership degrees.

**Author Contributions:** Concept of this paper, R.W.; methodology, R.W.; writing—original draft preparation, R.W.; writing—review and editing, Y.L.; funding acquisition, R.W. and Y.L.

**Funding:** This study was supported by the National Natural Science Foundation of China (no. 71371156) and the Doctoral Innovation Fund Program of Southwest Jiaotong University (D-CX201727).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

The ranking of potential green suppliers can be obtained by the proposed approach without the consensus-reaching process as below.

**Step 1:** Obtain the normalized evaluation matrices of decision makers

Combined with the Steps 1.1~1.3 in Section 5.1, we can obtain the normalized q-ROF evaluation matrix *Q<sup>k</sup>* = *ak ij* .

4×6 **Step 2:** Aggregation of individual evaluation matrices

**Steps 2.1~2.2:** According to the subjective weight of decision makers *w* = (1/3, 1/3, 1/3) *T*, we can utilize Equations (21)~(23) to calculate the weighted support degree of *a<sup>k</sup> ij* as:

$$T^1 = \begin{pmatrix} 0.4667 & 0.4667 & 0.5667 & 0.5667 & 0.6333 & 0.6333 & 0.6333 \\ 0.5667 & 0.5667 & 0.6333 & 0.6000 & 0.6333 & 0.5667 \\ 0.6333 & 0.6000 & 0.6333 & 0.6333 & 0.6333 & 0.6667 \\ 0.5667 & 0.2333 & 0.6000 & 0.6333 & 0.6333 & 0.4667 \end{pmatrix};$$

$$T^2 = \begin{pmatrix} 0.5000 & 0.3667 & 0.5667 & 0.5667 & 0.6333 & 0.6000 \\ 0.6000 & 0.4667 & 0.6000 & 0.6333 & 0.6333 & 0.6000 \\ 0.6333 & 0.5667 & 0.6333 & 0.6333 & 0.4000 & 0.6333 \\ 0.6000 & 0.4333 & 0.6000 & 0.6333 & 0.6333 & 0.4667 \end{pmatrix};$$

$$T^3 = \begin{pmatrix} 0.3667 & 0.5000 & 0.6000 & 0.6000 & 0.6000 & 0.6333 \\ 0.5667 & 0.5667 & 0.6333 & 0.6333 & 0.6000 & 0.5667 \\ 0.6000 & 0.5667 & 0.6000 & 0.6000 & 0.4000 & 0.6000 \\ 0.5667 & 0.4000 & 0.5333 & 0.6000 & 0.6000 & 0.2667 \end{pmatrix}.$$

Then, the weights associated with *a<sup>k</sup> ij* can be determined as:

$$
\xi^1 = \begin{pmatrix}
0.3385 & 0.3385 & 0.3310 & 0.3310 & 0.3356 & 0.3356 \\
0.3310 & 0.3406 & 0.3356 & 0.3288 & 0.3356 & 0.3310 \\
0.3356 & 0.3380 & 0.3356 & 0.3356 & 0.2881 & 0.3356 \\
0.3310 & 0.3033 & 0.3380 & 0.3356 & 0.3356 & 0.3492
\end{pmatrix};
$$

$$
\xi^2 = \begin{pmatrix}
0.3261 & 0.2971 & 0.3310 & 0.3310 & 0.3356 & 0.3288 \\
0.3380 & 0.3188 & 0.3288 & 0.3356 & 0.3356 & 0.3380 \\
0.3356 & 0.3310 & 0.3356 & 0.3356 & 0.3356 & 0.3356 \\
0.3380 & 0.3116 & 0.3380 & 0.3386 & 0.3356 & 0.3356 \\
\end{pmatrix};
$$

$$
\xi^3 = \begin{pmatrix}
0.2971 & 0.3261 & 0.3380 & 0.3380 & 0.3288 & 0.3356 \\
0.3310 & 0.3406 & 0.3356 & 0.3356 & 0.3288 & 0.3310 \\
0.3288 & 0.3310 & 0.3288 & 0.3288 & 0.3288 & 0.3754 \\
0.3310 & 0.3043 & 0.3239 & 0.3288 & 0.3288 & 0.3754
\end{pmatrix}.
$$

**Step 2.3:** Use the q-ROFPWA operator to fuse the evaluation matrix *Q<sup>k</sup>* to obtain the collective evaluation matrix *Q* as shown in Table A1.


**Table A1.** Collective evaluation matrix.

**Step 3:** Determine the weights of criteria.

**Step 3.1:** We adopt the subjective weights of criteria in the literature [16] as *λ<sup>S</sup>* = (0.180, 0.090, 0.130, 0.130, 0.310, 0.160) *T*.

**Step 3.2:** Based on the collective evaluation matrix *Q*, we construct the programming model, i.e., Equation (26), then, the objective weights of criteria can be determined as *λ<sup>O</sup>* = (0.187, 0.157, 0.157, 0.186, 0.152, 0.161) *T*.

**Step 3.3:** Set the importance coefficient of subjective weights *ϕ* = 0.5; we can obtain the comprehensive weights of criteria as *λ* = (0.183, 0.124, 0.143, 0.158, 0.231, 0.161) *T*.

**Step 4:** Rank the green suppliers using the TODIM method under the q-ROF environment (*θ* = 1).

**Step 4.1:** Utilize Equation (31) to compute the relative weight *λjr* of criteria *Cj* concerning the reference criteria *Cr* as:

$$
\lambda\_{1r} = 0.7922, \lambda\_{2r} = 0.5368, \lambda\_{3r} = 0.6190, \lambda\_{4r} = 0.6840, \lambda\_{5r} = 1.0000, \lambda\_{6r} = 0.6970.
$$

**Step 4.2:** Compute the dominance degree of green supplier *Ai* over each green supplier as:

$$
\delta = \begin{pmatrix}
0 & -0.8829 & -4.1540 & -3.8594 \\
\end{pmatrix}.
$$

**Step 4.3:** Compute the global value of green supplier *Ai* by Equation (34):

$$
\Phi(A\_1) = 0.4074, \Phi(A\_2) = 0, \Phi(A\_3) = 1, \Phi(A\_4) = 0.6645.
$$

**Step 4.4:** Based on the global values of green suppliers, the ranking of potential green suppliers can be determined as *A*<sup>3</sup> > *A*<sup>4</sup> > *A*<sup>1</sup> > *A*2. The green supplier *A*<sup>3</sup> is the best choice for the electric automobile company.

#### **Appendix B**

The ranking of potential green suppliers can be obtained by the IF-TOPSIS method [16] as below.

**Step 1:** According to the linguistic terms of decision makers in Table 2 and the relationships between linguistic terms and intuitionistic fuzzy numbers in the literature [16], we transform the linguistic terms into IF evaluation matri ces of decision makers; then, the intuitionistic fuzzy weighted average operator [77] is utilized to fuse the individual evaluation information to determine the collective evaluation matrix as presented in Table A2.


**Table A2.** Collective evaluation matrix.

**Step 2:** According to the type of criteria, we can obtain the IF positive ideal solution *a*<sup>+</sup> and IF negative ideal solution *a*− as:

*a*<sup>+</sup> = ((0.2348, 0.6649),(0.7116, 0.1817),(0.3458, 0.5944),(0.6366, 0.2621),(1.0000, 0.0000),(0.6698, 0.2289)),

*a*− = ((1.0000, 0.0000),(0.3650, 0.5278),(1.0000, 0.0000),(0.1037, 0.8243),(0.5358, 0.4217),(0.3458, 0.5944)).

**Step 3:** Utilize the maximum average weighted distance method to construct a programming model as:

$$\begin{array}{c} \max \sum\_{i=1}^{m} \sum\_{j=1}^{n} \lambda\_{j}^{O} d\left(a\_{ij}, a^{-}\right) \\ \text{s.t. } \sum\_{j=1}^{n} \left(\lambda\_{j}^{O}\right)^{2} = 1, 0 \le \lambda\_{j}^{O} \le 1. \end{array} \tag{A1}$$

Then, we can use the Lagrange function to solve this model, and the objective weights of criteria are obtained as *λ<sup>O</sup>* = (0.253, 0.122, 0.217, 0.186, 0.117, 0.105) *T*.

**Step 4:** Set the importance coefficient of subjective weights *ϕ* = 0.5, combined with the subjective weight vector of criteria *λ<sup>S</sup>* = (0.180, 0.090, 0.130, 0.130, 0.310, 0.160) *<sup>T</sup>*, we can obtain the comprehensive weights of criteria as *λ* = (0.217, 0.106, 0.173, 0.158, 0.213, 0.133) *<sup>T</sup>*. Furthermore, the weighted IF evaluation matrix can be determined as presented in Table A3.

**Table A3.** Weighted IF evaluation matrix.


**Step 5:** Utilize the following equations to calculate the distances between each green supplier and the IF positive ideal solution *a*<sup>+</sup> and IF negative ideal solution *a*−, respectively.

$$S\_i^+ = \sum\_{j=1}^n \left( \left| \mu\_{i\bar{j}} - \mu\_{\bar{j}}^+ \right| + \left| v\_{i\bar{j}} - v\_{\bar{j}}^+ \right| \right), \tag{A2}$$

$$S\_i^- = \sum\_{j=1}^n \left( \left| \mu\_{i\bar{j}} - \mu\_{\bar{j}}^- \right| + \left| v\_{i\bar{j}} - v\_{\bar{j}}^- \right| \right). \tag{A3}$$

Subsequently, the relative closeness coefficient of each green supplier concerning the positive ideal solution can be computed by:

$$\text{CC}\_{i} = \frac{S\_{i}^{-}}{S\_{i}^{-} + S\_{i}^{+}}.\tag{A4}$$

Thus, the result can be obtained as *CC*<sup>1</sup> = 0.3430, *CC*<sup>2</sup> = 0.4743, *CC*<sup>3</sup> = 0.5533, *CC*<sup>4</sup> = 0.3520.

**Step 6:** According to the relative closeness coefficient value of each green supplier, we can determine the ranking of the green supplier as *A*<sup>3</sup> > *A*<sup>2</sup> > *A*<sup>4</sup> > *A*1; the green supplier *A*<sup>3</sup> is the best choice for the electric automobile company.

#### **Appendix C**

The ranking of potential green suppliers can be obtained by the fuzzy TODIM method [75] as below.

**Step 1:** Because of the linguistic terms utilized in the literature [75] are divided into five grades, we reconstruct the relationships between linguistic terms and triangular fuzzy numbers as presented in Table A4 to implement the numerical example in this paper.


**Table A4.** Linguistic terms and the corresponding triangular fuzzy numbers.

**Step 2:** According to Tables 2 and A4, we can transform the linguistic evaluation information of decision makers into the corresponding triangular fuzzy numbers. The weights of decision makers are considered equal in the literature [75]; thus, the collective evaluation matrix can be obtained as shown in Table A5.


**Step 3:** To obtain a more objective comparison result, we adopt the weights of criteria in the Section 5.1 as *λ* = (0.191, 0.125, 0.140, 0.156, 0.230, 0.158) *T*.

**Step 4:** Rank the green suppliers using the fuzzy TODIM method (*θ* = 1); similar to the improved TODIM method in this paper, compute the relative weight *λjr* of criteria *Cj* concerning the reference criteria *Cr* as

*λ*1*<sup>r</sup>* = 0.8304, *λ*2*<sup>r</sup>* = 0.5435, *λ*3*<sup>r</sup>* = 0.6087, *λ*4*<sup>r</sup>* = 0.6783, *λ*5*<sup>r</sup>* = 1.0000, *λ*6*<sup>r</sup>* = 0.6870.

**Step 5:** Compute the dominance degree of green supplier *Ai* over each green supplier:


**Step 6:** Compute the global value of green supplier *Ai*:

$$
\Phi(A\_1) = 0.3102, \Phi(A\_2) = 0, \Phi(A\_3) = 1, \Phi(A\_4) = 0.8072.
$$

**Step 7:** Based on the global values of green suppliers, the ranking of potential green suppliers can be determined as *A*<sup>3</sup> > *A*<sup>4</sup> > *A*<sup>1</sup> > *A*2. The green supplier *A*<sup>3</sup> is the best choice for the electric automobile company.

#### **References**


© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Review* **Application of MCDM Methods in Sustainability Engineering: A Literature Review 2008–2018**

#### **Mirko Stojˇci´c 1, Edmundas Kazimieras Zavadskas 2,\*, Dragan Pamuˇcar 3, Željko Stevi´c <sup>1</sup> and Abbas Mardani <sup>4</sup>**


Received: 16 February 2019; Accepted: 5 March 2019; Published: 8 March 2019

**Abstract:** Sustainability is one of the main challenges of the recent decades. In this regard, several prior studies have used different techniques and approaches for solving this problem in the field of sustainability engineering. Multiple criteria decision making (MCDM) is an important technique that presents a systematic approach for helping decisionmakers in this field. The main goal of this paper is to review the literature concerning the application of MCDM methods in the field of sustainable engineering. The Web of Science (WoS) Core Collection Database was chosen to identify 108 papers in the period of 2008–2018. The selected papers were classified into five categories, including construction and infrastructure, supply chains, transport and logistics, energy, and other. In addition, the articles were classified based on author, year, application area, study objective and problem, applied methods, number of published papers, and name of the journal. The results of this paper show that sustainable engineering is an area that is quite suitable for the use of MCDM. It can be concluded that most of the methods used in sustainable engineering are based on traditional approaches with a noticeable trend towards applying the theory of uncertainty, such as fuzzy, grey, rough, and neutrosophic theory.

**Keywords:** sustainability; engineering; multi-criteria decision-making

#### **1. Introduction**

The emergence of the concept of sustainability has been motivated by natural catastrophes, environmental contamination, depletion of natural resources, and other incidents. According to *Our Common Future* (Brundtland Report) adopted by the World Commission on Environment and Development in 1987 [1], sustainability implies an integrative concept that includes environmental, economic, and social aspects. These three aspects are often referred to as the three pillars of sustainability. In this way, sustainability has become a modern principle that explains the long-term relationship between the present and future generations [2]. At the same time, the term "sustainable development' has emerged, which implies "meet[ing] the needs of the present generation without compromising the ability of future generations to meet their own needs" [3]. Although there are many definitions of sustainable development [4], this is one of the most frequently quoted. In order to

achieve the balance between the three pillars of sustainability, it is necessary to define the links and interactions between them, i.e., it is necessary to know how they influence each other [5].

In order to achieve sustainability, sustainable engineering is proposed as a potential solution that implies the application of different methods. Examples may include the construction of facilities made of materials that provide energy efficiency, finding energy forms that do not release carbon dioxide into the atmosphere, designing electric vehicles, etc. According to some authors, sustainable engineering implies significantly more serious considerations of environmental and social aspects [6]. Sustainable engineering thereby observes the system as part of a global ecosystem. According to Abraham [7], the following basic principles of sustainable engineering can be set out:


Engineering is the application of scientific and mathematical principles for practical objectives, such as the processes, manufacture, design, and operation of products, while accounting for constraints invoked by environmental, economic, and social factors. There are various factors needing to be considered in order to address engineering sustainability, which is critical for the overall sustainability of human development and activity. In recent decades, decision-making theory has been a subject of intense research activity [8], due to its wide applications in different areas, such as sustainable engineering and environmental sustainability. The decision-making theory approach has become an important means of providing real-time solutions to uncertainty problems, especially for sustainable engineering and environmental sustainability problems in engineering processes. In the recent decades, several techniques and methods have been used for solving problems in the areas of environmental sustainability and sustainable engineering. Multiple criteria decision making (MCDM) is an important method that has been applied in various areas of sustainable engineering. Several prior studies have employed MCDM techniques in different areas of sustainable engineering [9–18]. In addition, several prior papers have reviewed the application MCDM and fuzzy sets theory in different areas of engineering and sustainability [11,19–26].

The main goal of the paper is to review the literature regarding the application of MCDM methods in the field of sustainable engineering. Another goal is to synthesize different areas of engineering and show effective ways of solving various problems in the field by applying various MCDM methods in various forms of uncertainty. Moreover, this review can be very useful for other studies in various areas of sustainable engineering by showing how MCDM methods can be adequate tools for decision-making processes in sustainable engineering. Furthermore, this paper highlights new, important information for all the participants in MCDM processes in sustainable engineering. In addition, this paper, to the authors' knowledge, is the first review of the literature in the area of sustainable engineering from the perspective of the application of MCDM methods.

The remainder of the paper is structured as follows. Section 2 presents the methodology, in which our algorithm for collecting and processing the articles is presented and explained in detail. Section 3 discusses the primary results of the review, i.e., the total number of MCDM articles in the field of science and technology, with an emphasis on the field of sustainable engineering. The results have been presented by various areas and the structure of the published articles has been presented by journal. Section 4 provides a detailed review of various engineering fields including, construction and

infrastructure, supply chains, transport and logistics, energy, and other. In this section, the application of MCDM methods in each of the above areas is explained in detail. Section 5 presents our conclusions.

#### **2. Methodology**

This paper reviews the collected literature on the topic of MCDM methods in sustainable engineering. In addition to searching in the Web of Science (WoS) Core Collection Database, articles were searched in online journal databases, using Google Scholar and the Google search engine using the following keywords: MCDM, sustainability, and sustainable engineering. Their combinations were also used when searching as follows: MCDM + sustainability + engineering, MCDM + sustainable engineering, MCDM + sustainability, and MCDM + engineering. All the collected articles were published in the period of 2008–2018. The research methodology is shown in Figure 1.

**Figure 1.** Brief research procedure.

By searching the WoS Core Collection Database, 4712 articles related to the application of MCDM methods in various fields of science and technology have been identified, of which 329 articles deal with the application of MCDM methods in sustainable engineering. In parallel, in the search of online journal databases with impact factors, 108 articles were found, and they were divided into five sub-areas. Based on this, the results of the primary review of articles (by publication year, by area, and by journal) are provided, while a detailed analysis and review of these articles are presented in Section 4.

#### **3. Primary Review Results**

By searching the Web of Science Core Collection database, 4712 articles (November 2018) dealing with the application of MCDM methods in various fields of science and technology were found, as shown in Figure 2.

**Figure 2.** Number of articles on the application of multiple criteria decision making (MCDM) methods in various fields of science and technology.

Figure 2 shows the top 25 areas in which studies applying MCDM methods can be categorized, indicating the number of articles for each area. It appears that the largest number of articles belong to the field of computer science and artificial intelligence (546 articles), while the application of MCDM methods in operational research occupies the second position (500 articles). The smallest number of articles has been published in the field of transport technology (61). It can be concluded that these areas are currently up to date.

In terms of the articles on the application of MCDM methods in sustainable engineering, the Web of Science Core Collection database contains 329 articles, as shown in Figure 3.

**Figure 3.** Number of articles on the application of MCDM methods in sustainable engineering.

Figure 3 shows the top 25 areas in which studies applying MCDM methods in sustainable engineering can be categorized. The largest number of articles belongs to the field of civil engineering (61), and the smallest number belongs to the fields of urban studies (6). It can be observed that the field of transportation science technology, materials science multidisciplinary, environmental studies, energy fuels and computer science software are also at the lower end. In the second position is the area of industrial engineering, followed by operational research, etc.

Figure 4 provides a review of the collected articles by publication years. There is an evident increase in the number of articles in the last few years, because environmental protection, waste minimization, renewable energy sources, energy efficiency, and the concept of sustainability in general have become increasingly frequent and significant subjects of research in many studies in the 21st century [27,28]. In addition, it appears that in 2008, there was not a single published article related to the application of MCDM methods in sustainable engineering.

Table 1 provides an overview of the number of articles collected by particular journals.



**Table 1.** *Cont.*

Based on Table 1, it can be concluded that most of the collected articles have been published in the journal *Sustainability* (14 articles), which represents 12.96% of the total number. The *Journal of Cleaner Production* can be ranked second with 10 articles or 9.26% of the total articles. Out of a total of 47 journals, 31 have published one article related to MCDM methods in sustainable engineering. It is important to note that all these journals have impact factors.

#### **4. Detailed Review Results**

All the collected articles (108 articles) on the topic of applications of MCDM methods in sustainability engineering have been classified into 5 categories: construction and infrastructure, supply chains, transport and logistics, energy, and other. It is important to mention that some areas of engineering, such as mechanical engineering, have not been taken into consideration because of the lack of articles regarding such topics. For each of the above categories, a detailed analysis of the aim and importance of the application of the individual MCDM method has been provided, and the results of the review have also been given in a table. Figure 5 shows the subdivision into the 5 subcategories with the number of articles in each subcategory.

**Figure 5.** Division of research subjects into five sub-areas.

#### *4.1. Civil Engineering and Infrastructure*

In the domain of architecture and construction, increasing attention is being paid to energy efficiency and smart buildings, and therefore, it is necessary to go towards sustainability in the design and construction of facilities and infrastructure. Consequently, it is required to select adequate materials as well. In this section, a detailed analysis of the 26 collected articles in the field of construction and infrastructure is presented.

In their work, Birgani and Yazdandoost [29] provided a framework for a new approach to addressing flood problems in urban areas. In many cases, due to unforeseen and abundant precipitation, the existing drainage network cannot receive large amounts of precipitation. For this reason, for the selection between several alternatives of the sewer system, an integrated approach that implies the sustainability and application of multi-criteria decision-making methods, i.e. the adaptive analytical hierarchy process (AHP), entropy and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) was proposed. The framework was applied to the case study for a part of the city of Tehran, Iran. The problem of floods in urban areas due to abundant precipitation was also discussed in [30]. Based on the sustainability criteria, using the AHP method for determining the weights and the Preference Ranking Organization Method for the Enrichment of Evaluations II (PROMETHEE II) for the final ranking of the alternatives, a framework for the selection of an optimum drainage system was proposed. The implementation of the framework was carried out using the example of Buraydah City, Qassim, Saudi Arabia.

Construction is an area that interacts enormously with the natural environment. A large percentage of raw materials are obtained from the earth, and in their treatment and processing and the construction of buildings, certain environmental pollution is inevitable. Lombera and Rojo [31] use the Spanish MIVES (Integrated Value Model for Sustainable Assessment) methodology to define the criteria for the sustainability of industrial buildings and to select the optimum solution with regards to them. Generally speaking, the MIVES methodology combines multi-criteria decision-making and multi-attribute utility theory (MAUT), including a value function concept and weight assignment by the AHP method [32]. A similar study was presented in a study by del Cano et al. [33], in which authors also used the MIVES method but in combination with Monte Carlo simulation in order to assess the sustainability of concrete structures. For the same purpose, de la Fuente et al. [34] applied fuzzy-MIVES. Moreover, de la Fuente et al. [34] also applied the MIVES methodology together with the AHP method in order to reduce the subjective human impact on the selection of sewage pipe material. Akhtar et al. [35] solved the same problem using only the AHP method. The MIVES methodology was also used in a study by de la Fuente et al. [36], assessing the sustainability of alternatives—the types of concrete and their reinforcement for their application in tunnels, depending on environmental, social, and economic criteria. The case study was carried out for the city of Barcelona. Pons and de la Fuente [37] used MIVES to select the most suitable concrete pillars as structural components of buildings, while Pujadas et al. [32] constructed a framework for the evaluation of heterogeneous public investments using this methodology, which is a step towards sustainable urban planning. Different economic, environmental, and social aspects were considered, with five criteria and eight indicators.

The problem of monitoring, repairing, and the returning to function of steel bridge structures is a major challenge for engineers, especially because it is necessary to make key decisions, and wrongly made decisions can be very costly. In order to exclude subjectivity in selecting alternatives in this case, Rashidi et al. [38] presented the decision support system (DSS), in which the simplified AHP (S-AHP) method is used. S-AHP combines the simple multi-attribute rating technique (SMART) and the AHP method. The aim is to help engineers in planning the safety, functionality, and sustainability of steel bridge structures. Jia et al. [39] presented a framework for the selection of bridge construction between the accelerated bridge construction (ABC) method and conventional alternatives, using TOPSIS and fuzzy TOPSIS methods.

In their work, Formisano and Mazzolani [40] presented a new procedure for the selection of the optimum solution for seismic retrofitting of existing reinforced concrete (RC) buildings, as well as optimum solutions for vertical upgrading of existing masonry constructions. The procedure involved the application of three MCDM methods, namely TOPSIS, elimination and choice expressing reality (ELECTRE), and VIseKriterijumska Optimizacija i Kompromisno Rešenje (VIKOR). In two case studies, these methods showed the same results. In their work, Terracciano et al. [41] selected cold-formed, thin-walled steel structures for vertical reinforcement and energy retrofitting systems of existing masonry constructions. The TOPSIS method for selecting alternatives based on structural, economic, environmental, and energy criteria was used.

Improving traditional buildings into modern ones must comply with technical regulations, energy requirements, comfort requirements, and the preservation of existing architecture. Siozinyte et al. [42] applied the AHP and TOPSIS grey MCDM methods to select the optimum solution for modernizing traditional buildings.

Khoshnava et al. [43] applied MCDM methods to select energy efficient, ecological, recyclable materials for building with respect to the three pillars of sustainability. In order to evaluate 23 criteria in the selection of materials, they used the decision-making trial and evaluation laboratory (DEMATEL) hybrid MCDM method together with the fuzzy analytic network process (FANP). Akadiri et al. [44] used fuzzy extended AHP (FEAHP) in order to select sustainable building materials.

In a study by Ozcan-Deniz and Zhu [45], the analytic network process (ANP) method was used to select the most environmentally friendly method for the construction of a highway, because such construction can have a great impact on the environment. Possible alternatives included different types of materials, operations, and project conditions. Constructing traffic infrastructure can greatly increase the level of safety for participants, but also reduce traffic jams. In their work, Stevic et al. [46] selected the locations for the construction of roundabouts using the rough best–worst method (BWM) and the rough weighted aggregated sum product assessment (WASPAS) approach based on the New Rough Hamy Aggregator.

In their work, Rashid et al. [47] used MCDM methods to select sustainable concrete, which implies a mixture of conventional coarse aggregate and ceramic waste aggregate. The AHP and TOPSIS methods were used to select the best performing concrete in terms of the pressure it can endure and its impact on the environment.

During and even after the construction of facilities, a large amount of natural resources is used, which adversely affects the environment. Most systems for evaluating the sustainability of facilities take into account only the environmental aspect and the environmental impact. However, it is necessary to take into account all three basic principles of sustainability, and thus Raslanas et al. [48], in their work, developed a system for evaluating the sustainability of recreational facilities, using the AHP method. Because so-called "green buildings" are environmentally friendly, attention is increasingly being given to the selection of methods for their construction. Taking into account that this is a very complex task, the application of MCDM methods is indispensable, and in the study by Tsai et al. [49], DEMATEL, ANP, and zero–one goal programming (ZOGP) methods were applied.

The selection of construction project managers plays a key role for the entire construction process. Zavadskas et al. [50] used the MCDM approach to this problem and applied AHP and additive ratio assessment (ARAS) methods. The alternatives were selected based on the criteria of education, experience, and personal abilities and skills.

When building larger facilities, i.e., implementing capital projects, it is very important to select a proper transport route for the procurement of raw materials and materials. Marzouk and Elmesteckawi [51] selected the best alternative for the construction of a power plant using the SMART method.

Because the number of vehicles on the roads is increasing every day, the number of parking spaces can hardly follow this trend. Using the MCDM method, Palevicius et al. [52] indicated the worst parking conditions in Vilnius, Lithuania, with all three aspects of sustainability, using simple additive weighting (SAW), TOPSIS, complex proportional assessment (COPRAS), and AHP method. Table 2 summarizes the applied MCDM methods in the sub-area of civil engineering and infrastructure.


**Table 2.** MCDM methods in the sub-area of civil engineering and infrastructure.

Based on Table 2, it can be concluded that the AHP method is one of the most frequently applied. In addition, it appears that AHP, as well as other methods, can be synthesized with other MCDM methods, but also with other theories such as fuzzy and grey numbers.

#### *4.2. Supply Chain Management*

Supply chains present a very complex field involving many participants. The aim of the complete supply chain is to find an optimum from the perspective of all the participants, which is a rather complex task [53–55].

Supply chain management in terms of sustainability in a number of industries is an increasingly frequent topic of research. Therefore, this section provides an analysis of 22 articles on this topic. In the review by Seuring [56], MCDM, and particularly AHP, was listed as one of the quantitative methods for improving the supply chain management. Additionally, based on the review, it can be concluded that the social component of sustainability is paid the least attention. In their review paper, Zimmer et al. [57] analyzed the use of various models to support decision-making on sustainable supplier selection. The models that were stated as the most commonly used were the mathematical/analytic ones, which include MCDM. Significantly, the biggest percentage of application belongs to the AHP method, followed by ANP, etc. The selection of suppliers, according to many authors, is one of the most demanding problems of sustainable supply chain management. Therefore, numerous methods for ranking suppliers have been developed to date, and Fallahpour et al. [58] used the fuzzy modifications of the AHP and TOPSIS methods. The abovementioned authors used the fuzzy preferences programming (FPP) method to reach the relative weights of criteria, while the fuzzy TOPSIS method was used to rank suppliers. In order to validate the methods, a case study was conducted on a real system. The fuzzy approach in combination with the TOPSIS MCDM method was

used by Govindan et al. [59] to assess the sustainable performance of suppliers. In order to perform the selection of suppliers in terms of sustainability and at the same time to take into account the business goals of the company, Dai and Blackhurst [60] presented an integrated approach based on AHP and the quality function deployment (QFD) method with four hierarchical phases. Rezaei et al. [61] presented a new methodology for the selection of suppliers consisting of three phases, where the central phase is the application of the BWM method of multi-criteria decision-making. The methodology presented can be particularly useful for companies that are looking for new markets. For the selection of suppliers, Azadnia et al. [62] proposed an integrated approach that, in addition to the fuzzy AHP method (FAHP), is based on multi-objective mathematical programming, as well as on rule-based weighted fuzzy method. According to Su et al. [63], the assessment of sustainable supply chain management and the selection of suppliers are performed using grey theory in combination with the DEMATEL method. Luthra et al. [64] presented an integrated approach to selecting suppliers consisting of a combination of AHP and VIKOR methods based on 22 criteria for all three aspects of sustainability. Because thermal power plants are the main source of electricity in China, it is necessary to make a selection of sustainable suppliers of raw materials in order to achieve sustainable development of the company. According to Zhao and Guo [65], an integrated approach is based on the fuzzy entropy–TOPSIS method. MCDM methods can be used to assess the degree of organizational sustainability of a company, as presented in [66]. Hsu et al. [67] presented a hybrid approach based on several MCDM methods in order to select suppliers in terms of carbon emissions. The observed framework for the selection of suppliers has been applied to the case of a hotel in Taiwan. A similar study was carried out by Kuo et al. [68] on the example of electronic industry. The evaluation of the supplier performance in the field of electronic industry in order to implement green supply chains is a topic of research in the study by Chatterjee et al. [17]. The authors used rough DEMATEL–ANP (R'AMATEL) in combination with rough multi-attribute ideal real comparative analysis (R'MAIRCA) method. Liu et al. [69] selected the suppliers of fresh products using the BWM and multi-objective optimization on the basis of the ratio analysis (MULTIMOORA) method.

Because innovation plays a very important role in sustainability, Gupta and Sarkis [70] presented a framework for ranking and selecting the criteria for sustainable innovations in supply chain management. This framework is based on the BWM method, and its applicability and efficiency were tested on several manufacturing companies in India. In their work, Validi et al. [71] dealt with the sustainability of the food supply chain. The TOPSIS method was used for the purpose of ranking the traffic routes, taking into account CO2 emissions and total transport costs.

A quantitative assessment of the performance of a sustainable supply chain was presented in Erol et al. [72] with regard to all three aspects of sustainability. Due to the presence of indeterminacy, it is very difficult to estimate certain criteria, which is why the authors used fuzzy techniques in addition to MCDM. More precisely, the fuzzy entropy and fuzzy MAUT methods were used. Das and Shaw [73] proposed a methodology based on the AHP and Fuzzy TOPSIS methods for selecting a sustainable supply chain, taking into account carbon emissions and various social factors. In the study by Entezaminia et al. [74], the AHP method was used to evaluate products in the supply chain according to environmental criteria such as recyclability, biodegradability, energy consumption, and product risk.

The application of information and communication technologies in supply chains can bring numerous benefits to an organization, and among the most important is sustainability. Luthra et al. [75] proposed the application of delphi and fuzzy DEMATEL methods for identifying and evaluating the guidelines for the application of these technologies in sustainable initiatives in supply chains. In Padhi et al. [76], a framework that identifies sustainable processes in supply chains for individual industries in India was presented. The ranking of industry branches was carried out using six fuzzy MCDM methods. Table 3 provides a summary of the applied MCDM methods for the sub-area of supply chain management. The decision-making process requires the prior definition and fulfillment of certain factors, especially when it comes to complex areas, such as supply chain management [77].



In the sub-area of supply chain management, based on the table, it is apparent that most authors apply the AHP and TOPSIS methods. As mentioned in the previous section, their applications can be combined with other methods.

#### *4.3. Transport and Logistics*

As in other engineering disciplines, MCDM methods are also applied in the field of transport and logistics. This section provides a review of 23 articles dealing with the above issues. In Mardani et al. [78], a review of the methods used to solve problems in transport systems was provided. The articles were systematically categorized into 10 groups, one of which was sustainability. The authors stated that according to the number of articles published on MCDM in combination with sustainability, this category could be ranked sixth.

In Jeon et al. [79], the application of MCDM methods in selecting sustainable transport plans based on the sustainability index is examined. The weighted sum model (WSM) method was used. In their work, Cadena and Magro [80] presented a new methodology for assigning weight coefficients to sustainability criteria in transport projects. In order to solve the problem of inaccuracy and subjectivity, the REMBRANDT and Delphi methods were applied.

Because the traffic system is the lifeblood of every country and one of the basis for its economic development, Baric et al. [81] proposed the application of the AHP method in selecting the best road section design in urban conditions. The tested model on the real system showed reliable results. One of the disadvantages of the AHP method is that it requires a large number of inputs. In order to solve this problem, Inti and Tandon [82] presented a modified AHP method with the characteristics of the additive transitivity of fuzzy relations. The model was tested in the selection of contractors for the construction of transport infrastructure.

In order to improve transport sustainability, one of the solutions is the application of various alternative fuels and vehicle drives. Mitropoulos and Prevedouros [83], in this way, assessed the characteristics of vehicles using the sustainability index. The identified indicators were classified into five categories of sustainability—environment, technology, energy, economy, and users—followed by the application of the WSM method for their aggregation. Additionally, Mohamadabadi et al. [84] selected the type of fuel for vehicles based on three basic aspects of sustainability. The PROMETHEE method was used for the ranking of alternatives based on five criteria. Intermodal transport can greatly improve the sustainability of the transport system. It is necessary to select the optimum location of terminals in terms of different requirements of different participants in a transport process. Therefore, Zecevic et al. [85] proposed a new hybrid MCDM model for the location selection. Sustainable transport systems have become necessary nowadays, primarily in large cities due to various adverse environmental impacts. An approach to selecting the best alternative of transport systems based on 24 criteria, classified into three categories, was defined in a study by Awasthi et al. [86]. The approach consists of three steps, and the TOPSIS method is applied in combination with fuzzy theory in order to evaluate the criteria and the selection of an alternative. Castillo and Pitfield [87] proposed the evaluative and logical approach to sustainable transport indicator compilation (ELASTIC) framework for selecting the sustainability indicators of the transport system using the AHP and SAW methods. Although, in recent years, improvements have been evidently made to methods of transport planning, according to Lopez and Monzon [88], it is necessary to apply a multidisciplinary approach based on Geographic Information System (GIS) in order to increase the level of sustainability in transport. In addition, it is necessary to integrate multi-criteria decision-making methods within the proposed approach. In his work, Simongati [89] presented a model for the selection of FREIGHT INTEGRATOR with MCDM methods and sustainability indicators. The aforementioned term represents a provider of door-to-door transport services, using different modes of transport in an efficient and sustainable way. The selection of alternatives is based on SAW and PROMETHEE methods. The assessment of transport system sustainability of some European countries based on selected economic, environmental, and social indicators is presented in the work of Bojkovic et al. [90]. The ELECTRE I method has been used together with its modification based on the absolute significance threshold (AST). A framework for the selection of sustainable transport projects in urban areas of developing countries was proposed in the work of Jones et al. [91].

The selection of alternatives is based on the localized sustainability score index using the AHP method. In addition to the AHP method, in order to assess the sustainability of various transport solutions, such as mode sharing, multimodal transport, and intelligent transport systems, Awasthi and Chauhan [92] used the Dempster–Shafer theory in the proposed hybrid approach. While the AHP method serves primarily to rank the criteria based on the weights, the Dempster–Shafer theory allows the synthesis of multiple sources of information. Dimi´c et al. [93] developed a model for strategic transport management based on Strengths, Weakness, Opportunities, Treats (SWOT) analysis, fuzzy Delphi, and DEMATEL–ANP methods.

Sustainability is a very important concept in logistics, and reverse logistics as one of its subgroups can greatly improve efficiency and the environmental aspect of business. Wang et al. [94] presented a method for identifying the collection mode for used components. A hybrid approach based on AHP and entropy weight (AHP–EW) method was used to estimate the weights of particular criteria, while the multi-attributive border approximation area comparison (MABAC) method was used to rank the alternatives. Different initiatives for city logistics (e.g., the proper location of distribution centers) can significantly contribute to raising the level of sustainability in the city. That is precisely the subject of research in Awasthi and Chauhan [95]. The MCDM methods used in the work were AHP and Fuzzy TOPSIS. Mavi et al. [96], using the fuzzy step-wise weight assessment ratio analysis (SWARA) and fuzzy MOORA methods, selected a third-party provider of reverse logistics service in the plastics industry.

One of the most current problems in logistics and supply chains is the selection of the location of the logistics center in terms of sustainability. Rao et al. [97] used the fuzzy multi-attribute group decision-making (MAGDM) approach to address the problem. Turskis and Zavadskas [98] approached the problem of selecting the location of the logistics center with the fuzzy ARAS (ARAS–F) method, while Pamucar et al. [99] used the DEMATEL–MAIRCA method for the same purpose.

Logistics are closely linked to the processing industry. Therefore, it is necessary to identify the factors that influence their interaction. For this purpose, Jiang et al. [100] applied the grey DEMATEL-based ANP method (DANP). Table 4 provides a summary of the applied MCDM methods for the sub-area of transport and logistics.


**Table 4.** MCDM methods in the sub-area of transport and logistics.

Table 4 indicates which MCDM methods are used in the field of transport and logistics. In this case, the AHP method is also the most applied MCDM method.

#### *4.4. Energy*

This section provides a review of the application of MCDM methods in the field of energy. 24 articles were analyzed, and the results have been given in textual and tabular formats. Developing renewable energy sources is a growing trend in the world on a day-to-day basis, especially when it comes to solar energy. The selection of an optimum location for the installation of photovoltaic systems is of great importance, because it can reduce the cost of the project and also ensure the maximum production of electricity. It sufficiently proves the high sustainability of such sources. Al Garni and Awasthi [101] selected the location of solar systems based on MCDM methods and GIS. The AHP method was used to evaluate the weights of feasibility criteria that directly affect the performance of the solar system. A similar study was also presented in the work of Diaz-Cuevas et al. [102], where spatial information instead of GIS was provided with the PostgreSQL-PostGIS database, which was based on Structured Query Language (SQL). The AHP method is used to determine the weights of the criteria. GIS is also necessary in selecting the location of wind farms that are also a very significant alternative source of energy. According to Sanchez-Lozano et al. [103], fuzzy MCDM methods are used to determine the weights of the criteria and the selection of an optimum alternative in solving this problem.

The selection of the optimum type of renewable energy sources using MCDM methods based on the hesitant fuzzy linguistic (HFL) term set was presented in the work of Buyukozkan and Karabulut [104]. The proposed methodology was tested using the example of the selection between several alternatives in the territory of Turkey. A similar study was presented in the work of Wu et al. [105], where a case study for China was conducted. Based on the AHP and TOPSIS methods, it was found that solar systems are the best solution. Yazdani et al. [18] presented a new hybrid approach for the selection of renewable energy technology, using DEMATEL, ANP, COPRAS, and WASPAS methods. Zhang et al. [106] used the improved MCDM method based on fuzzy measure and integral to select the "pure" form of energy between several alternatives. In their research, Troldborg et al. [107] dealt with the same issue and applied the PROMETHEE method. In their work, Klein and Whalley [108] selected between 13 renewable and non-renewable energy sources based on eight criteria. According to Tsoutsos et al. [109], the selection of an optimum renewable energy source in Crete, Greece was carried out with the PROMETHEE I and PROMETHEE II methods. Countries rich in fossil fuels are forced to seek alternative energy sources in order to reduce CO2 emissions. Pamucar et al. [110] applied the linguistic neutrosophic numbers pairwise–combinative distance-based assessment (LNN PW–CODAS) to select the optimum energy production technology in Libya. In Pamucar et al. [111], a model for the selection of a location for the construction of wind farms based on GIS in combination with two MCDM methods, BWM and MAIRCA, was presented.

Generated electricity planning is of great importance for the electric power system of a country. Mirjat et al. [112] proposed the application of the AHP method for assessing the sustainability of four types of energy models. A case study was carried out for Pakistan. The European Union is developing its energy plans, and MCDM methods find their application in the ranking of plans. According to Balezentis and Streimikiene [113], for this purpose, WASPAS, ARAS, and TOPSIS methods should be used. The selection of the best energy project between several alternatives, using MCDM methods, was considered in the work of Buyukozkan and Karabulut [104].

Because electric vehicles are becoming increasingly common on roads in the world, it is necessary to provide stations for charging them at optimum locations. Zhao and Li [114] presented a methodology based on MCDM methods. The criteria of the expanded concept of sustainability, which in addition to the traditional three aspects also includes technology, were selected based on fuzzy delphi, while the selection of the best alternative was performed using the fuzzy grey relation analysis (GRA)–VIKOR method. Guo and Zhao [115] dealt with the same issues. In order to eliminate subjectivity when selecting the location of charging stations, in addition to the basic criteria of sustainability, 11 sub-criteria were defined, in which the weights were determined on the basis of literature research, opinion of experts, and feasibility studies. The specific location selection was completed using the fuzzy TOPSIS method.

Nuclear energy implies low values of CO2 emissions into the atmosphere, which is necessary in terms of the concept of sustainability. Gao et al. [116] presented a framework for selecting the best option for a nuclear fuel cycle at a plant. In order to determine the weights of the criteria, fuzzy AHP and criteria importance through intercriteria correlation (CRITIC) were used, and the selection of alternatives was performed using the TOPSIS and PROMETHEE II methods. The selection of the optimum energy option for a thermal power plant was the subject of research in the work of Skobalj et al. [117]. The selection between seven alternatives, including revitalization and additional production by alternative energy sources, was performed on the basis of the sustainability index, which was determined by the analysis and synthesis of parameters under information deficiency (ASPID) method. The application of MCDM methods in order to select a sustainable energy solution has not been omitted even when it comes to hydroelectric power plants in the work of Vucijak et al. [118]. According to Streimikiene et al. [119], the selection between several alternative technologies for the sustainable production of electricity can be performed with the MULTIMOORA and TOPSIS methods (Barros et al. [120]). Maxim [121] also deals with the same issues in his work. He used a modified SWING method for ranking technologies. Energy is the key to the economic and social development of a particular area. In their work, Jovanovic et al. [122] proposed a new approach based on the predictions of different energy scenarios in urban areas and the application of MCDM methods for

evaluating them. Biomass implies a multitude of resources, such as plant waste, animal waste, food waste, etc. Ioannou et al. [123] used MCDM methods in their research to select the location of a biomass power plant. Table 5 provides a summary of the applied MCDM methods for the sub-area of energy.



As can be seen from Table 5, in the field of energy, MCDM methods are mainly used to solve problems of selecting the optimum type of renewable and non-renewable energy sources. In most cases, the AHP method is applied.

#### *4.5. Other Engineering Disciplines*

In addition to four previously analyzed areas of the application of MCDM methods in sustainable engineering, uncategorized works are discussed in this section. This includes 13 articles from various fields of engineering, and their detailed analysis is given below. Creating a sustainable environmental management system is of great importance for reducing environmental pollution. Khalili and Duecker [124] created a system for selecting the best solution using the ELECTRE III method. In their research, Egilmez et al. [125] applied the intuitionistic fuzzy decision making (IFDM) approach, which is the integration of fuzzy logic and MCDM theory, in order to rank and select a city (in the US and Canada) with the highest degree of environmental sustainability. According to Alwaer and Clements-Croome [126], the model for the assessment of smart household sustainability includes the application of the AHP method. The level of sustainability of temporary housing units for the accommodation of persons after natural disasters can be assessed using the MIVES method according to Hosseini et al. [127]. A review of the MCDM methods used in assessing the sustainability of the system is shown in Diaz-Balteiro et al. [128], and it can be concluded that the AHP method takes the leading position in a number of applications. In Rosen et al. [129], a new method for assessing the sustainability of renewed contaminated surfaces was developed. The proposed sustainable choice of remediation (SCORE) method is a tool for selecting between several alternatives for possible

land remediation. Ren et al. [130] developed a generic framework for the selection of sustainable technology for the treatment of sewage sludge in urban areas, using three MCDM methods: sum weighted method (SWM), digraph model, and TOPSIS. Using a MCDM method, Ren et al. [131] selected industrial systems from the aspect of sustainability. Because fossil fuel reserves are limited and atmospheric pollution is increasing, it is necessary to stimulate bio-diesel consumption. Sivaraja and Sakthivel [132] applied FAHP–TOPSIS, FAHP–VIKOR, and FAHP–ELECTRE to select the best blend of the specified fuel. In their research, Zavadskas et al. [133] selected the site for the incineration of waste, taking into account all the sustainability criteria using the new extension of the WASPAS method, the WASPAS single-valued neutrosophic Set (WASPAS–SVNS). It is known that the global population is growing each year, and it is necessary to provide an adequate amount of food. For this reason, Debnath et al. [134] selected the project portfolio for agricultural production by applying grey DEMATEL and MABAC methods. Huang et al. [135] presented a hybrid MCDM approach for the selection of materials for the production of particulate matter sensors. In their paper, Zhang et al. [136] dealt with the problem of evaluating the supply of rare minerals. For this purpose, fuzzy AHP and PROMETHEE methods were used. The application of the MCDM method within the decision support system can be of great importance to assist in emergency situations such as forest fires. The development of such a system is described in Ioannou et al. [137]. Table 6 provides a summary of the applied MCDM methods for the sub-area of other engineering disciplines.



The application of MCDM methods in other engineering disciplines is reduced to the environmental aspect of sustainability. Problems such as environmental pollution, soil contamination, air pollution, and the selection of the best fossil fuel are just a few that are solved by applying MCDM methods, of which AHP is most frequently used, according to Table 6.

#### **5. Conclusions**

In this paper, representative studies that include the application of multi-criteria decision-making models in the field of sustainability engineering have been presented. A review of about 108 studies related to the application of multi-criteria decision-making methods in the field of civil engineering and infrastructure, supply chain management, transport and logistics, energy, and other engineering disciplines provides interesting conclusions that can be useful for researchers who deal with the application of MCDM models in different engineering areas.

This literature review has shown that sustainable engineering is an area that is quite suitable for the use of MCDM. It is not surprising that the number of publications related to environmental protection, waste minimization, renewable energy sources, energy efficiency, and the concept of sustainability have tripled in the last decade. Switching to the concept of renewable energy has influenced researchers to try to exploit and improve available knowledge in decision-making.

Most of the methods used in sustainable engineering are based on traditional approaches with a noticeable trend of applying the theory of uncertainty, such as fuzzy, grey, rough, and neutrosophic theory. It can be said that the selection between existing MCDM methods is also a multi-criteria problem. Each of the methods has its advantages and disadvantages, and it is not possible to claim that any method is more suitable than others. The same applies to the selection of uncertainty theory for a considered multi-criteria problem. The choice of the method depends largely on the preferences of decision-makers and analysts. It is therefore important to consider the convenience, validity, and accessibility of methods for a problem considered. Mukhametzyanov and Pamucar [138] emphasize that the choice of method can significantly influence the decision-making process. They also emphasized that several methods should be used in a decision-making process in order to obtain a sustainable and high-quality decision. This is also an explanation of the observed trend of using a large number of methods in the literature. By analyzing the prior research presented in this review, it can be concluded that in the field of civil engineering and infrastructure, MCDM methods in most cases help to solve the problems that arise when selecting methods of building structures and roads. These problems attach great importance as objects require reinforcements in the case of seismic activities, and also, in the trend of the construction of green, ecological houses. In the sub-section of civil engineering and infrastructure there are 6 papers dealing with this topic, which amounts to 23.07% of the total. The most common method is AHP, which is used in 13 papers or 50% of the total. MCDM methods are most commonly used in this field in combination with fuzzy theory. A total of three papers (11.54%) have been analyzed that integrate fuzzy principles along with other MCDM methods. In the field of supply chain management, the selection of the supplier is the most common problem that is solved using the MCDM method. It is necessary to select the most modern supplier in terms of sustainability, but also from the point of view of the customer or user of the service. Out of the total number of analyzed papers in this field, 11 or 50% deal with this problem, and the most commonly used methods are TOPSIS and AHP with 6 papers or 27.27% each. The combination of the MCDM method with the most common fuzzy theory is represented in 8 papers or in 36.36% of the total. The analysis has shown that the selection of the location of terminals and logistics centers from the aspect of sustainability is the most important problem that is solved by MCDM methods in the field of transport and logistics. Of the total number of articles in this field, 4 or 17.39% include a subject of research that is related to the choice of location. In this case, the AHP method is the most commonly applied. More precisely, it was applied in 7 papers or 30.43% of the total number. In this field, it is often a combination of MCDM methods with fuzzy theory. It is the same number as in the previous sub-area: 8 papers or 34.78% of the total. MCDM methods in the field of energy are used to a large extent for the choice of a certain type or mode of energy production. Alternatives most often include renewable but also conventional sources of energy. In this area there are 9 such papers, or 37.50% of the total, while the number of papers in which the AHP method is applied is 8 or 33.33%. Fuzzy theory is most commonly combined with MCDM methods in this field and is present in 3 papers or 12.5% of the total. When it comes to other engineering disciplines, the application of the MCDM method is mainly to assess the sustainability of buildings, land, waste treatment technologies, and cities. Within the mentioned area, 6 papers dealing with this topic were analyzed, or 46.15% of the total number. The AHP method is also the most commonly applied in this field and is applied in 4 papers or 30.77% of the total. In addition, the application of fuzzy theory is used, along with the other methods. In this sub-area, fuzzy principles were applied in 3 papers or in 23.08% of the total.

Based on the analysis of papers and problems that are solved using the MCDM method, it can be concluded that the AHP method has the broadest application when it comes to sustainable engineering. Generally, the total number of papers involving the use of the AHP method was 38 or 35.19% of the total. Among the other theories that integrate with MCDM, fuzzy theory stands out in cases of uncertainty and imprecision with a total of 25 papers or 23.15% of the total.

It can be concluded that there has been a significant increase in the application of MCDM models in all engineering areas in the last decade. The complexity of synchronous problems forces researchers to search for more flexible and simpler methods. Therefore, it is expected that there will be a further increase in works that consider the application of existing MCDM models and the development of new models for multi-criteria decision-making. It is also expected that the validation of results using multiple methods, the development of interactive systems to support the decision-making, and the improvement of fuzzy, grey, rough, and neutrosophic theory for the consideration of uncertainty will encourage researchers in the field of sustainable engineering to expand further research towards the creation of hybrid models, upgrading the existing MCDM models.

**Author Contributions:** Each author has participated and contributed sufficiently to take public responsibility for appropriate portions of the content.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflicts of interest.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Symmetry* Editorial Office E-mail: symmetry@mdpi.com www.mdpi.com/journal/symmetry

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18