**Multi-Attribute Decision Making Based on Stochastic DEA Cross-E**ffi**ciency with Ordinal Variable and Its Application to Evaluation of Banks' Sustainable Development**

#### **Jinpei Liu 1,2, Mengdi Fang 1, Feifei Jin 1,\*, Chengsong Wu <sup>1</sup> and Huayou Chen <sup>3</sup>**


Received: 14 February 2020; Accepted: 17 March 2020; Published: 18 March 2020

**Abstract:** Multi-attribute decision making (MADM) is a cognitive process for evaluating data with different attributes in order to select the optimal alternative from a finite number of alternatives. In the real world, a lot of MADM problems involve some random and ordinal variables. Therefore, in this paper, a MADM method based on stochastic data envelopment analysis (DEA) cross-efficiency with ordinal variable is proposed. First, we develop a stochastic DEA model with ordinal variable, which can derive self-efficiency and the optimal weight of each attribute for all decision making units (DMUs). To further improve its discrimination power, cross-efficiency as a significant extension is proposed, which utilizes peer DMUs' optimal weight to evaluate the relative efficiency of each alternative. Then, based on self-efficiency and cross-efficiency of all DMUs, we construct corresponding fuzzy preference relations (FPRs) and consistent fuzzy preference relations (FPRs). In addition, we obtain the priority weight vector of all DMUs by utilizing the row wise summation technique according to the consistent FPRs. Finally, we provide a numerical example for evaluating operation performance of sustainable development of 15 listed banks in China, which illustrates the feasibility and applicability of the proposed MADM method based on stochastic DEA cross-efficiency with ordinal variable.

**Keywords:** stochastic DEA; multi-attribute decision making; ordinal variable; cross-efficiency

#### **1. Introduction**

Sustainable development (SD) is a widely used phrase and idea, which firstly emerged in the context of environmental concerns [1–3]. However, with the development of society and economy, we gradually realized the significance of sustainable development of economy. To some extent, the operation performance of banks can reflect economic trends. Therefore, it is important to maintain the sustainable development of banks. Recently, sustainable development of banks has become a hotspot. Munir and Gallagher [4] proposed that optimizing the benefits and costs can improve sustainable development of banks. Xue et al. [5] considered that adjusting and optimizing the layout of the physical branches of commercial banks is crucial to its sustainable development. Jiang and Han [6] suggested that adopting diversification strategy is beneficial to achieving sustainable development of banks.

Multi-attribute decision making (MADM) is one of the most common and popular research fields in the theory of decision science [7]. It assumes that there exists a set of alternatives with multiple attributes which decision makers (DMs) need to evaluate. The purpose of MADM is to select the optimal one from a finite number of alternatives. Generally speaking, each MADM problem includes two parts: classifying and ranking. Classifying can be considered as the grouping of the alternatives based on the similarities of attributes. Ranking is defined as the rank of alternatives from the optimal to the worst [8]. In recent years, some methods have been proposed to handle MADM problems, such as total sum (TS) method [9], simple additive weighting (SAW) method [10], the analytic hierarchy process (AHP) method [11], multiplicative analytic hierarchy process (MAHP) method [12], the technique for order preference by similarity to ideal solution (TOPSIS) method [13], and data envelopment analysis (DEA) method [14].

In MADM problems, we need some decision making information including attribute values and attribute weights, which denote the characteristics of alternatives and relative importance of attributes, respectively [7]. Nevertheless, attribute values are known, so we have to obtain attribute weights by the aforementioned approaches. However, DEA is a nonparametric programming efficiency rating technique for evaluating the relative efficiency of DMUs with multiple inputs and outputs, whose evaluation results come from input and output data [15–17]. Compared with other methods, attribute weights derived by DEA is relatively objective. Therefore, DEA has been widely applied in many fields for different purposes [18–20], such as assessment of environmental sustainability [21,22], supplier selection [23–25], and evaluation of the influence of E-marketing on hotel performance [26].

Due to the inherent complexity and competition of the real world, MADM problems often involve some random and ordinal variables. However, previous DEA studies have been undertaken in a deterministic environment, which cannot solve the above situation. Therefore, it is necessary to incorporate the stochastic variable into DEA. Then, we propose the stochastic DEA model with ordinal variable. Recently, stochastic DEA has become the research hotspot. A well-known method to extend DEA to the case of random inputs and outputs is to utilize chance constrained programming (CCP) [27], which was proposed by Charnes and Cooper [28]. The CCP admits random data variations and permits constraint violations up to the specified probability limit. Khodabakhshi et al. [29] extended the super-efficiency DEA model to an input-oriented super-efficiency stochastic DEA model by CCP. The major contributions on the stochastic DEA may be attributed to the work of Sengupta [30]. A prominent characteristic of his study is that stochastic DEA is transformed into a deterministic equivalent [31]. In addition to Sengupta's work, Sueyoshi [31] proposed a DEA future analysis method that considered how to integrate future information into DEA, and then applied it to restructure strategy of a Japanese petroleum company. Wu et al. [32] followed the Cooper's approach to develop stochastic DEA model by considering undesirable outputs with weak disposability.

However, although stochastic DEA model can evaluate MADM problems with random and ordinal variables, the following disadvantages also exist. One is that DEA identifies many efficient alternatives where efficiency score is equivalent to one, and cannot further discriminate them. Another is that the ranking order of all alternatives cannot involve appraisal of peer DMUs and influences the accuracy and persuasion of evaluation results in MADM problems. Faced with these drawbacks, many researchers have taken efforts to modify DEA methods, including weight restriction, super-efficiency, cross-efficiency and so on [33]. The weight restriction methods commonly attach additional constrains to relative weights, including absolute weight restriction [34], common weights [35], and cone ratio restriction [36]. Nevertheless, all weight restriction approaches use priority information or predefined parameters, thus they are subjective to some extent. Andersen and Petersen [37] proposed a super efficiency method for ranking DMUs. The cross-efficiency evaluation method is an important extension, which is invented to utilize peer DMUs' optimal weights to appraise relative efficiency of each DMU [38–41]. Compared with the traditional DEA approach concentrating on self-efficiency evaluation, cross-efficiency evaluation method has the following main characteristics: (1) taking peer-evaluation of all alternatives into account and guaranteeing a unique ranking order for whole DMUs, (2) eliminating unrealistic weight schemes without predetermining any weight restrictions, and (3) effectively distinguishing better DMUs and poor DMUs [42]. Owing to these superiorities, cross-efficiency evaluation results can be more reasonable and acceptable. Therefore, cross-efficiency has been widely applied to diverse fields, including R&D project selections and the ranking of universities' comprehensive ability. However, there are few DEA methods to handle the issue that

MADM problems involve some random and ordinal variables. To further extend the application of DEA on aforementioned MADM problems, we develop a MADM method based on stochastic DEA cross-efficiency with ordinal variable.

The main purpose of this paper is to address the MADM problems with random and ordinal variables. Therefore, we propose a MADM method based on stochastic DEA cross-efficiency with ordinal variable. The major characteristics of this method are presented as follows. One is that both stochastic variable and ordinal variable are incorporated into DEA model, which is considerably consistent with the actual circumstance. The other is that it simultaneously considers the self-efficiency and cross-efficiency in evaluation process of MADM problems, and then constructs corresponding consistent FPRs. Subsequently, we calculate the priority weight vector of all alternatives by utilizing the row wise summation technique and derive the full ranking order of them.

The rest of this paper is organized as follows. In Section 2, we briefly introduce the traditional CCR model and its correlative properties. In Section 3, we develop a MADM method based on stochastic DEA cross-efficiency with ordinal variable. Section 4 gives a numerical example for evaluating operation performance of sustainable development of 15 listed banks in China, which illustrates the applicability of this proposed approach. Finally, some conclusions and future research work are presented in Section 5.

#### **2. Preliminaries**

In this section, we briefly review some basic concepts of DEA model. DEA is a data-oriented methodology for identifying efficiency production frontiers and evaluating the relative efficiency of DMUs that multiple inputs of production factors produce certain amount outputs [43]. Suppose that there are n DMUs to be evaluated, where each DMU is characterized by its production process of consuming m inputs to generate s outputs. For convenience, the inputs and outputs of *DMUj*(*j* = 1, 2, ··· , *n*) are denoted as *xij*(*i* = 1, 2, ··· , *m*) and *yrj*(*r* = 1, 2, ··· ,*s*), respectively. To evaluate performance of specific *DMUk*, Charnes et al. [44] proposed the following model to calculate its relative efficiency under the assumption of constant returns to scale (CRS).

$$\begin{aligned} \min & \theta\_k \\ \text{s.t.} & \sum\_{j=1}^n \lambda\_j y\_{rj} \ge y\_{rk}, r = 1, 2, \dots, s, \\ & \sum\_{j=1}^n \lambda\_j \mathbf{x}\_{lj} \le \theta\_k \mathbf{x}\_{ik}, i = 1, 2, \dots, m, \\ & \lambda\_j \ge 0, j = 1, 2, \dots, n. \end{aligned} \tag{1}$$

The above model is called input-oriented CCR model, where λ*<sup>j</sup>* are the nonnegative multipliers used to aggregate existing DMUs into a virtual one [45], θ*<sup>k</sup>* is the relative efficiency score of *DMUk*. To understand the CCR model clearly, we give the dual form of the CCR model:

$$\begin{aligned} \max z &= \sum\_{r=1}^{s} \omega\_r y\_{rk} \\ \text{s.t.} & \sum\_{r=1}^{s} \omega\_r y\_{rj} - \sum\_{i=1}^{m} \mu\_i x\_{ij} \le 0, j = 1, 2, \dots, n, \\ & \sum\_{\substack{m=1 \\ i=1}}^{m} \mu\_i x\_{ik} = 1, \\ & \omega\_r \ge 0, r = 1, 2, \dots, s; \mu\_i \ge 0, i = 1, 2, \dots, m. \end{aligned} \tag{2}$$

where *xij* and *yrj* are the inputs and outputs of *DMUj*(*j* = 1, 2, ··· , *n*), μ*<sup>i</sup>* and ω*<sup>r</sup>* are the input and output weights. *xik* and *yrk* are the inputs and outputs of specific *DMUk*, respectively. The optimal solution of the objective function is the relative efficiency of *DMUk*. If the efficiency score of *DMUk* is less than one, the *DMUk* is defined as DEA inefficient. Conversely, if the efficiency score is equal to one, the *DMUk* is considered as DEA efficient. In the following, we extend the CCR model by incorporating discretionary variable, ordinal variable and stochastic variable. Then, we develop a MADM method.

#### **3. Multi-Attribute Decision Making Method**

Generally speaking, MADM is an evaluation process where the optimal alternative needs to be chosen from a finite number of feasible alternatives based on a set of attributes [8]. Owing to the inherent complexity and competition of real world, MADM problems often involve some random and ordinal variables. However, the traditional DEA approach assumes that all inputs and outputs are discretionary where they are under the control of management, thus it insufficiently addresses the above situation. Therefore, we propose a stochastic DEA model with ordinal variable, which constructs production frontiers that incorporate inefficiency and stochastic error [45].

#### *3.1. Stochastic DEA Model with Ordinal Variable*

The basic CCR model supposes that all inputs and outputs are deterministic. In other words, they are under the control of management [45]. However, in the real world, there are many situations where some inputs and outputs are out of the control of management. Hence, the aforementioned models need to be modified to adapt to these circumstances. First, we assume that *I* denotes the set including all input variables, and then divide them into two categories: a set of discretionary inputs *ID*(*i* = 1, 2, ··· , *p*), and a set of ordinal inputs *IO*(*i* = *p* + 1, *p* + 2, ··· , *m*). We rewrite model (3) in a new form as follows:

$$\begin{aligned} \max z &= \sum\_{r=1}^{s} \omega\_{r} y\_{rk} \\ \text{S.t.} & \sum\_{r=1}^{s} \omega\_{r} y\_{rj} - \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ij}^{1} - \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ij}^{2} \le 0, \\ & \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ik}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ik}^{2} = 1, \\ & \omega\_{r} \ge 0, \mu\_{i}^{1} \ge 0, \mu\_{i}^{2} \ge 0, j = 1, 2, \dots, n. \end{aligned} \tag{3}$$

The above model simultaneously considers discretionary and ordinal inputs. The model (3) is an output-oriented model in which we find the optimal output value on the condition that the input values are fixed. The optimal solution of the objective function of model (3) is the self-efficiency of the specific *DMUk*. The symbols μ<sup>1</sup> *<sup>i</sup>* and <sup>μ</sup><sup>2</sup> *<sup>i</sup>* represent weight multipliers of the discretionary inputs and ordinal inputs, respectively.

It is notable that this study pays attention to real situations where we can control the quantity of inputs, while being unable to control the outputs. The reason is that the quantity of outputs relies on many external factors such as economic factors, political factors and other social factors. Therefore, the output is commonly considered as stochastic variable. The traditional DEA model for performance evaluation is deterministic type, which does not take the random errors of output variable into account in production process. However, stochastic DEA constructs production frontiers that incorporate both inefficiency and stochastic error, which moves the frontiers closer to the bulk of the producing units [45]. Therefore, the measured technical efficiency of DMUs is improved comparing to the deterministic model. In this subsection, we introduce the stochastic outputs into model (3). Suppose that all stochastic outputs are denoted by \$*yrj*(*<sup>r</sup>* <sup>=</sup> 1, 2, ··· ,*s*), and each \$*yrj* has a certain probability distribution. The following model (4) is developed:

$$\begin{aligned} \max & \mathbf{E} \left( \sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{rk} \right) \\ \text{S.t.} & \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ik}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ik}^{2} = 1, \\ & \Pr \left( \frac{\sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{lj}}{\sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ij}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ij}^{2}} \le \beta\_{j} \right) \ge 1 - \alpha\_{j}, \\ & \omega\_{r} \ge 0, \mu\_{j}^{1} \ge 0, \mu\_{j}^{2} \ge 0, j = 1, 2, \cdots, n. \end{aligned} \tag{4}$$

The above model is designed to evaluate the expected efficiency of the specific *DMUk*. The inequality constraint guarantees that the probability of the efficiency score of *DMUj* less than or equal to <sup>β</sup>*<sup>j</sup>* should be higher than 1 <sup>−</sup> <sup>α</sup>*j*. The symbols (ω*r*, <sup>μ</sup><sup>1</sup> *<sup>i</sup>* , <sup>μ</sup><sup>2</sup> *<sup>i</sup>* ) represent weight multipliers of stochastic outputs, discretionary inputs and ordinal inputs, respectively. Pr denotes a probability and the superscript "~" expresses that \$*yrj* is a stochastic variable. The other symbol <sup>β</sup>*<sup>j</sup>* is a predefined value whose range is between 0 and 1. β*<sup>j</sup>* stands for a desirable level of efficiency of *DMUj*, which is determined by outside conditions including decision level of management or market circumstances [31]. Meanwhile, α*<sup>j</sup>* is also a prescribed value whose range is between 0 and 1. It is considered as an allowable risk level that violates the related constraints.

To obtain the computational feasibility, the stochastic DEA model should convert into the deterministic DEA model. In this paper, we utilize the CCP technique to transform the second constraint of model (4) into the following form.

$$\Pr\left(\frac{\sum\_{r=1}^{s} \omega\_r \overline{y}\_{rj} - \sum\_{r=1}^{s} \omega\_r \overline{y}\_{rj}}{\sqrt{\Pi\_j}} \le \frac{\beta \left| \sum\_{i=1}^{p} \mu\_i^1 \mathbf{x}\_{ij}^1 + \sum\_{i=p+1}^{m} \mu\_i^2 \mathbf{x}\_{ij}^2 \right|}{\sqrt{\Pi\_j}} \right) \ge 1 - \alpha\_{j\prime} j = 1, 2, \cdots, n,\tag{5}$$

where *yrj* is the expected value of \$*yrj* and

$$\mathcal{U}\_{j} = \begin{pmatrix} \omega\_{1} & \omega\_{2} & \cdots & \omega\_{s} \end{pmatrix} \times \begin{pmatrix} \mathcal{V}(\widetilde{y\_{1}}) & \operatorname{Cov}(\widetilde{y\_{1}}, \widetilde{y\_{2}}) & \cdots & \operatorname{Cov}(\widetilde{y\_{1}}, \widetilde{y\_{s}}) \\ \operatorname{Cov}(\widetilde{y\_{2}}, \widetilde{y\_{1}}) & \mathcal{V}(\widetilde{y\_{2}}) & \cdots & \operatorname{Cov}(\widetilde{y\_{2}}, \widetilde{y\_{s}}) \\ \vdots & \vdots & \ddots & \vdots \\ \operatorname{Cov}(\widetilde{y\_{s}}, \widetilde{y\_{1}}) & \operatorname{Cov}(\widetilde{y\_{s}}, \widetilde{y\_{2}}) & \cdots & \operatorname{V}(\widetilde{y\_{s}}) \end{pmatrix} \times \begin{pmatrix} \omega\_{1} \\ \omega\_{2} \\ \vdots \\ \omega\_{s} \end{pmatrix} j = 1, 2, \cdots, n. \tag{6}$$

*Uj*(*j* = 1, 2, ··· , *n*) represents the variance-covariance matrix of the *DMUj* where the symbol "*V*" stands for a variance and the symbol "*Cov*" denotes a covariance. To follow the CCP technique, this subsection introduces a new variable which follows the standard normal distribution with zero mean and unity variance.

$$\widetilde{Z}\_{j} = \frac{\sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{r\bar{j}} - \sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{r\bar{j}}}{\sqrt{l\mathcal{U}\_{\bar{j}}}}, j = 1, 2, \cdots, n. \tag{7}$$

Therefore, the Formula (5) can be rewritten as follows:

$$\Pr\left\{\overline{Z}\_{j} \le \frac{\beta\_{j}\left\{\sum\_{i=1}^{p} \mu\_{i}^{1} x\_{ij}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} x\_{ij}^{2}\right\} - \sum\_{r=1}^{s} \alpha\_{r} \overline{y}\_{rj}}{\sqrt{\Pi\_{j}}} \right\} \ge 1 - \alpha\_{j}, j = 1, 2, \cdots, n. \tag{8}$$

After a simple transformation, we can obtain the following formula.

$$\frac{\beta\_j \left\{ \sum\_{i=1}^p \mu\_i^1 \mathbf{x}\_{ij}^1 + \sum\_{i=p+1}^m \mu\_i^2 \mathbf{x}\_{ij}^2 \right\} - \sum\_{r=1}^s \omega\_r \overline{y}\_{rj}}{\sqrt{\Pi\_j}} \ge \Phi^{-1} \{ 1 - \alpha\_j \}, j = 1, 2, \dots, n. \tag{9}$$

where Φ represents a cumulative normal distribution function and Φ−<sup>1</sup> denotes its inverse function. Based on Equation (9), the model (4) can be rewritten as follows:

$$\begin{array}{l} \max\limits\_{\begin{subarray}{c} \boldsymbol{\mu} \sim \mathcal{D}\_{r} \\ \boldsymbol{\mu} = 1 \end{subarray}} \sum\_{\begin{subarray}{c} \boldsymbol{\mu}\_{1}^{1} \\ \boldsymbol{\mu}\_{i}^{1} = \boldsymbol{\mu}\_{i}^{1} \end{subarray}} \mu\_{i} \overline{\mathcal{J}\_{r}} \boldsymbol{x}\_{\text{ik}}^{1} + \sum\_{\begin{subarray}{c} \boldsymbol{\mu}\_{i}^{1} \\ \boldsymbol{\mu}\_{i}^{2} \end{subarray}} \mu\_{i}^{2} \mathbf{x}\_{\text{ik}}^{2} = \mathbf{1}, \\\ \boldsymbol{\beta}\_{i} \Big{(} \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ij}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ij}^{2} \Big{)} - \sum\_{r=1}^{s} \omega\_{r} \overline{\mathcal{J}\_{ij}} \\\ \omega\_{r} \ge 0, \mu\_{i}^{1} \ge 0, \mu\_{i}^{2} \ge 0, j = 1, 2, \cdots, n. \end{array} \tag{10}$$

The second inequality constraint of model (10) includes quadratic expression and brings computational difficulty. To further simplify the computational process, we suppose that each stochastic output is denoted by \$*yrj* <sup>=</sup> *yrj* <sup>+</sup> *hrj*δ(*<sup>r</sup>* <sup>=</sup> 1, 2, ··· ,*s*; *<sup>j</sup>* <sup>=</sup> 1, 2, ··· , *<sup>n</sup>*), where *yrj* is the expected value of \$*yrj* and *hrj* is its standard deviation. <sup>δ</sup> is assumed to follow a standard normal distribution *N*(0, 1). *Bj* represents the covariance matrix of *DMUj*. Under such an assumption, *Bj* can be defined as follows:

$$B\_{j} = \begin{pmatrix} h\_{1j}^{2} & h\_{1j}h\_{2j} & \cdots & h\_{1j}h\_{sj} \\ h\_{2j}h\_{1j} & h\_{2j}^{2} & \cdots & h\_{2j}h\_{sj} \\ \vdots & \vdots & \ddots & \vdots \\ h\_{sj}h\_{1j} & h\_{sj}h\_{2j} & \cdots & h\_{sj}^{2} \end{pmatrix} \tag{11}$$

Hence, *Uj* can be rewritten as the following form,

$$\mathbf{M}\_{j} = \begin{pmatrix} \omega\_{1} & \omega\_{2} & \cdots & \omega\_{s} \end{pmatrix} \times \begin{pmatrix} h\_{1j}^{2} & h\_{1j}h\_{2j} & \cdots & h\_{1}h\_{sj} \\ h\_{2j}h\_{1j} & h\_{2j}^{2} & \cdots & h\_{2}h\_{sj} \\ \vdots & \vdots & \ddots & \vdots \\ h\_{sj}h\_{1j} & h\_{sj}h\_{2j} & \cdots & h\_{sj}^{2} \end{pmatrix} \times \begin{pmatrix} \omega\_{1} \\ \omega\_{2} \\ \vdots \\ \omega\_{s} \end{pmatrix} = \left(\sum\_{r=1}^{s} \omega\_{1}h\_{rj}\right)^{2}, \forall r = 1,2,\cdots,s; j = 1,2,\cdots,n. \tag{12}$$

By incorporating Equation (12) into model (10), then the stochastic DEA model with ordinal variable can be transformed into the following equivalent linear programming:

$$\begin{array}{lcl}\max \sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{rk} \\ \text{S.t.} \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ik}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ik}^{2} = 1, \\ \quad \beta\_{j} \Big( \sum\_{i=1}^{p} \mu\_{i}^{1} \mathbf{x}\_{ij}^{1} + \sum\_{i=p+1}^{m} \mu\_{i}^{2} \mathbf{x}\_{ij}^{2} \Big) - \sum\_{r=1}^{s} \omega\_{r} \overline{y}\_{rj} \geq \sum\_{r=1}^{s} \omega\_{r} h\_{lrj} \Phi^{-1} \Big( 1 - \alpha\_{j} \big), \\ \quad \omega\_{r} \ge 0, \mu\_{i}^{1} \ge 0, \mu\_{i}^{2} \ge 0, j = 1, 2, \cdots, n. \end{array} \tag{13}$$

Here, the dual form of model (13) is presented as follows:

$$\begin{aligned} &\min \theta\_k \\ &S.t. \sum\_{j=1}^n \lambda\_j \Big(\boldsymbol{\beta}\_j \mathbf{x}\_{ij}^1\big) \le \theta\_k \mathbf{x}\_{ik'}^1 i = 1, 2, \dots, p, \\ &\sum\_{j=1}^n \lambda\_j \Big(\boldsymbol{\beta}\_j \mathbf{x}\_{ij}^2\big) \le \theta\_k \mathbf{x}\_{ik'}^2 i = p + 1, p + 2, \dots, m, \\ &\sum\_{j=1}^n \lambda\_j \Big[\overline{y}\_{rj} + h\_{rj} \Phi^{-1} \Big(1 - \alpha\_j\big)\Big] \ge \overline{y}\_{rk'} r = 1, 2, \dots, s, \\ &\lambda\_j \ge 0, 0 \le \alpha\_j \le 1, 0 \le \beta\_j \le 1, j = 1, 2, \dots, n. \end{aligned} \tag{14}$$

We can derive the optimal weights ( ω*r*, μ<sup>1</sup> *<sup>i</sup>* , <sup>μ</sup><sup>2</sup> *<sup>i</sup>* ) of outputs and inputs by solving model (13). Based on the optimal weights of *DMUk*, the cross-efficiency of *DMUj* is calculated by the following formula:

$$E\_{kj} = \frac{\sum\_{r=1}^{s} \omega\_{rk} \overline{y}\_{rj}}{\sum\_{i=1}^{p} \mu\_{ik}^{1} x\_{ij}^{1} + \sum\_{i=p+1}^{m} \mu\_{ik}^{2} x\_{ij}^{2}}, k, j = 1, 2, \cdots, n, k \neq j. \tag{15}$$

which is the peer evaluation of *DMUk* to *DMUj*. Then, we obtain the cross-efficiency matrix.

$$E = \begin{pmatrix} E\_{11} & E\_{12} & \cdots & E\_{1n} \\ E\_{21} & E\_{22} & \cdots & E\_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ E\_{n1} & E\_{n2} & \cdots & E\_{nn} \end{pmatrix} \tag{16}$$

However, we cannot derive priority weight vector of all DMUs by cross-efficiency matrix *E*. Therefore, we need to construct corresponding preference relations to yield the priority weight vector of whole alternatives.

#### *3.2. Constructing the Consistent Fuzzy Preference Relations for Ranking DMUs*

It is known that traditional ways to construct a preference relation are based on experts' subjective evaluation involving their professional knowledge and ideas, which lead to different preference information for different experts [46]. However, compared with traditional approaches, using the pairwise efficiency derived by DEA method to construct a preference relation is more objective. In this subsection, we present the following specific procedures of construction process. First, we can obtain the efficiency scores *Ekk*, *Ekj*, *Ejk*, *Ejj*(*k*, *j* = 1, 2, ··· , *n*) by solving model (13) and calculating Equation (15). Then, we construct corresponding fuzzy preference relations (FPRs)*<sup>R</sup>* = ! *rkj*" *n*×*n* , the element of R is defined as follows:

$$r\_{kj} = \frac{E\_{kk} + E\_{jk}}{E\_{kk} + E\_{kj} + E\_{jk} + E\_{jj}}, r\_{j\bar{j}} = 0.5, j = 1, 2, \cdots, n. \tag{17}$$

where *<sup>R</sup>* = ! *rkj*" *<sup>n</sup>*×*<sup>n</sup>* is characterized by *rkj* <sup>+</sup> *rjk* <sup>=</sup> 1 and *rjj* <sup>=</sup> 0.5. *rkj* represents the evaluation of unit *<sup>k</sup>* over unit *j*. If *rkj* > 0.5, it denotes that unit *k* is superior to unit *j*. Conversely, if *rkj* < 0.5, it stands for that unit *<sup>j</sup>* is superior to unit *<sup>k</sup>*. Based on FPRs *<sup>R</sup>* = ! *rkj*" *n*×*n* , we can construct corresponding consistent FPRs *<sup>A</sup>* = ! *akj*" *<sup>n</sup>*×*<sup>n</sup>* by utilizing the following formulas.

$$c\_k = \sum\_{j=1}^{n} r\_{kj} = \sum\_{j=1}^{n} \frac{E\_{kk} + E\_{jk}}{E\_{kk} + E\_{jk} + E\_{jj} + E\_{kj}}, k = 1, 2, \cdots, n. \tag{18}$$

$$a\_{kj} = \frac{c\_k - c\_j}{2(n-1)} + 0.5.\tag{19}$$

Based on the consistent FPRs *<sup>A</sup>* = ! *akj*" *n*×*n* , we can derive the priority weight vector of all alternatives by using the row wise summation technique and obtain the whole ranking order. The priority weight vector *vk*(*k* = 1, 2, ··· , *n*) of *DMUk* is calculated by the following equation,

$$\upsilon\_k = \frac{\sum\_{j=1}^n a\_{kj}}{\sum\_{k=1}^n \sum\_{j=1}^n a\_{kj}} = \frac{\sum\_{j=1}^n a\_{kj} + \frac{n}{2} - 1}{n(n-1)}.\tag{20}$$

In summary, we show the detailed procedures of MADM method based on stochastic DEA cross-efficiency with ordinal variable.

Step 1: Solve model (13); we obtain the self-efficiency *Ekk*(*k* = 1, 2, ··· , *n*) and the optimal weights μ1<sup>∗</sup> *<sup>i</sup>* (*<sup>i</sup>* <sup>=</sup> 1, 2, ··· , *<sup>p</sup>*), <sup>μ</sup>2<sup>∗</sup> *<sup>i</sup>* (*i* = *p* + 1, *p* + 2, ··· , *m*),ω<sup>∗</sup> *<sup>r</sup>*(*r* = 1, 2, ··· ,*s*).

Step 2: Utilize Formula (15) to calculate the cross-efficiency *Ekj*(*k j*, *k*, *j* = 1, 2, ··· , *n*) by the optimal weights of other peer DMUs.

Step 3: Use Equation (17) to calculate the value of *rkj*(*k*, *j* = 1, 2, ··· , *n*) and construct the FPRs *<sup>R</sup>* = ! *rkj*" .

*n*×*n* Step 4: Construct corresponding consistent FPRs *<sup>A</sup>* = ! *akj*" *<sup>n</sup>*×*<sup>n</sup>* based on the FPRs *<sup>R</sup>* <sup>=</sup> ! *rkj*" *<sup>n</sup>*×*<sup>n</sup>* by utilizing Equations (18) and (19).

Step 5: Obtain the priority weight vector *vk*(*k* = 1, 2, ··· , *n*) of *DMUk* by calculating the Formula (20).

Step 6: Rank all alternatives in accordance with the descending order of priority weight vector *vk*(*k* = 1, 2, ··· , *n*) and select the optimal one.

#### **4. Example and Discussion**

With the development of society, we gradually realize the significance of sustainable development of economy. To some extent, the operation performance of banks can reflect the economic trend. Therefore, it is important to maintain sustainable development of banks. In this section, we provide a numerical example for evaluating operation performance of sustainable development of 15 listed banks in China, which illustrates practicability and validity of the proposed MADM method based on stochastic DEA cross-efficiency with ordinal variable. The 15 listed banks are Bank of China (*DMU*1), Construction Bank of China (*DMU*2), Industrial and Commercial Bank of China (*DMU*3), Agricultural Bank of China (*DMU*4), Industrial Bank Co., Ltd. (*DMU*5), Bank of Communications (*DMU*6), Shanghai Pudong Development Bank (*DMU*7), Ping An Bank Co., Ltd. (*DMU*8), China Minsheng Bank (*DMU*9), China Merchants Bank (*DMU*10), China Citic Bank (*DMU*11), China Everbright Bank (*DMU*12), Huaxia Bank (*DMU*13), Beijing Bank (*DMU*14) and Shanghai Bank (*DMU*15), respectively. Owing to operating similar business, these banks compete with each other. Then, we want to know the bank with the best performance under the same conditions. Therefore, we have to evaluate the relative performance of all listed banks by aforementioned method and obtain a full ranking of them. Here, we employ the intermediation approach to determine input and output factors of these banks. Compared with other approaches, this method is more suitable for evaluating the whole bank and superior in evaluating efficiency of bank's profitability. Then, it also reduces heavy computation and is considerably consistent with bank's daily operation. Therefore, based on the intermediation approach, we determine four input factors (m = 4) and two output factors (s = 2). The input factors consist of (i) fixed assets (*x*1), which stand for the capital value of tangible assets; (ii) labor costs (*x*2), which refer to the costs of the full-time employees; (iii) interest expense (*x*3) and the number of branches (*x*4). The output factors include the amount of the loan (*y*1) and the amount of deposit (*y*2). Among these six attributes, *x*1, *x*<sup>2</sup> and *x*<sup>3</sup> are considered as the discretionary variables, *x*<sup>4</sup> is the ordinal variable, *y*<sup>1</sup> and *y*<sup>2</sup> are assumed

as the stochastic variables. Our data come from the national Tai'an database. Table 1 gives a summary of the inputs and outputs. Table 2 gives order ranking for branches' number of all listed banks. Table 3 gives descriptive statistics of raw data.


**Table 1.** Input and output variables.



**Table 3.** Descriptive statistics of raw data.


There are two parameters which are not part of the given database: α and β. We run the stochastic DEA model (13) in Matlab software with different values for these parameters to see the sensitivity of the result. Table 4 shows self-efficiency scores of 15 listed banks which are calculated with diverse combinations between α = {0.05, 0.1, 0.2} and β = {0.8, 0.85, 0.9, 0.95, 1}. It presents the values of three statistics of self-efficiency, including the minimum, maximum and the mean. As suggested by Sueyoshi [31], regular trends are found in Table 4. It is notable that the mean, the maximum and the minimum of the self-efficiency increase as α or β increases. However, there are two cases that exist in Table 4 and cannot be viewed as exceptions. One is that an increase in β from 0.95 to 1 decreases the maximum of self-efficiency from 1 to 0.9754 when α = 0.1. The other is that the maximum of self-efficiency has no variation between β = 0.95 and β = 1 under the condition of α = 0.2. It is obvious that there is smaller difference among self-efficiency scores under the condition that α or β chooses diverse values. Therefore, we choose α = 0.1 and β = 0.95 for the rest of the paper.

With the original data, we complete Step 1 of the developed method. In the following, we will accomplish Step 2 to 6. In Step 2, we use the optimal attribute weights of each bank to calculate the cross-efficiency of the 15 listed banks by utilizing Formula (15) and the results are presented in Table 5. In Table 5, *Ekj*(*k* = 1, 2, ··· , 15) denotes the peer evaluation of *DMUk* to *DMUj*. In Step 3, we utilize the Formula (17) to calculate the value of *rkj*(*k*, *j* = 1, 2, ··· , 15) and construct corresponding FPRs *<sup>R</sup>* = ! *rkj*" <sup>15</sup>×15. Table <sup>6</sup> shows the values of the FPRs *<sup>R</sup>*. In Step 4, we construct the consistent FPRs *<sup>A</sup>* = ! *akj*" <sup>15</sup>×<sup>15</sup> by using Equations (18) and (19). Table <sup>7</sup> presents the values of the consistent FPRs *<sup>A</sup>*. In Step 5, we obtain the priority weight vector of each listed bank by utilizing Equation (20). In Step 6,

we can select the optimal one by ranking all listed banks in accordance with the descending order of priority weight vector *vk*(*k* = 1, 2, ··· , 15) and the result is documented in Table 8.


**Table 4.** Self-efficiency under different α and β.

**Table 5.** Cross-efficiency and self-efficiency score.



**Table 6.** Fuzzy preference relations *R*.

**Table 7.** Consistent fuzzy preference relations *A*.



**Table 7.** *Cont*.

**Table 8.** Ranking of 15 listed banks.


We can obtain the ranking of all listed banks:

$$\begin{aligned} \text{DMUL}\_{10} > \text{DMUL}\_{6} > \text{DMUL}\_{2} > \text{DMUL}\_{3} > \text{DMUL}\_{4} > \text{DMUL}\_{1} > \text{DMUL}\_{8} > \text{DMUL}\_{11} \\ > \text{DMUL}\_{7} > \text{DMUL}\_{9} > \text{DMUL}\_{5} > \text{DMUL}\_{12} > \text{DMUL}\_{15} > \text{DMUL}\_{14} > \text{DMUL}\_{13}. \end{aligned} \tag{21}$$

From the ranking in Table 7, we find that the optimal DMU is selected as *DMU*10. It is obvious that China Merchants Bank is the listed bank with the best operation performance. However, according to self-efficiency of all listed banks, we derive the following ranking:

$$\begin{aligned} DMU\_{10} &= DMU\_6 = DMU\_2 > DMU\_3 > DMU\_1 > DMU\_4 > DMU\_8 > DMU\_{11} \\ &> DMU\_5 > DMU\_7 > DMU\_9 > DMU\_{12} > DMU\_{15} > DMU\_{14} > DMU\_{13}. \end{aligned} \tag{22}$$

We cannot select the best bank in accordance with the above ranking result. In addition, the ranking result obtained from the developed method is different from that derived by traditional DEA approach. The stochastic DEA cross-efficiency with ordinal variable method effectively distinguishes all listed banks and yields the whole ranking. Meanwhile, it can greatly avoid impact of subjectivity of experts and strengthen the discrimination power. Therefore, our proposed method is reliable and valid compared with the traditional DEA method.

#### **5. Conclusions**

In this article, we proposed MADM method based on stochastic DEA cross-efficiency with ordinal variable and applied it to evaluating operation performance of sustainable development of 15 listed banks in China. First, we obtained self-efficiency scores of each bank and optimal attribute weights by solving stochastic DEA model. Then, we calculated cross-efficiency of all listed banks by utilizing the optimal attribute weights. Subsequently, according to self-efficiency and cross-efficiency of whole banks, we constructed corresponding FPRs and consistent FPRs. Finally, we used the row wise summation technique to derive the priority weight vector of all listed banks. Based on the unique ranking order of whole banks, we selected the best one.

In summary, the developed MADM method based on stochastic DEA cross-efficiency with ordinal variable is proved effective for evaluating MADM problems. The advantages of this approach are presented as follows. One is that it simultaneously incorporates stochastic variable and ordinal variable, which is considerably consistent with actual circumstances. The other is that it takes cross-efficiency into account in evaluation process of MADM problems and constructs corresponding FPRs, which guarantee the objectivity and persuasion of evaluation results. Furthermore, it requires no assumption of the functional relationships between multiple inputs and multiple outputs of alternatives, and all evaluation results come from original data. However, our method exists some limitations. One is that the stochastic output variable is assumed to follow standard normal distribution and directly applied to the stochastic DEA model. Standard normal distribution is one of the many probability distributions, we need to examine whether other distributions can be used for stochastic DEA model. Another is that the value of parameters α and β is predefined. We have no mature approach to find the optimal value of parameters α and β.

In the future research, we intend to design an integrated method that combines DEA with multiplicative FPRs to handle performance evaluation of MADM problems. Another is that we need to consider the relation among different types of variables in MADM problems and extend existing DEA methods to address it.

**Author Contributions:** All of the authors contributed to this research. F.J. and J.L. provided case and idea. H.C. and M.F. were responsible for data collection and analysis. J.L. and C.W. provided revised advice. M.F. wrote the first draft of the article. All authors have read and agreed to the published version of the manuscript.

**Funding:** The work was supported by the National Natural Science Foundation of China (Nos. 71901001, 71501002, 71871001, 71771001), the Construction Fund for Scientific Research Conditions of Introducing Talents in Anhui University (No. S020118002/085), the Key Research Project of Humanities and Social Sciences in Colleges and Universities of Anhui Province (Nos. SK2019A0013, SK2018A0605), the Humanities and Social Sciences Planning Project of the Ministry of Education (No. 20YJAZH066), Project of Anhui Ecological and Economic Development Research Center, the Natural Science Foundation of Hefei University (No. 17ZR06ZDA), and the Natural Science Foundation for Distinguished Young Scholars of Anhui Province (No. 1908085J03).

**Conflicts of Interest:** The authors declare no conflicts of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
