*2.2. Bonferroni-OWA*

In relation to soft mathematics and with respect to models that relate to the theory of aggregation [32,38], there is the extension of Bonferroni that allows us to add, organize, and relate information objectively and subjectively simultaneously. These models are the same ones that are applicable in artificial intelligence. This operator is called the BON-OWA. Compared to other models such as traditional statistics, the BON-OWA allows us to obtain important results by treating information simultaneously [29].

Decision-making seeking to reduce uncertainty can improve the results by applying the Bonferroni average since it builds confidence intervals and maintains the global confidence coefficient [17]. The operator is defined as follows:

$$\mathbf{B}(\mathbf{a}\_1, \mathbf{a}\_2, \dots, \mathbf{a}\_n) = \left(\frac{1}{\mathbf{n}} \frac{1}{1 - \mathbf{n}} \sum\_{\substack{\mathbf{j} = 1 \\ \mathbf{j} \neq \mathbf{k}}}^{\mathbf{n}} \mathbf{j} = 1 \quad \mathbf{a}\_{\mathbf{j}}^{\mathbf{p}, \mathbf{q}}\right)^{\frac{1}{\mathbf{r} + \mathbf{q}}} \tag{13}$$

**Definition 7.** *The Bonferroni OWA is mean-type aggregation operator. The main characteristics of the Bonferroni average (Bonferroni 1950) are that the arguments a must be greater than or equal to 0, and the parameters p and q must be greater than or equal to 0. The algorithm that combines the OWA operator and the Bonferroni average can be defined as:*

$$\text{BON}-\text{OWA}(\mathbf{a}\_1, \dots, \mathbf{a}\_n) = \left(\frac{1}{n}\sum\_{i} a\_i^p \text{OWA}\_W(V^i)\right)^{\frac{1}{\tau+q}},\tag{14}$$

*where OWAW V<sup>i</sup> represents the expression* ⎛⎜⎜⎝ 1 *<sup>n</sup>*−1 ∑*n j* = 1 *j* = *i aqj* ⎞⎟⎟⎠ *with (Vi) being the vector of all*

*ajs except ai and w being an n* − 1 *vector Wi associated with αi whose components wij are the OWA weights. Let W be an OWA weighting vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* ∑*i wi* = 1*. Then, we can define this aggregation as OWAW V<sup>i</sup>* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*)*, where <sup>a</sup>πk*(*j*) *is the largest element in the tuple Vi and wi* = 1 *<sup>n</sup>*−1*for all i*.

**Definition 8.** *The Bonferroni IOWA (BON-IOWA) (Blanco-Mesa* et al. *2019b) is a mean-type aggregation operator that is defined as follows.*

$$\text{BON} - \text{IOWA}(\langle u\_1, a\_1 \rangle, \dots, \langle u\_{\text{n}}, a\_{\text{n}} \rangle) = \left(\frac{1}{n} \sum\_{i} a\_i^{\text{r}} IOWA\_W(\mathcal{V}^i)\right)^{\frac{1}{\tau + q}},\tag{15}$$

*where V<sup>i</sup> is the vector of all aj except ai. Let W be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* ∑*i wi* = 1*, where the weights are associated according to the largest value of ui, and ui is the order-inducing variable. Then, we can define this aggregation as IOWAW V<sup>i</sup>* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*)*, where <sup>a</sup>πk* (*j*) *is the largest element in the <sup>n</sup>*−*1 tuple Vi = Vi* = (*<sup>u</sup>*1, *<sup>a</sup>*1,...,*ui*−1, *ai*−<sup>1</sup>,*ui*+1, *ai*+<sup>1</sup>,...,*un*, *an*).

**Definition 9.** *The Bonferroni HOWA (BON-HOWA) [31] is a mean-type aggregation operator that has an associated weighting vector W with wi* ∈ [0, 1] *and* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n such that:*

$$\text{HON} - \text{HOWA}(a\_1, \dots, a\_n) = \left(\frac{1}{n} \sum\_{i} a\_i^r HOWA\_W\left(V^i\right)\right)^{\frac{1}{r+q}},\tag{16}$$

*where V<sup>i</sup> is the vector of all ajs except ai. Let W be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n. Thus, the sum of the weights wj is bounded to n or can be unbounded if the weighting vector W* = − ∞ ≤ ∑*nj*=<sup>1</sup> *wj* ≤ ∞*. Then, we can define this aggregation as HOWAW V<sup>i</sup>* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*)*, where <sup>a</sup>πk* (*j*) *is the largest element in the <sup>n</sup>*−*1 tuple Vi = Vi* = (*<sup>a</sup>*1,..., *ai*−1, *ai*+1,..., *an*).

**Definition 10.** *The Bonferroni PrOWA (BON-PrOWA) [41] is a mean-type aggregation operator that has an associated weighting vector W:*

$$\text{PRON} - \text{ProWA}(\langle u\_1, a\_1 \rangle, \langle u\_2, a\_2 \rangle, \dots, \langle u\_n, a\_n \rangle) = \frac{1}{n} \Big( \sum\_{i=1}^{n-1} a\_i^r \text{ProWA}\_W(V^i) \Big)^{\frac{1}{r+q}}, \quad \text{(17)}$$

*where V<sup>i</sup> is the vector of all ajs except ai. Let Wi be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* ∑*nj*=<sup>1</sup> *wj* = 1. *Wi is the vector of weights (associated with the vector Vi) of all wjs except wi*. *r and q are parameters such that r*, *q* ≥ 0*. The ais are the some prioritized ais, where column "i" is omitted to perform the sorting. A vector of n* − *1 elements remains. r is the exponent of ai.*

**Definition 11.** *The Bonferroni IHOWA (BON-IHOWA) [31] is a mean-type aggregation operator that has an associated weighting vector W, where wi* ∈ [0, 1] *and* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n, such that:*

$$BON-IIOWA(\langle u\_1, a\_1 \rangle, \dots, \langle u\_n, a\_n \rangle) = \left(\frac{1}{n} \sum\_{i} a\_i^r IHOWA\_W(V^i)\right)^{\frac{1}{r+q}},\tag{18}$$

*where V<sup>i</sup> is the vector of all ajs except ais. Let W be an OWA weighting vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n. The weights are associated according*

*to the largest value of ui, and ui is the order-inducing variable. Likewise, the sum of the weights wj is bounded to n or can be unbounded if the weighting vector W* = − ∞ ≤ ∑*n j*=1 *wj* ≤ ∞*. Then, we can define this aggregation as IHOWAw Vi* = ∑*<sup>n</sup>*=<sup>1</sup> *j*=1 *wia<sup>π</sup>k*(*j*) , *where <sup>a</sup>πk*(*j*) *is the largest element in the n* −*1 tuple Vi = Vi* = (*<sup>u</sup>*1, *<sup>a</sup>*1,...,*ui*−1, *ai*−<sup>1</sup>,*ui*+1, *ai*+<sup>1</sup>,...,*un*, *an*).

#### **3. New Propositions—Bonferroni Prioritized Induced Heavy OWA Operator**

In this section, a new proposition considering the theoretical aspects and the revision of the definitions of each of the methods necessary for its proposal is presented. Here, it is important to mention that the authors of a previous work [35] established an approach that improves the evaluation of the transparency index that considers the degree of importance, reordering and weight factors. This approach seeks to improve the integration of information by considering their interrelationship, their interdependence and the importance of the information and including a nonlimited to zero weighting vector and an induced weighting vector capable of assigning weights according to the highly complex conditions of the analyzed phenomena [35,42]. Thus, this approach offers a better way to understand the information than just the measurement [42]. In this sense, the proposition presented is called the Bonferroni prioritized induced heavy OWA operator (BON-PrIHOWA). From this main proposal, the BON-PrOWA, PrIOWA and PrHOWA are also presented. Each of the propositions is presented below.

**Proposition 1.** *The Bonferroni PrOWA (BON-PrOWA) is a mean-type aggregation operator that has an associated weighting vector W:*

$$\text{BON} - \text{ProWA}(\langle u\_1, a\_1 \rangle, \langle u\_2, a\_2 \rangle, \dots, \langle u\_n, a\_n \rangle) = \frac{1}{n} \Big( \sum\_{i=1}^{n-1} a\_i^r \text{ProWA}\_W(V^i) \Big)^{\frac{1}{r+q}}, \tag{19}$$

*where Vi is the vector of all ajs except ai. Let Wi be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* ∑*n j*=1 *wj* = 1. *Wi is the vector of weights (associated with the vector Vi) of all wjs except wi*. *r and q are parameters such that r*, *q* ≥ 0*, The ais are some prioritized ais, where column "i" is omitted to perform the sorting. A vector of n* − *1 elements remains. r is the exponent of ai*. *PrOWA W Vi* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*) *, where <sup>a</sup>πk* (*j*) *is the largest element in the n* − *1 tuple Vi = Vi* = (*<sup>a</sup>*1,..., *ai*−1, *ai*+1,..., *an*).

**Proposition 2.** *The Bonferroni PrIOWA is a mean-type aggregation operator that has an associated weighting vector W:*

$$\text{BON} - \text{PrIOWA}(\langle u\_1, a\_1 \rangle, \langle u\_2, a\_2 \rangle, \dots, \langle u\_n, a\_n \rangle) = \frac{1}{n} \left( \sum\_{i=1}^{n-1} a\_i^r \text{PrOWA}\_W \left( V^i \right) \right)^{\frac{1}{r+q}}, \tag{20}$$

*where Vi is the vector of all ajs except ai. Let Wi be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* ∑*n j*=1 *wj* = 1. *Wi is the vector of weights (associated with the vector Vi) of all wjs except wi, the weights are associated according to the largest value of ui and ui is the order-inducing variable. r and q are parameters such that r*, *q* ≥ 0*, The ais are some prioritized ais, where column "i" is omitted to perform the sorting. A vector of n* − *1 elements remains. r is the exponent of ai. Then, PrIOWA W Vi* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*) *, where <sup>a</sup>πk* (*j*) *is the largest element in the n* − *1 tuple Vi = Vi* = (*<sup>u</sup>*1, *<sup>a</sup>*1,...,*ui*−1, *ai*−<sup>1</sup>,*ui*+1, *ai*+<sup>1</sup>,...,*un*, *an*).

**Proposition 3.** *The Bonferroni PrHOWA is a mean-type aggregation operator that has an associated weighting vector W:*

$$\text{PRON} - \text{PrHOWA}(\langle u\_1, a\_1 \rangle, \langle u\_2, a\_2 \rangle, \dots, \langle u\_n, a\_n \rangle) = \frac{1}{n} \Big( \sum\_{i=1}^{n-1} a\_i^r \text{PrOMA}\_W \left( V^i \right) \Big)^{\frac{1}{r+q}}, \tag{21}$$

*where V<sup>i</sup> is the vector of all ajs except ai. Let Wi be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n*. *Wi is the vector of weights (associated with the vector Vi) of all wjs except wi. Thus, the sum of the weights wj is bounded to n or can be unbounded if the weighting vector W* = − ∞ ≤ ∑*nj*=<sup>1</sup> *wj* ≤ ∞. *r and q are parameters such that r*, *q* ≥ 0*. The ais are some prioritized ais, where column "i" is omitted to perform the sorting. A vector of n* − *1 elements remains. r is the exponent of ai*. *PrHOWAW V<sup>i</sup>* = ∑*<sup>n</sup>*−<sup>1</sup> *j*=1 *wia<sup>π</sup>k* (*j*)*, where <sup>a</sup>πk* (*j*) *is the largest element in the n* − *1 tuple Vi = Vi* = (*<sup>a</sup>*1,..., *ai*−1, *ai*+1,..., *an*).

**Proposition 4.** *The BON-PrIHOWA on V<sup>i</sup> is the vector of all ajs except ai. Let Wi be an OWA weighing vector of dimension n* − 1 *with components wi* ∈ [0, 1]*, where* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n*.

$$\text{BON} - PrIHOWA}(\langle u\_1, a\_1 \rangle, \langle u\_2, a\_2 \rangle, \dots, \langle u\_n, a\_n \rangle) = \frac{1}{n} \left( \sum\_{l=1}^{n-1} a\_l' PrIHOWA} \langle V^l \rangle \right)^{\frac{1}{r+q}},\tag{22}$$

*where Wi is the vector of weights (associated with the vector Vi) of all wjs except wi. Let W be an OWA weighting vector of dimension n* − 1 *with components wi* ∈ [0, 1] *when* 1 ≤ ∑*nj*=<sup>1</sup> *wj* ≤ *n, where the weights are associated according to the largest value of ui and ui is the order-inducing variable. The induced ui given to the elements ai is given in an ascending or a descending manner according to the criteria of each decision maker. Therefore, each element ai has an associated induced ui. Likewise, the sum of weights wj is bounded to n or can be unbounded if the weighting vector W* = −∞ ≤ ∑*nj*=<sup>1</sup> *wj* ≤ ∞*. Likewise, r and q are parameters such that r*, *q* ≥ 0*. The ais are the same prioritized ais, where column "i" is omitted to perform the sorting. A vector of n* − *1 elements remains. r is the exponent of ai. Then, we can define this aggregation as Pr IHOWAw V<sup>i</sup>* = ∑*<sup>n</sup>*=<sup>1</sup> *j*=1 *wia<sup>π</sup>k*(*j*) *, where <sup>a</sup>πk*(*j*) *is the largest element in the <sup>n</sup>*−*1 tuple Vi = Vi* = (*<sup>u</sup>*1, *<sup>a</sup>*1,...,*ui*−1, *ai*−<sup>1</sup>,*ui*+1, *ai*+<sup>1</sup>,...,*un*, *an*).

#### **4. Evaluation of the Transparency Websites in Mexico**

## *4.1. Aggregation Operators Calculation*

The objective of this paper is to use and apply the operators proposed in Section 3 to rank the transparency websites of the states in Mexico. As mentioned previously, government transparency is vital for the development of countries, and therefore, the possibility of using web pages to report and be able to make complaints and reports is of the utmost importance to facilitate interaction with users. In Mexico, the transparency websites are measured and ranked using five components, which are as follows [14]:


The questionnaire used to measure these websites has 63 items, and within the present investigation, the data from the last evaluation are used, which is that of 2017. The main problem of the actual ranking is that all five components have the same importance to the ranking. Because of that, not all states seek ways to improve their transparency because one good component can improve the final score, even when some components have a score of 0. The qualification of each component for each of the 32 states of Mexico is given in Table A1. Finally, the steps to use the BON-PrOWA operator and other extensions are as follows.

*Step 1.* Locate different experts that give information regarding each of the components of the ranking of transparency websites. The information that will be requested is (a) weights, (b) heavy weights and (c) induced values. The profile of the experts for this article was as follows: (a) they had minimum of five years of experience within the governmen<sup>t</sup>

sector, specifically in areas related to transparency; and (b) they work or worked directly with governmen<sup>t</sup> transparency websites.

*Step 2.* With the information provided by each expert, generate different classifications using the BON-OWA, BON-IOWA, BON-HOWA and BON-IHOWA operators.

*Step 3.* With the results obtained in Step 3, unify the information of the different experts based on the BON-PrOWA, BON-PrIOWA, BON-PrHOWA and BON-PrIHOWA operators, where the results of each expert are given a specific weight according to their experience in the field.

*Step 4.* Finally, the results are compared and analyzed.

To more clearly visualize the process to obtain the results, a simplified graph is presented (see Figure 1).

**Figure 1.** Flowchart of the steps to use the Bonferroni prioritized induced heavy ordered weighted average (BON-PIHOWA) operator.

#### *4.2. Evaluation of the Determinants of Transparency*

*Step 1.* The information was provided by five experts. The conditions for being selected were as follows: (a) must be an active worker in an institution related to transparency and (b) must have more than 10 years in a similar position. The information provided by the experts is given in Tables 1–3.



**Table 2.** Heavy weights (heavy weights are the weights that will be used to calculate the heavy ordered weighted average (HOWA) operator. Their difference with the weights in Table 1 is that heavy weights are not bounded to <sup>∑</sup>*nj*=<sup>1</sup> *wj* = 1; in this sense, the weighting vector can be from 1≤ <sup>∑</sup>*nj*=<sup>1</sup>*wj* ≤ *n*) provided by the experts.


**Table 3.** Induced values provided by the experts. Induced values are the values that will be used in the induced ordered weighted average (IOWA) operator, instead of a reordering step based on the value of the arguments, in this case, will be based on the induced value determined by the experts, generating a different reordering between the arguments and the weights. Also, the weights used in the Bonferroni induced ordered weighted average (Bon-IOWA) and Bonferroni prioritized induced ordered weighted average (Bon-PIOWA) operators are from Table 1 and for the Bonferroni induced heavy ordered weighted average (Bon-IHOWA) and Bonferroni prioritized induced heavy ordered weighted average (Bon-PIHOWA) operators are from Table 2.


*Step 2.* With the information provided in Step 1, generate the results using the BON-OWA, BON-IOWA, BON-HOWA and BON-IHOWA operators to understand the process that has been performed. An example using the information of expert 1 for the state of Zacatecas will be explained in detail, assuming that the process will be the same for all other states and experts. The values of *q* and *p* are equal to 1.

The first thing is determine the vectors *<sup>V</sup>i*, and the results are

> *V*1 = (90, 56, 85.71, 60) *V*2 = (56, 85.71, 60, 100) *V*3 = (85.71, 60, 100, 90) *V*4 = (60, 100, 90, 56) *V*5 = (100, 90, 56, 85.71)

Next, the BON-OWA operator is applied. Then, a weight is assigned to each attribute according to a maximum criterion, and the results are

$$\begin{array}{l} V\_{1} = [(90 \times 0.30) + (56 \times 0.15) + (85.71 \times 0.20) + (60 \times 0.20)] = 64.54\\ V\_{2} = [(56 \times 0.15 + 85.71 \times 0.20 + 60 \times 0.20 + 100 \times 0.30)] = 67.54\\ V\_{3} = (85.71 \times 0.15 + 60 \times 0.15 + 100 \times 0.20 + 90 \times 0.20) = 59.86\\ V\_{4} = (60 \times 0.15 + 100 \times 0.30 + 90 \times 0.20 + 56 \times 0.15) = 65.40\\ V\_{5} = (100 \times 0.30 + 90 \times 0.20 + 56 \times 0.15 + 85.71 \times 0.15) = 69.26\\ BON - OWA = \left(\frac{[(64.54 \times 100) + (67.54 \times 90) + (59.86 \times 56) + (65.40 \times 85.71) + (69.26 \times 60)]}{5}\right)^{\frac{1}{1+1}}\\ BON - OWA = 71.62 \end{array}$$

All the results for each state and expert are presented in Table A2.

In the case of the calculation for the BON-IOWA operator, the vectors *Vi* are the same as those used in the BON-OWA operator. The next step is the association of the weights with the attributes that in this case will be performed by using the induced variables instead of the values of the attributes. Here, the results for Zacatecas are the following.

$$\begin{array}{l} V\_1 = [(90 \times 0.20) + (56 \times 0.30) + (85.71 \times 0.20) + (60 \times 0.15)] = 64.1\\ V\_2 = [(56 \times 0.20) + (85.71 \times 0.30) + (60 \times 0.20) + (100 \times 0.15)] = 66.11\\ V\_3 = [(85.71 \times 0.20) + (60 \times 0.20) + (100 \times 0.15) + (90 \times 0.15)] = 59.86\\ V\_4 = [(60 \times 0.30) + (100 \times 0.20) + (90 \times 0.15) + (56 \times 0.15)] = 65.40\\ V\_5 = [(100 \times 0.20) + (90 \times 0.30) + (56 \times 0.15) + (85.71 \times 0.15)] = 68.26\\ BON - IOMA = \underbrace{\left[\frac{[(64.11 \times 100) + (66.11 \times 90) + (59.86 \times 56) + (65.40 \times 85.71) + (68.26 \times 60)]}{5}\right]^{\frac{1}{1+1}}}\_{BON} \\ \text{BON - OWA = 71.29} \end{array}$$

All the results for each state and expert are presented in Table A3.

In the case of the BON-HOWA operator, the vectors *Vi* are also the same, but the weights will the ones presented in Table 2 and will be ordered with the arguments with a maximum criterion. Therefore, the results for Zacatecas are the following.

*V*1 = [(90 × 0.30) + (56 × 0.20) + (85.71 × 0.20) + (60 × 0.20)] = 67.34 *V*2 = [(56 × 0.20) + (85.71 × 0.20) + (60 × 0.20) + (100 × 0.30)] = 70.34 *V*3 = [(85.71 × 0.20) + (60 × 0.20) + (100 × 0.20) + (90 × 0.20)] = 67.14 *V*4 = [(60 × 0.20) + (100 × 0.30) + (90 × 0.20) + (56 × 0.20)] = 71.20 *V*5 = [(100 × 0.30) + (90 × 0.20) + (56 × 0.20) + (85.71 × 0.20)] = 76.34 *Bon* − *HOWA* = [(67.34×<sup>100</sup>)+(70.34×<sup>90</sup>)+(67.14×<sup>56</sup>)+(71.20×85.71)+(76.34×<sup>60</sup>)] 5 1 1+1 *BON* − *HOWA* = 74.17

All the results for each state and expert are presented in Table A4.

Finally, the BON-IHOWA operator is constructed. The vectors *Vi* are the same as the other operators, but the weights will be the ones in Table 2 and will be ordered based on the induced values of Table 3 The results for Zacatecas are the following.

$$\begin{aligned} V\_1 &= [(90 \times 0.20) + (56 \times 0.30) + (85.71 \times 0.20) + (60 \times 0.20)] = 66.91 \\ V\_2 &= [(56 \times 0.20) + (85.71 \times 0.30) + (60 \times 0.20) + (100 \times 0.20)] = 68.91 \\ V\_3 &= [(85.71 \times 0.20) + (60 \times 0.20) + (100 \times 0.20) + (90 \times 0.20)] = 67.14 \\ V\_4 &= [(60 \times 0.30) + (100 \times 0.20) + (90 \times 0.20) + (56 \times 0.20)] = 71.20 \\ V\_5 &= [(100 \times 0.20) + (90 \times 0.30) + (56 \times 0.20) + (85.71 \times 0.20)] = 75.34 \\ BON - IHOWA &= \left(\frac{[(66.91 \times 100) + (68.91 \times 90) + (67.14 \times 56) + (71.20 \times 85.71) + (75.34 \times 60)]}{5}\right)^{\frac{1}{7.1}} \\ BON - IHOWA &= 73.86 \end{aligned}$$

All the results for each state and expert are presented in Table A5. *Step 3.* With all the results obtained in Step 2, the results for the BON-POWA, BON-PIOWA, BON-PHOWA and BON-PIHOWA operators can be obtained. The weights associated with each expert are the following: *e*1 = 0.30, *e*1 = 0.10, *e*1 = 0.20, *e*1 = 0.15 and *e*1 = 0.25. The result for each operator for Zacatecas is as follows.

*BON* − *POWA* = [(71.62 × 0.30) + (72.77 × 0.10) + (72.60 × 0.20) + (71.12 × 0.15) + (72.47 × 0.25)] = 72.07 *BON* − *PIOWA* = [(71.29 × 0.30) + (71.97 × 0.10) + (71.77 × 0.20) + (70.77 × 0.15) + (71.32 × 0.25)] = 71.39 *BON* − *PHOWA* = [(74.17 × 0.30) + (75.28 × 0.10) + (76.06 × 0.20) + (73.77 × 0.15) + (75.48 × 0.25)] = 74.93 *BON* − *PIHOWA* = [(73.86 × 0.30) + (74.63 × 0.10) + (75.21 × 0.20) + (73.44 × .15) + (74.75 × 0.25)] = 74.37

> The results for all the states are presented in Table A6.

#### *4.3. Discussion of the Results*

After an analysis of the different results obtained and presented in Tables A2–A6, the main changes that are found are as follows.

Based on the top 10 results of the different aggregation operators and experts, the first four positions do not change at all with the different aggregation operators and experts. In this sense, even when the importance of each component varies, the four best states remain the same: Zacatecas, Oaxaca, Nuevo Leon and Puebla. Then, according to the aggregation operator and expert that we analyze, the ranking can change. For example, in ranks five and six, we usually find the states of San Luis Potosi and Nayarit, respectively, but with the use of the BON-IHOWA operator, the positions change to Nayarit and San Luis Potosi, respectively. The other remaining positions vary, but the states remain the same and are Tlaxcala, Sonora, Yucatan and Queretaro.

Based on the bottom 10 results, the first four positions (as in the case of the top 10) remain the same considering the different aggregation operators and experts. In this sense, the worst states are Chihuahua, Ciudad de Mexico, Aguascalientes and Campeche. Then, the fifth and sixth positions are Tabasco and Guerrero depending on the aggregation operator and expert. Finally, positions seven to ten can change drastically. For example, for expert 1, from the BON-OWA operator, Chiapas is considered among the bottom 10 states and Jalisco is not; however, according to the information provided by expert 2, Jalisco is among the bottom 10 and Chiapas is not. This is important because in this process, it is possible to see that depending on the importance that is given to the information, the states can be or cannot be in the bottom 10 list.

The same analysis can be performed for the states in the middle of the ranking, and they change positions based on the different experts and aggregation operators. First, the top 10 of the lists does not change at all, but it is possible to see some notable changes as the ones explained in the bottom 10 analysis. This information is important for policymakers and governments to analyze to change and implement public policies according to the deficiency of each state, which can vary depending on the importance given to the components. Additionally, as seen, the ranking changes, and the benefits and governmen<sup>t</sup> support for the states can be rearranged because of their positions in the ranking.
