A Hierarchical Aggregation Approach for Indicators Based on Data Envelopment Analysis and Analytic Hierarchy Process

Abstract

This research proposes a hierarchical aggregation approach using Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP) for indicators. The core logic of the proposed approach is to reflect the hierarchical structures of indicators and their relative priorities in constructing composite indicators (CIs), simultaneously. Under hierarchical structures, the indicators of similar characteristics can be grouped into sub-categories and further into categories. According to this approach, we define a domain of composite losses, i.e., a reduction in CI values, based on two sets of weights. The first set represents the weights of indicators for each Decision Making Unit (DMU) with the minimal composite loss, and the second set represents the weights of indicators bounded by AHP with the maximal composite loss. Using a parametric distance model, we explore various ranking positions for DMUs while the indicator weights obtained from a three-level DEA-based CI model shift towards the corresponding weights bounded by AHP. An illustrative example of road safety performance indicators (SPIs) for a set of European countries highlights the usefulness of the proposed approach.

1. Introduction

Individual indicators are multidimensional measures that can assess the relative positions of entities (e.g., countries) in a given area [1]. A Composite indicator (CI) is a mathematical aggregation of individual indicators into a single score. Two simple but popular aggregation methods in the context of multi-criteria decision-making (MCDM) are the weighted sum (WS) method and the weighted product (WP) method [2]. Some researchers have recently pointed out that the WP method may have some advantages over the WS method in CI construction [3,4,5]. However, the assignment of weights to indicators is still a main source of difficulty in the application of these methods. Fortunately, the recent methodological advances in operations research and management science (OR/MS) have provided us with two powerful tools, namely data envelopment analysis (DEA) and analytic hierarchy process (AHP), which can be used as weighting and aggregation tools in CI construction.

Data Envelopment Analysis is a nonparametric method to assess the relative efficiency of a group of DMUs based on their distance from the best-practice frontier. In this method each DMU can freely choose its own weights to maximize its performance [6].

The standard DEA models are formulated using multiple inputs and multiple outputs of DMUs. The application of this group of models in CI construction can be found in [7,8,9]. However, in recent years much more attention has been focused on the application of a new group of DEA models in the field of composite indicators which is known as the “benefit of the doubt” (BOD) approach. In the BOD approach, all indicators are treated as outputs without explicit inputs, i.e., the property of “the larger is the better” [10,11,12].

In light of the possibility of neglecting the priority of various indicators, some critics have questioned the validity and stability of CIs obtained via DEA. Decision makers (DMs), in some contexts, have value judgments concerning the relative priority of indicators that should be taken into account in CI construction.

Alternatively, AHP is a systematic MCDM method to generate the true or approximate weights based on the well-defined mathematical structures of pairwise comparison metrics.

The application of AHP in CI construction provides a priori information about the relative priority of indicators [13,14,15]. AHP usually involves three basic functions: structuring complexities, measuring on a ratio-scale and synthesizing [16]. One of the advantages of AHP is its high flexibility to be combined with the other OR/MS techniques [17]. AHP can be combined with DEA in different ways. The most common approach is the estimation of parameters of weight restrictions on the DEA models. AHP estimates the appropriate values for the parameters in the absolute weight restrictions [18], relative weight restrictions [19,20,21,22,23], virtual weight restrictions [24,25] and restrictions on changes of input (output) units [26].

There are a number of other methods that do not necessarily apply additional restrictions to a DEA model. Such as converting the qualitative data in DEA to the quantitative data using AHP [27,28,29,30,31,32,33,34], ranking the efficient/inefficient units in DEA models using AHP in a two stage process [35,36,37], weighting the efficiency scores obtained from DEA using AHP [38], weighting the inputs and outputs in the DEA structure [39,40,41,42], constructing a convex combination of weights using AHP and DEA [43] and estimating missing data in DEA using AHP [44].

The recent studies by Pakkar [45,46,47,48,49,50] demonstrate the effects of imposing weight bounds on the different variants of DEA models using AHP. To this end, AHP has been applied in single-level DEA models [47,48,49,50] and two-level DEA models [45,46]. Due to the complexity of the hierarchical structures of indicators, this paper applies AHP into an additive three-level DEA model in the context of CI construction. Theoretically, the approach proposed in this paper may also be considered as the additive form of the multiplicative three-level DEA-based CI approach to constructing CIs proposed by [51]. In a three-level hierarchy, the indicators of similar characteristics can be grouped into sub-categories and further into categories. A three-level DEA model entirely reflects the characteristics of a generalized multiple level DEA model developed in [52,53]. Since the proposed approach uses AHP in an additive three-level DEA-based model, it contributes to the set of methods currently available for CI construction.

2. Methodology

This research has been organized to proceed along the following stages (Figure 1):

• Computing the composite value of each DMU using one-level DEA-based CI model (4). The computed composite values are applied in three-level DEA-based CI model (6).
• Computing the priority weights of indicators for all DMUs using AHP, which impose weight bounds into model (6).
• Obtaining an optimal set of weights for each DMU using three-level DEA-based CI model (6) (minimum composite loss η).
• Obtaining an optimal set of weights for each DMU using model (6) bounded by AHP (maximum composite loss κ). Note that if the AHP weights are added to model (6), we obtain model (10).
• Measuring the performance of each DMU in terms of the relative closeness to the priority weights of indicators. For this purpose, we develop parameter-distance model (11). Increasing a parameter in a defined range of composite loss we explore how much a DM can achieve its goals. This may result in various ranking positions for a DMU in comparison to the other DMUs.

Figure 1. A hierarchical aggregation approach for indicators using a three-level Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP).

Figure 1. A hierarchical aggregation approach for indicators using a three-level Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP).

2.1. DEA-Based CI Model

A DEA-based CI model can be formulated similar to a classical DEA model in which all data are treated as outputs without explicit inputs [54]. In the following, and in line with the more common CI terminology, we will often refer to outputs as “indicators”. In order to eliminate the scale differences between all (output) indicators, and moreover, to ensure that all of them are in the same direction of change the normalized counterparts of indicators, using the distance to reference method, are computed as follows [1]:

$$y_{rj}=\frac{{\widehat{y}}_{rj}}{{\widehat{y}}_{r(\max )}},\hspace{1em}{\widehat{y}}_{r(\max )}=\max \{{\widehat{y}}_{r1},{\widehat{y}}_{r2},\mathrm{...},{\widehat{y}}_{rn}\}\hspace{1em}\text{for desirable indicators}$$

(1)

$$y_{rj}=\frac{{\widehat{y}}_{r(\min )}}{{\widehat{y}}_{rj}},\hspace{1em}{\widehat{y}}_{r(\min )}=\min \{{\widehat{y}}_{r1},{\widehat{y}}_{r2},\mathrm{...},{\widehat{y}}_{rn}\}\text{\hspace{1em}for undesirable indicators}$$

(2)

where $y_{rj}$ is the normalized value of (output) indicator $r$ ($r=1,\mathrm{2..},s$) for DMU $j$ (j = 1, 2,…, n). Now assume that all DMUs have unit input $i$ ($i=1,2,\mathrm{...},m$). Then the fractional CCR-DEA model can be developed as follows [55]:

$$\text{Max}\hspace{1em}C{I}_k=\frac{{\displaystyle \sum _{r=1}^s{u}_r{y}_{rk}}}{{\displaystyle \sum _{i=1}^m{v}_i}}$$

(3)

$$\frac{{\displaystyle \sum _{r=1}^s{u}_r{y}_{rj}}}{{\displaystyle \sum _{i=1}^m{v}_i}}\le 1\hspace{1em}\forall j$$

$$u_r,v_i>0\hspace{1em}\forall r,i$$

where $C{I}_k$ is the composite indicator of DMU under assessment. $k$ is the index for the DMU under assessment where $k$ ranges over 1, 2,…, $n$. $v_i$ and $u_r$ are the weights of input i ($i=1,\hspace{0.17em}2,\mathrm{...},m$) and (output) indicator r ($r=1,\mathrm{2..},s$). The first set of constraints assures that if the computed weights are applied to a group of $n$ DMUs, ($j=1,2,\mathrm{...},n$), they do not attain a composite score of larger than 1. The second set of constraints indicates the non-negative conditions for the model variables. Introducing the constraint ${\displaystyle \sum _{i=1}^m{v}_i}=1$ and performing the operation of substitution, an equivalent linear model can be formulated as follows:

$$\text{Max}\hspace{1em}C{I}_k={\displaystyle \sum _{r=1}^s{u}_r{y}_{rk}}$$

(4)

$${\displaystyle \sum _{r=1}^s{u}_r{y}_{rj}}\le 1\forall j$$

$$u_r>0\forall r.$$

Model (4) looks like a DEA model without inputs that extends the standard DEA methodology to the field of CI construction.

2.2. Three-Level DEA-Based CI Model

We develop a three-level DEA model to aggregate the performance of indicators under the (sub) category they belong to by a weighted-average method (Figure 2). Let $y_{l{l}^{\prime }rj}$ be the value of indicator $r$ ($r=1,2,\mathrm{...},s$) of sub-category $l^{\prime }$ $(l^{\prime }=1,2,\mathrm{...},S^{\prime })$ of category $l$ $(l=1,2,\mathrm{...},S)$ for DMU $j$ ($j=1,2,\mathrm{...},n$) after normalizing the original data. Let $u_{l{l}^{\prime }r}$ be the internal weight of indicator $r$ of sub-category $l^{\prime }$ of category $l$ while ${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}=1}$. Then the value of sub-category $l^{\prime }$ of category $l$ for the DMU ${}_j$ is defined as $y_{l{l}^{\prime }j}={\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}{y}_{l{l}^{\prime }rj}}$. Let $p_{l{l}^{\prime }}$ be the internal weight of sub-category $l^{\prime }$ of category $l$ while ${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{p}_{l{l}^{\prime }}=1}$. Then the value of category $l$ is defined as $y_{lj}={\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{p}_{l{l}^{\prime }}{y}_{l{l}^{\prime }j}}$. Let $p_l$ be the weight of category $l$. To develop a linear model, the new multiplier of indicator $r$ of sub-category $l^{\prime }$ of category $l$ is defined as: $u_{l{l}^{\prime }r}^{\prime }=p_l{p}_{l{l}^{\prime }}{u}_{l{l}^{\prime }r}$. Similarly, the new multiplier of sub-category $l^{\prime }$ of category $l$ is defined as: ${p^{\prime }}_{l{l}^{\prime }}=p_l{p}_{l{l}^{\prime }}$. Consequently, a linear three-level DEA model for indicators can be developed as follows:

$$\text{Max\hspace{1em}}C{I}_k={\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rk}}}}$$

(5)

$${\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rj}}}}\le 1\forall j$$

$${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}\forall l$$

$${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}\hspace{1em}\forall l,l^{\prime }$$

$$u_{l{l}^{\prime }r}^{\prime },p_{l{l}^{\prime }}^{\prime },p_l>0\forall l,l^{\prime },r.$$

Figure 2. A three-level DEA framework for hierarchical indicators.

Figure 2. A three-level DEA framework for hierarchical indicators.

We develop our formulation based on the generalized distance model [56,57] in such a way that the hierarchical structures of indicators, using a weighted-average approach, are taken into consideration [52,53]. Let $C{I}_k^{*}$ ($k=1,2,\mathrm{...},n$) be the best attainable composite value for the DMU under assessment, calculated from model (4). We want the composite value $C{I}_k(u_{l{l}^{\prime }r}^{\prime })$, calculated from the set of weights $u_{l{l}^{\prime }r}^{\prime }$, to be closest to $C{I}_k^{*}$. The degree of closeness between $C{I}_k(u_{l{l}^{\prime }r}^{\prime })$ and $C{I}_k^{*}$ is measured as $D_t={[C{I}_k^{*}-C{I}_k{(u_{l{l}^{\prime }r}^{\prime })}^t]}^{1/t}$ with $t\ge 1$, where $D_t$ is a distance measure and $t$ represents the distance parameter. Our definition of “closest” is that the largest distance is at its minimum. On the other hand, the largest distance completely dominates when $t=\infty$. For $t=\infty$, the distance measure is reduced to $D_{\infty }=\underset{u_{l{l}^{\prime }r}^{\prime }}{\max }\{C{I}_k^{*}-C{I}_k(u_{l{l}^{\prime }r}^{\prime })\}$. Hence we choose the form of the minimax model: $\underset{u_{l{l}^{\prime }r}^{\prime }}{\min }\underset{k=1,\mathrm{...},n}{\max }\{C{I}_k^{*}-C{I}_k(u_{l{l}^{\prime }r}^{\prime })\}$ to minimize a single deviation which is equivalent to the following linear model:

$$\text{Min\hspace{1em}}\eta$$

(6)

$$C{I}_k^{*}-{\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rk}\le \eta }}}$$

$${\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rj}}}}\le C{I}_j^{*}\forall j$$

$${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}\hspace{1em}\forall l$$

$${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}\hspace{1em}\forall l,l^{\prime }$$

$$u_{l{l}^{\prime }r}^{\prime },p_{l{l}^{\prime }}^{\prime },p_l>0\forall l,l^{\prime },r$$

$$\eta \ge 0.$$

Model (6) identifies the minimum composite loss $\eta$ (eta) needed to arrive at an optimal set of weights. The first constraint ensures that each DMU loses no more than $\eta$ of its best attainable composite value, $C{I}_k^{*}$. The second set of constraints satisfies that the composite values of all DMUs are less than or equal to their upper bound of $C{I}_j^{*}$.

Two sets of constraints are added to model (6): ${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}$ and ${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}$, where $u_{l{l}^{\prime }r}^{\prime }$ are indicator multipliers. This implies that the sum of weights under each (sub-) sub-category equals to the weight of that (sub-) sub-category. It should be noted that the original (or internal) weights used for calculating the weighted averages are obtained as $u_{l{l}^{\prime }r}=u_{l{l}^{\prime }r}^{\prime }/p_{l{l}^{\prime }}^{\prime }$ and $p_{l{l}^{\prime }}=p_{l{l}^{\prime }}^{\prime }/p_l$.

2.3. Prioritizing Indicator Weights Using AHP

Model (6) identifies the minimum composite loss $\eta$ (eta) needed to arrive at a set of weights of indicators by the internal mechanism of DEA. On the other hand, the priority weights of indicators, and the corresponding (sub) categories are defined out of the internal mechanism of DEA by AHP (Figure 3).

Figure 3. The AHP model for prioritizing indicators.

Figure 3. The AHP model for prioritizing indicators.

In order to more clearly demonstrate how AHP is integrated into the three-level DEA-based CI model, this research presents an analytical process in which indicator weights are bounded by the AHP method. The AHP procedure for imposing weight bounds may be broken down into the following steps:

Step 1: A decision maker makes a pairwise comparison matrix of different criteria, denoted by $A$, with the entries of $a_{lq}$ $(l=q=1,2,\mathrm{...},S)$. The comparative importance of criteria is provided by the decision maker using a rating scale. Saaty [16] recommends using a 1–9 scale.

Step 2: The AHP method obtains the priority weights of criteria by computing the eigenvector of matrix $A$ (Equation (7)), $w={(w_1,w_2,\mathrm{...},w_S)}^T$, which is related to the largest eigenvalue, ${\lambda }_{\max }$.

$$Aw={\lambda }_{\max }w$$

(7)

To determine whether or not the inconsistency in a comparison matrix is reasonable the random consistency ratio, $C.R.$, can be computed by the following equation:

$$C.R.=\frac{{\lambda }_{\max }-N}{(N-1)R.I.}$$

(8)

where $R.I.$ is the average random consistency index and $N$ is the size of a comparison matrix. In a similar way, the priority weights of (sub-) sub-criteria under each (sub-) criterion can be computed. To obtain the weight bounds for indicator weights in the three-level DEA-based CI model, this study aggregates the priority weights of three different levels in AHP as follows:

$${\overline{u}}_{l{l}^{\prime }r}=w_l{e}_{l{l}^{\prime }}{f}_{l{l}^{\prime }r},\hspace{1em}{\displaystyle \sum _{l=1}^S{w}_l=1},\hspace{1em}{\displaystyle \sum _{l^{\prime }=1}^{s^{\prime }}{e}_{l{l}^{\prime }}=1}\text{\hspace{1em}and\hspace{1em}}{\displaystyle \sum _{r=1}^s{f}_{l{l}^{\prime }r}=1}$$

(9)

where $w_l$ is the priority weight of criterion $l$ ($l=1,\mathrm{...},S$) in AHP, $e_{l{l}^{\prime }}$ is the priority weight of sub-criterion $l^{\prime }$ $(l^{\prime }=1,2,\mathrm{...},S^{\prime })$ under criterion $l$ and $f_{l{l}^{\prime }r}$ is sub-sub-criterion $r$ ($r=1,\mathrm{...},s$) under sub-criterion $l^{\prime }$.

In order to estimate the maximum composite loss $\kappa$ (kappa) necessary to achieve the priority weights of indicators for each DMU the following linear program is proposed:

$$\text{Min\hspace{1em}}\kappa$$

(10)

$$u_{l{l}^{\prime }r}^{\prime }=\alpha {\overline{u}}_{l{l}^{\prime }r}\forall l,l^{\prime },r$$

$$C{I}_k^{*}-{\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rk}\le \kappa }}}$$

$${\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rj}}}}\le C{I}_j^{*}\hspace{1em}\forall j$$

$${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}\hspace{1em}\forall l$$

$${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}\hspace{1em}\forall l,l^{\prime }$$

$$u_{l{l}^{\prime }r}^{\prime },p_{l{l}^{\prime }}^{\prime },p_l>0\hspace{1em}\forall l,l^{\prime },r$$

$$\kappa \ge 0,\hspace{1em}\alpha >0.$$

The first sets of constraints change the AHP computed weights to weights for the new system by means of a scaling factor $\alpha$. The scaling factor $\alpha$ is added to avoid the possibility of contradicting constraints leading to infeasibility or underestimating the relative composite scores of DMUs [58]. The optimal solution to model (10) produces a set of weights for indicators that are used to compute the performance of DMUs.

It should be noted that incorporating absolute weight bounds, using AHP, for indicator weights in a DEA-based CI model is consistent with the common practice of constructing composite indicators. According to this practice, the priority weights of indicators can be used directly in an aggregation function to synthetize indicators’ values into composite values [1]. In addition, this form of placing restrictions on indicator weights simply allows us to identify a specific range of variation between two systems of weights obtained from models (6) and (10).

2.4. A Parametric Distance Model

We can now develop a parametric distance model for various discrete values of parameter $\theta$ such that $\eta \le \theta \le \kappa$. Let $u_{l{l}^{\prime }r}^{\prime }(\theta )$ be the weights of indicators for a given value of parameter $\theta$, where indicators are under sub-category $l^{\prime }$ $(l^{\prime }=1,2,\mathrm{...},S^{\prime })$ of category $l$ ($l=1,2,\mathrm{...},S$). Let ${u^{\prime }}_{l{l}^{\prime }r}^{*}(\kappa )$ be the priority weights of indicators under sub-category $l^{\prime }$ of category $l$, obtained from model (10). Our objective is to minimize the total deviations between $u_{l{l}^{\prime }r}^{\prime }(\theta )$ and ${u^{\prime }}_{l{l}^{\prime }r}^{*}(\kappa )$ with the shortest Euclidian distance measure subject to the following constraints:

$$\text{Min\hspace{1em}}{Z}_k(\theta )={\left({\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{(u_{l{l}^{\prime }r}^{\prime }-{u^{\prime }}_{l{l}^{\prime }r}^{*}(\kappa ))}^2}}}\right)}^{1/2}$$

(11)

$$C{I}_k^{*}-{\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rk}\le \theta }}}$$

$${\displaystyle \sum _{l=1}^S{\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }{y}_{l{l}^{\prime }rk}\le \theta }}}\le C{I}_j^{*}\hspace{1em}\forall j$$

$${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}\hspace{1em}\forall l$$

$${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}\hspace{1em}\forall l,l^{\prime }$$

$$u_{l{l}^{\prime }r}^{\prime },p_{l{l}^{\prime }}^{\prime },p_l>0\hspace{1em}\forall l,l^{\prime },r.$$

Because the range of deviations computed by the objective function is different for each DMU, it is necessary to normalize it by using relative deviations rather than absolute ones [59]. Hence, the normalized deviations can be computed by:

$${\Delta }_k(\theta )=\frac{Z_k^{*}(\eta )-Z_k^{*}(\theta )}{Z_k^{*}(\eta )}$$

(12)

where $Z_k^{*}(\theta )$ is the optimal value of the objective function for $\eta \le \theta \le \kappa$. We define ${\Delta }_k(\theta )$ as a measure of closeness which represents the relative closeness of each DMU to the weights obtained from model (10) in the range [0, 1]. Increasing the parameter $(\theta )$, we improve the deviations between the two systems of weights obtained from models (6) and (10) which may lead to different ranking positions for each DMU in comparison to the other DMUs. It should be noted that in a special case where the parameter $\theta =\kappa =0$, we assume ${\Delta }_k(\theta )$ = 1.

3. A Numerical Example: Road Safety Performance Indicators

In this section we present the application of the proposed approach to assess the road safety performance of a set of 13 European countries (or DMUs): Austria (AUT), Belgium (BEL), Finland (FIN), France (FRA), Hungary (HUN), Ireland (IRL), Lithuania (LTU), Netherlands (NLD), Poland (POL), Portugal (PRT), Slovenia (SVN), Sweden (SWE) and Switzerland (CHE). The data for eleven hierarchical indicators that compose SPIs for these countries have been adopted from [52]. The eight SPIs related to alcohol and speed are undesirable indicators while the three SPIs related to protective systems are desirable ones. The resulting normalized data based on Equations (1) and (2) are presented in Table 1.

Taking the percentage of speed limit violation on rural roads as an example, Slovenia performs the best (1.000) while Poland the worst (0.014) and all other countries’ values lie within this interval.

The results of the AHP model for prioritizing hierarchical SPIs as constructed by the author in Expert Choice software are presented in Table 2. One can argue that the priority weights of SPIs must be judged by road safety experts. However, since the aim of this section is just to show the application of the proposed approach on numerical data, we see no problem to use our judgment alone.

Table 1. Normalized data on the eleven hierarchical safety performance indicators (SPIs).

Countries	Alcohol		Speed						Protective Systems
	% of Drivers above Legal Alcohol Limit In Roadside Police Tests	% of Alcohol Related Fatalities	Mean Speed			Speed Limit Violation (%)			Seat Belt		Child Restraint
			Motorways	Rural Roads	Urban Roads	Motorways	Rural Roads	Urban Roads	Daytime Seatbelt Wearing Rate in Front Seats of Light Vehicles (%)	Daytime Seatbelt Wearing Rate in Rear Seats of Light Vehicles (%)	Daytime Usage Rate of Child Restraints (%)
AUT	0.116	0.463	0.938	0.781	0.802	0.766	0.051	0.254	0.904	0.699	0.863
BEL	0.068	0.654	0.846	0.743	0.768	0.348	0.029	0.222	0.799	0.488	0.729
FIN	0.593	0.136	0.963	0.729	0.907	0.409	0.023	0.323	0.911	0.922	0.716
FRA	0.263	0.123	0.933	0.787	0.838	0.505	0.037	0.318	1.000	1.000	0.937
HUN	0.279	0.283	0.955	0.793	0.817	0.362	0.033	0.230	0.727	0.501	0.433
IRL	0.237	0.119	0.945	0.762	0.724	1.000	0.032	0.223	0.901	0.914	0.857
LTU	0.555	0.321	1.000	0.713	0.714	0789	0.025	0.318	0.609	0.366	0.404
NLD	0.081	1.000	0.899	0.740	0.881	0.454	0.020	0.234	0.959	0.890	0.758
POL	0.091	0.438	0.806	0.697	0.647	0.290	0.015	0.165	0.799	0.589	0.905
PRT	0.137	0.610	0.847	0.618	0.919	0.302	0.014	0.360	0.881	0.574	0.591
SVN	0.122	0.078	0.964	1.000	0.713	0.480	1.000	0.163	0.874	0.551	0.672
SWE	1.000	0.357	0.883	0.717	0.870	0.241	0.019	0.259	0.973	0.927	1.000
CHE	0.277	0.230	0.943	0.757	1.000	0.710	0.043	1.000	0.887	0.805	0.895

Table 1. Normalized data on the eleven hierarchical safety performance indicators (SPIs).

Countries	Alcohol		Speed						Protective Systems
	% of Drivers above Legal Alcohol Limit In Roadside Police Tests	% of Alcohol Related Fatalities	Mean Speed			Speed Limit Violation (%)			Seat Belt		Child Restraint
			Motorways	Rural Roads	Urban Roads	Motorways	Rural Roads	Urban Roads	Daytime Seatbelt Wearing Rate in Front Seats of Light Vehicles (%)	Daytime Seatbelt Wearing Rate in Rear Seats of Light Vehicles (%)	Daytime Usage Rate of Child Restraints (%)
AUT	0.116	0.463	0.938	0.781	0.802	0.766	0.051	0.254	0.904	0.699	0.863
BEL	0.068	0.654	0.846	0.743	0.768	0.348	0.029	0.222	0.799	0.488	0.729
FIN	0.593	0.136	0.963	0.729	0.907	0.409	0.023	0.323	0.911	0.922	0.716
FRA	0.263	0.123	0.933	0.787	0.838	0.505	0.037	0.318	1.000	1.000	0.937
HUN	0.279	0.283	0.955	0.793	0.817	0.362	0.033	0.230	0.727	0.501	0.433
IRL	0.237	0.119	0.945	0.762	0.724	1.000	0.032	0.223	0.901	0.914	0.857
LTU	0.555	0.321	1.000	0.713	0.714	0789	0.025	0.318	0.609	0.366	0.404
NLD	0.081	1.000	0.899	0.740	0.881	0.454	0.020	0.234	0.959	0.890	0.758
POL	0.091	0.438	0.806	0.697	0.647	0.290	0.015	0.165	0.799	0.589	0.905
PRT	0.137	0.610	0.847	0.618	0.919	0.302	0.014	0.360	0.881	0.574	0.591
SVN	0.122	0.078	0.964	1.000	0.713	0.480	1.000	0.163	0.874	0.551	0.672
SWE	1.000	0.357	0.883	0.717	0.870	0.241	0.019	0.259	0.973	0.927	1.000
CHE	0.277	0.230	0.943	0.757	1.000	0.710	0.043	1.000	0.887	0.805	0.895

Table 2. The AHP hierarchical model for SPIs.

Objective Level	Criteria Level	Sub-Criteria Level	Sub-Sub-Criteria Level
Prioritizing road user behavior	Alcohol $w_1=$ 0.2727	% of drivers above legal alcohol limit $e_{11}=$ 0.333	% of drivers above legal alcohol limit $f_{111}=$ 1.000
		% of alcohol-related fatalities $e_{12}=$ 0.667	% of alcohol-related fatalities $f_{121}=$ 1.000
	Speed $w_2=$ 0.5454	Mean speed $e_{21}=$ 0.60	Mean speed of vehicles on motorways $f_{211}=$ 0.081
			Mean speed of vehicles on rural roads, $f_{212}=$ 0.342
			Mean speed of vehicles on urban roads, $f_{213}=$ 0.577
		Speed limit violations $e_{22}=$ 0.40	% of vehicles exceeding the speed limit on motorways $f_{221}=$ 0.081
			% of vehicles exceeding the speed limit on rural roads $f_{222}=$ 0.342
			% of vehicles exceeding the speed limit on urban roads $f_{223}=$ 0.577
	Protective systems $w_3=$ 0.1818	Seat belt $e_{31}=$ 0.40	Daytime seatbelt wearing rate in front seats of light vehicles (%) $f_{311}=$ 0.60
			Daytime seatbelt wearing rate in rear seats of light vehicles (%) $f_{312}=$ 0.40
		Child Restraint $e_{32}=$ 0.60	Daytime usage rate of child restraints (%) $f_{321}=$ 1.000

Table 2. The AHP hierarchical model for SPIs.

Objective Level	Criteria Level	Sub-Criteria Level	Sub-Sub-Criteria Level
Prioritizing road user behavior	Alcohol $w_1=$ 0.2727	% of drivers above legal alcohol limit $e_{11}=$ 0.333	% of drivers above legal alcohol limit $f_{111}=$ 1.000
		% of alcohol-related fatalities $e_{12}=$ 0.667	% of alcohol-related fatalities $f_{121}=$ 1.000
	Speed $w_2=$ 0.5454	Mean speed $e_{21}=$ 0.60	Mean speed of vehicles on motorways $f_{211}=$ 0.081
			Mean speed of vehicles on rural roads, $f_{212}=$ 0.342
			Mean speed of vehicles on urban roads, $f_{213}=$ 0.577
		Speed limit violations $e_{22}=$ 0.40	% of vehicles exceeding the speed limit on motorways $f_{221}=$ 0.081
			% of vehicles exceeding the speed limit on rural roads $f_{222}=$ 0.342
			% of vehicles exceeding the speed limit on urban roads $f_{223}=$ 0.577
	Protective systems $w_3=$ 0.1818	Seat belt $e_{31}=$ 0.40	Daytime seatbelt wearing rate in front seats of light vehicles (%) $f_{311}=$ 0.60
			Daytime seatbelt wearing rate in rear seats of light vehicles (%) $f_{312}=$ 0.40
		Child Restraint $e_{32}=$ 0.60	Daytime usage rate of child restraints (%) $f_{321}=$ 1.000

Solving model (6) for the country under assessment, we obtain an optimal set of weights with minimum composite loss $(\eta )$. Since the raw data are normalized, the weights obtained from this model are meaningful and have an intuitive explanation. As a result, we can later set meaningful bounds on the weights in terms of the relative priority of indicators. Taking Austria as an example in Table 3, with the composite value of one obtained from model (4), we can observe that the alcohol-related fatality rate is 26 times less important than the mean speed of vehicles on motorways and about 3.2 times less important than the daytime usage rate of child restraints. Clearly, the other indicators are ignored in this assessment by assigning zero weights which are equivalent to excluding those indicators from the analysis. This kind of situation can be remedied by including the opinion of experts in defining the relative priority of indicators.

Table 3. Optimal weights of hierarchical SPIs obtained from model (6) for Austria.

Weights of Categories	Weights of Sub-Categories	Weights of Sub-Sub-Categories
$p_1=$ 0.0361	$p_{11}^{\prime }=$ 0.0000	$u_{111}^{\prime }=$ 0.0000
	$p_{12}^{\prime }=$ 0.0361	$u_{121}^{\prime }=$ 0.0361
$p_2=$ 0.9415	$p_{21}^{\prime }=$ 0.9415	$u_{211}^{\prime }=$ 0.9415
		$u_{212}^{\prime }=$ 0.0000
		$u_{213}^{\prime }=$ 0.0000
	$p_{22}^{\prime }=$ 0.0000	$u_{221}^{\prime }=$ 0.0000
		$u_{222}^{\prime }=$ 0.0000
		$u_{223}^{\prime }=$ 0.0000
$p_3=$ 0.1160	$p_{31}^{\prime }=$ 0.0000	$u_{311}^{\prime }=$ 0.0000
		$u_{312}^{\prime }=$ 0.0000
	$p_{32}^{\prime }=$ 0.1160	$u_{321}^{\prime }=$ 0.1160
$\eta =0.0000$

Table 3. Optimal weights of hierarchical SPIs obtained from model (6) for Austria.

Weights of Categories	Weights of Sub-Categories	Weights of Sub-Sub-Categories
$p_1=$ 0.0361	$p_{11}^{\prime }=$ 0.0000	$u_{111}^{\prime }=$ 0.0000
	$p_{12}^{\prime }=$ 0.0361	$u_{121}^{\prime }=$ 0.0361
$p_2=$ 0.9415	$p_{21}^{\prime }=$ 0.9415	$u_{211}^{\prime }=$ 0.9415
		$u_{212}^{\prime }=$ 0.0000
		$u_{213}^{\prime }=$ 0.0000
	$p_{22}^{\prime }=$ 0.0000	$u_{221}^{\prime }=$ 0.0000
		$u_{222}^{\prime }=$ 0.0000
		$u_{223}^{\prime }=$ 0.0000
$p_3=$ 0.1160	$p_{31}^{\prime }=$ 0.0000	$u_{311}^{\prime }=$ 0.0000
		$u_{312}^{\prime }=$ 0.0000
	$p_{32}^{\prime }=$ 0.1160	$u_{321}^{\prime }=$ 0.1160
$\eta =0.0000$

It should be noted that the composite value of all countries calculated from model (6) is identical to that calculated from model (4). Therefore, the minimum composite loss for the country under assessment is $\eta =0$ (Table 4). This implies that the measure of relative closeness to the AHP weights for the country under assessment is ${\Delta }_k(\eta )=0$. On the other hand, solving model (10) for the country under assessment, we adjust the priority weights of hierarchical SPIs obtained from AHP in such a way that they become compatible with the weights’ structure in the three level DEA-based CI models. Table 5 presents the optimal weights of hierarchical SPIs as well as its scaling factor for all countries.

Table 4. Minimum and maximum losses in composite values for each country.

Countries	$C{I_k}^{*}$	η	κ
AUT	1.000	0.000	0.158
BEL	0.938	0.000	0.127
FIN	1.000	0.000	0.173
FRA	1.000	0.000	0.184
HUN	1.000	0.000	0.284
IRL	1.000	0.000	0.254
LTU	1.000	0.000	0.270
NLD	1.000	0.000	0.023
POL	0.955	0.000	0.224
PRT	0.978	0.000	0.139
SVN	1.000	0.000	0.208
SWE	1.000	0.000	0.040
CHE	1.000	0.000	0.000

Table 4. Minimum and maximum losses in composite values for each country.

Countries	$C{I_k}^{*}$	η	κ
AUT	1.000	0.000	0.158
BEL	0.938	0.000	0.127
FIN	1.000	0.000	0.173
FRA	1.000	0.000	0.184
HUN	1.000	0.000	0.284
IRL	1.000	0.000	0.254
LTU	1.000	0.000	0.270
NLD	1.000	0.000	0.023
POL	0.955	0.000	0.224
PRT	0.978	0.000	0.139
SVN	1.000	0.000	0.208
SWE	1.000	0.000	0.040
CHE	1.000	0.000	0.000

Table 5. Optimal weights of hierarchical SPIs obtained from model (10) for all countries.

Weights of Categories	Weights of Sub-Categories	Weights of Sub-Sub-Categories
$p_1=$ 0.4089	$p_{11}^{\prime }=$ 0.1362	$u_{111}^{\prime }=$ 0.1362
	$p_{12}^{\prime }=$ 0.2727	$u_{121}^{\prime }=$ 0.2727
$p_2=$ 0.8178	$p_{21}^{\prime }=$ 0.4907	$u_{211}^{\prime }=$ 0.0397
		$u_{212}^{\prime }=$ 0.1678
		$u_{213}^{\prime }=$ 0.2831
	$p_{22}^{\prime }=$ 0.3271	$u_{221}^{\prime }=$ 0.0265
		$u_{222}^{\prime }=$ 0.1119
		$u_{223}^{\prime }=$ 0.1888
$p_3=$ 0.2726	$p_{31}^{\prime }=$ 0.1090	$u_{311}^{\prime }=$ 0.0654
		$u_{312}^{\prime }=$ 0.0436
	$p_{32}^{\prime }=$ 0.1636	$u_{321}^{\prime }=$ 0.1636
$\alpha =$ 1.4993

Table 5. Optimal weights of hierarchical SPIs obtained from model (10) for all countries.

Weights of Categories	Weights of Sub-Categories	Weights of Sub-Sub-Categories
$p_1=$ 0.4089	$p_{11}^{\prime }=$ 0.1362	$u_{111}^{\prime }=$ 0.1362
	$p_{12}^{\prime }=$ 0.2727	$u_{121}^{\prime }=$ 0.2727
$p_2=$ 0.8178	$p_{21}^{\prime }=$ 0.4907	$u_{211}^{\prime }=$ 0.0397
		$u_{212}^{\prime }=$ 0.1678
		$u_{213}^{\prime }=$ 0.2831
	$p_{22}^{\prime }=$ 0.3271	$u_{221}^{\prime }=$ 0.0265
		$u_{222}^{\prime }=$ 0.1119
		$u_{223}^{\prime }=$ 0.1888
$p_3=$ 0.2726	$p_{31}^{\prime }=$ 0.1090	$u_{311}^{\prime }=$ 0.0654
		$u_{312}^{\prime }=$ 0.0436
	$p_{32}^{\prime }=$ 0.1636	$u_{321}^{\prime }=$ 0.1636
$\alpha =$ 1.4993

Note that the priority weights of AHP used for incorporating weight bounds on indicator weights in model (10) are obtained as ${\overline{u}}_{l{l}^{\prime }r}=\frac{u_{l{l}^{\prime }r}^{\prime }}{\alpha }$. Similarly, the priority weights of AHP at criteria level can be obtained as $w_l=\frac{p_l}{\alpha }$ while ${\displaystyle \sum _{l^{\prime }=1}^{S^{\prime }}{\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_l}}$ and ${\displaystyle \sum _{r=1}^s{u}_{l{l}^{\prime }r}^{\prime }=p_{l{l}^{\prime }}^{\prime }}$. In addition, The priority weights of AHP at sub-criteria and sub-sub-criteria levels can be obtained as $e_{l{l}^{\prime }}=p_{l{l}^{\prime }}^{\prime }/p_l$ and $f_{l{l}^{\prime }r}=u_{l{l}^{\prime }r}^{\prime }/p_{l{l}^{\prime }}^{\prime }$, respectively.

The maximum composite loss for each country to achieve the corresponding weights in model (10) is equal to $\kappa$ (Table 4). As a result, the measure of relative closeness to the priority weights of SPIs for the country under assessment is ${\Delta }_k(\kappa )$ = 1. Going one step further to the solution process of the parametric distance model (11), we proceed to the estimation of total deviations from the AHP weights for each country while the parameter $\theta$ is $0\le \theta \le \kappa$. Table 6 represents the ranking position of each country based on the minimum deviation from the priority weights of indicators for $\theta =0$. It should be noted that in a special case where the parameter $\theta =\kappa =0$ we assume ${\Delta }_k(\theta )=1.$

Table 6. The ranking position of each country based on the minimum distance to priority weights of SPIs.

Countries	$Z^{*}(\eta )$	Rank
AUT	0.332	6
BEL	0.705	11
FIN	0.438	9
FRA	0.338	7
HUN	0.591	10
IRL	0.331	5
LTU	0.381	8
NLD	0.021	2
POL	0.718	12
PRT	0.782	13
SVN	0.166	4
SWE	0.035	3
CHE	0.000	1

Table 6. The ranking position of each country based on the minimum distance to priority weights of SPIs.

Countries	$Z^{*}(\eta )$	Rank
AUT	0.332	6
BEL	0.705	11
FIN	0.438	9
FRA	0.338	7
HUN	0.591	10
IRL	0.331	5
LTU	0.381	8
NLD	0.021	2
POL	0.718	12
PRT	0.782	13
SVN	0.166	4
SWE	0.035	3
CHE	0.000	1

Table 6 shows that Switzerland (CHE) is the best performer in terms of the $CI$ value and the relative closeness to the priority weights of indicators in comparison to the other countries. Nevertheless, increasing the value of $\theta$ from 0 to $\kappa$ has two main effects on the performance of the other countries: improving the degree of deviations and reducing the value of composite indicator. This, of course, is a phenomenon, one expects to observe frequently. The graph of $\Delta (\theta )$ versus $\theta$, as shown in Figure 4, is used to describe the relation between the relative closeness to the priority weights of indicators and composite loss for each country. This may result in different ranking positions for each country in comparison to the other countries (Appendix A).

Figure 4. The relative closeness to the priority weights of indicators [∆(θ)], versus composite loss (θ) for each country.

Figure 4. The relative closeness to the priority weights of indicators [∆(θ)], versus composite loss (θ) for each country.

In order to clearly discover the effect of composite loss on the countries’ ranking as shown in Appendix A, we performed a Kruskal-Wallis test. The Kruskal-Wallis test compares the medians of rankings to determine whether there is a significant difference between them. The result of the test reveals that its p-value is quite smaller than 0.01. Therefore, we conclude that increasing composite loss in the whole range [0.0, 0.29] changes the countries’ ranking significantly. Note that at $\theta =0$ the countries can be ranked based on $Z_k^{*}(0)$ from the closest to the furthest from the priority weights of SPIs. For instance, at $\theta =0$, Sweden, Ireland and Austria with composite values of one, are ranked in 3rd, 5th and 6th places, respectively, while Belgium, Poland and Portugal with composite values of less than one are ranked in 11th, 12th and 13th places, respectively (Table 4 and Table 6). However, with a small composite loss at $\theta =0.01$, Belgium, Poland and Portugal take 3rd, 6th and 5th places in the rankings, respectively. Using this example, as a guideline, it is relatively easy to rank the countries in terms of distance to the priority weights of SPIs. At $\theta =0.02$, Sweden moves up into 3rd place again while Belgium drops into 4th place. It is clear that both measures, $Z_k^{*}(0)$ and ${\Delta }_k(\theta )$, are necessary to explain the ranking position of a country.

4. Conclusions

We develop a hierarchical aggregation approach based on DEA and AHP methodologies to construct composite indicators. We define two sets of weights of indicators in a three-level DEA framework. All indicators are treated as benefit type. The first set represents the weights of indicators with minimum composite loss. The second set represents the corresponding priority weights of hierarchical indicators, using AHP, with maximum composite loss. We assess the performance of each DMU in comparison to the other DMUs based on the relative closeness of the first set of weights to the second set of weights. Improving the measure of relative closeness in a defined range of composite loss, we explore the various ranking positions for the DMU under assessment in comparison to the other DMUs. To demonstrate the effectiveness of the proposed approach, we apply it to construct a composite road safety performance index for eleven hierarchical indicators that compose SPIs for 13 European countries.