Undergraduate Teaching Audit and Evaluation Using an Extended ORESTE Method with Interval-Valued Hesitant Fuzzy Linguistic Sets

Abstract

Undergraduate teaching audit and evaluation (UTAE) plays a substantial role in the teaching quality assurance and monitoring of universities. It achieves the goal of selecting the best university for promoting the quality of higher education in China. Generally, the UTAE is a complex decision-making problem by considering competing evaluation criteria. Moreover, the evaluation information on the teaching quality of universities is often ambiguous and hesitant because of the vagueness existing in human judgments. Previous studies on UTAE have paid subtle attention towards the managing of linguistic expressions and the performance priority of universities. The interval-valued hesitant fuzzy linguistic sets (IVHFLSs) can effectively describe uncertainty, hesitancy, and inconsistency inherent in decision-making process. The ORESTE (organísation, rangement et Synthèse de données relarionnelles, in French) is a new outranking decision-making method which can show detailed distinctions between alternatives. Therefore, in this study, we propose a new UTAE approach based on the VHFLSs and ORESTE method to resolve the prioritization of universities for selecting the optimal university to benchmark. Specifically, the presented method handles the hesitant and uncertain linguistic expressions of experts by adopting the IVHFLSs and determines the ranking of universities with an extended ORESTE approach. Finally, a practical UTAE example illustrates the feasibility the proposed approach and a comparison analysis provides grounding for the superiority of the integrated approach. When the obtained results are evaluated, U₂ has been determined as the best university. The results indicate the good performance of the proposed UTAE approach in evaluating and improving the teaching quality of universities.

1. Introduction

In recent years, universities have experienced many changes and developed quality assurance and monitoring systems to warrant improvement in the quality of their teaching and research [1]. To enhance the teaching performance of colleges and universities, the Ministry of Education of China proposed a student-centered undergraduate teaching evaluation system [2,3]. The assessment of university teaching quality has become an especially important role in the modern university system. Undergraduate teaching audit and evaluation (UTAE) is a new type of evaluation system, which is crucial for a university to enhance teaching quality and its quality assurance system [4]. It achieves the goal of inspection by assessing whether participating universities are meeting the goals they set for themselves every five years [5,6,7]. Therefore, it is of great important for the university to improve the quality of teaching through UTAE.

Normally, the UTAE is accomplished by a team of experts with different experience and knowledge to express their thoughts on the evaluation of each university. In real situations, it is more reasonable and accurate for experts to express their judgements by using linguistic terms. Moreover, they may hesitate about their assessments of universities because of information inadequacy or experience restriction. As an extension of linguistic term sets [8] and interval-valued hesitant fuzzy sets [9], the interval-valued hesitant fuzzy linguistic sets (IVHFLSs) are a new approach put forward by Wang et al. [10] for tackling uncertain linguistic information. By combining quantitative and qualitative evaluations, the IVHFLSs are able to express the real favorites of decision makers and reflect their uncertainty, hesitancy, and inconsistency [11]. The method is able to express two fuzzy traits of an object, i.e., a linguistic term and an interval-valued hesitant fuzzy element (IVHFE) [12]. The first is an assessment value and the second is the hesitancy for the provided assessment value and represents the interval-valued membership degrees for a given linguistic term. In the past years, the IVHFLSs have been the focus of researchers and applied in a lot of fields to handle uncertain linguistic decision-making information [13,14,15]. Therefore, it is promising to use the IVHFLSs to describe university teaching quality assessment provided by experts.

On the other hand, UTAE is a typical multicriteria decision-making (MCDM) problem, which determines the optima university from a set of alternatives by considering multiple evaluation criteria [7,16,17]. The ORESTE, initiated by Roubens [18], is one of the most popular outranking decision methods for choosing the best alternative. It analyzes the differences between alternatives in terms of preference relations, indifference relations, and incomparability relations [19,20]. Furthermore, the ORESTE does not require the crisp weight values of criteria in which the thresholds are objectively computed with few subjective elements [21,22,23]. Since its appearance, the ORESTE has been employed to solve various decision-making problems, which include electric vehicle selection [24], failure mode risk analysis [25], regional economic restorability assessment [26], sustainable battery supplier evaluation [27], and new energy investment assessment [28]. Therefore, it is of significance to make use of the ORESTE approach for evaluating and selecting the best university in UTAE.

According to the above analysis, the research problems addressed in this study are how to effectively describe the uncertain and imprecise teaching quality assessments given by experts and how to acquire a more rational and reliable ranking of a given set of universities to obtain the optimum one for benchmarking. The prioritization of university alternatives is a complex multicriteria decision-making (MCDM) problem for the identification of universities’ performance orders based on competing evaluation criteria. Therefore, this paper aims to design a new approach UTAE by extending the ORESTE method in an interval-valued hesitant fuzzy linguistic environment. To sum up, this paper makes the following vital contributions: (1) the IVHFLSs are adopted to handle the uncertain and hesitant assessment data of experts on the universities participating in UTAE; (2) the classical ORESTE method has been modified for the priority of the considered universities and acquiring the top one for benchmarking; and, finally, (3) a practical example is offered to illustrate the application of the proposed UTAE approach and make a compared analysis to further show its effectiveness and benefits.

The remaining part of this article is organized as follows: Section 2 provides a literature review of previous literature related to UTAE. Section 3 provides the basic concepts and definitions of IVHFLSs. In Section 4, a new UTAE method based on the IVHFLSs and an extended ORESTE model is proposed. Then, a case study is conducted to demonstrate the proposed UTAE approach in Section 5. Finally, Section 6 concludes this study and gives potential future research directions.

2. Literature Review

2.1. Researches on Hesitant Fuzzy Linguistic Methods

The hesitant fuzzy linguistic term sets (HFLTSs) were proposed by Rodriguez et al. [29] to assess a linguistic variable by using several linguistic terms. Based on the HFLTSs, decision makers give their judgments by using linguistic expressions based on comparative terms. Since its introduction, this method has been applied in a lot of fields. For example, Yong et al. [30] proposed a prospects and barriers analysis model for the development of energy storage sharing, in which the HFLTSs were used to collect evaluation information. Wang et al. [31] introduced a large-scale group approach for determining the best sitting charging station location, in which the HFLTSs were adopted to improve the elicitation process. Hui and Kexin [32] employed a HFLTS-based DEMATEL method for the dynamic assessment of a post-pandemic agricultural traceability system. Krishankumar et al. [33] utilized a multi-hesitant fuzzy linguistic-based Choquet integral approach to assess the renewable energy sources for smart cities’ demand satisfaction. Furthermore, Finger and Lima-Junior [34] put forward a hesitant fuzzy linguistic quality function deployment (QFD) method for expressing sustainable supplier development programs, and Erol et al. [35] introduced an integrated decision framework using QFD and HFLTSs for reducing the impact of the obstacles to the adoption of the circular economy through blockchain.

The HFLSs have made great progress in handling linguistic assessment information. However, under some situations, experts’ assessments are usually indeterminate or fuzzy as the complexity of the problem increases and human thoughts become more vague [36,37]. For the real decision-making problems, experts tended to give the assessments of criteria values in the form of IVHFLSs [10]. The possible interval-valued membership degrees that an evaluation object may associate with a linguistic term are embodied in the IVHFLSs as opposed to HFLSs. The probable interval-valued membership degrees in IVHFLSs, which are brought on by the hesitation and uncertainty of decision makers, are represented by the interval values. Interval values are a good way to depict real-world decision-making issues since they make it easier to express imprecise information than specified or single real numbers.

Recently, Krishankumar et al. [38] proposed a double hierarchy hesitant fuzzy linguistic model for the personalized priority of sustainable suppliers. Krishankumar et al. [33] developed a multi-hesitant fuzzy linguistic-based Choquet integral approach to evaluate renewable energy sources for smart cities’ demand satisfaction. Wu and Liao [19] presented a hesitant fuzzy linguistic method for group decision making with indefinite criteria weights by taking account of incomparability between alternatives. Zhang et al. [39] introduced an ELECTRE II approach by using the cosine similarity for assessing the performance of financial logistics enterprises with double hierarchy hesitant fuzzy linguistic data. Shi et al. [21] proposed an engineering characteristics prioritization model based on double hierarchy hesitant linguistic term sets to overcome the shortcomings of the traditional quality function deployment approach.

2.2. Researches on ORESTE Improvement

In previous studies, various extensions of the ORESTE method have been proposed to solve MCDM issues in different environments. For instance, Wu and Liao [19] provided a hesitant fuzzy linguistic ORESTE approach for group decision making with uncertain criteria weights and incomparability between alternatives. Tian et al. [24] developed an extended ORESTE model based on hesitant intuitionistic fuzzy sets to help consumers make electric vehicle purchase choices. Zhang et al. [26] suggested an interval type-2 fuzzy ORESTE model to assess the regional economic restorability under the stress of COVID-19. Liu et al. [28] established a model by using the ORESTE to evaluate new energy investments based on the cloud model. In Ref. [25], the traditional ORESTE method is generalized to the extended comparative linguistic environment to rank the failure modes for electro-mechanical actuators. In Ref. [21], an amended ORESTE model with double hierarchy hesitant linguistic data was proposed for the engineering characteristics ordering in quality function deployment. In Ref. [27], a prospect theory-based distributed linguistic ORESTE technique was reported for the assessment of sustainable battery suppliers of new energy vehicles. In Ref. [40], a comprehensive approach was discussed, incorporating ORESTE and the social participatory allocation network under the q-rung orthopair fuzzy environment. Moreover, the double hierarchy hesitant fuzzy linguistic ORESTE method [41] was presented to evaluate traffic congestion and the likelihood-based hybrid ORESTE method [42] was given to evaluate the thermal comfort in underground mines. Pan et al. [43] proposed an interval type-2 fuzzy ORESTE model to select waste-to-energy plant location. In addition, the T-spherical fuzzy ORESTE [44], the Pythagorean fuzzy ORESTE [45], and the interval-valued spherical fuzzy-ORESTE [46] methods have been proposed for group decision-making.

2.3. Researches on UTAE

With the increasing attention to higher education largely conditioned by the demands of society, undergraduate teaching quality has become extremely vital in Chinese universities. The UTAE is designed as an evaluation system of “Five-in-One” structure to encourage the founding of a quality assurance system and enhance the quality of teaching for universities. The five respects of UTAE include teaching basic state data, college testing, professional certification, international evaluation, as well as government, schools, specialized agencies, and social multi-evaluation [5]. Over the years, many studies have been conducted on the topic of UTAE. Lalla et al. [47] explored three split-ballot experiments to the traditional ordinal scales for the evaluation of teaching activity, and proposed extended standard procedures of fuzzy systems. Chen et al. [48] presented a methodology to evaluate teaching performance by utilizing the analytic hierarchy process (AHP) and comprehensive evaluation technique. La Rocca et al. [49] developed an integrated strategy of analysis using descriptive elements and modeling for solving student evaluation problems of teaching data. Bas et al. [50] proposed the use of sensitivity analysis to acquire weights of composite indicators for measuring the teaching activity in a university. Jiang et al. [51] put forward a teaching quality assessment model based on the enhanced fuzzy neural network for college English. Peng and Dai [52] presented two algorithms by using the multiparametric similarity measure and combinative distance to assess classroom teaching quality with q-rung orthopair fuzzy data. In addition, Peng [5] developed two kinds of single-valued neutrosophic reducible weighted Maclaurin symmetric mean (MSM) operator to determine university ranking in UTAE. Gong et al. [1] reported a method which combines q-rung orthopair fuzzy sets and the multi-attribute border approximation area comparison (MABAC) for the priority of universities in UTAE.

Recently, Huang et al. [53] developed a model for the teaching quality assessment of Chinese-foreign cooperation in running schools from the perspective of education for sustainable development. Zhang [54] proposed an improved back propagation (BP) algorithm to measure the teaching quality of business English. Sun [55] offered a neural network to assess collegiate English education based on the BP network’s application principle. In Ref. [56], an English teaching quality assessment model based on Gaussian process machine learning was suggested. In Ref. [57], the data envelopment analysis method was used for the technical efficiency performance of the higher education systems.

The above literature reviews on UTAE help us identify the following potential research challenges. First, various models have been used in UTAE to handle the imprecise and uncertain performance assessments of universities. Nevertheless, because of the increasing complexity of practical UTAE problems, the experts’ assessment information is often qualitative and the current research cannot reflect their hesitancy and inconsistency. Second, some MCDM approaches have been used for the ranking of universities in UTAE. However, no study has been conducted to employ the ORESTE method to determine the ranking orders of the given universities. Motivated by these challenges, this paper attempts to put forward a new UTAE approach by integrating the IVHFLSs and an extended ORESTE method to assess and select the optimal university. The proposed IVHFLS-ORESTE approach is able to express the subjective evaluation data of experts more accurately and obtain more reliable ranking results to identify the best university for benchmarking.

3. Preliminaries

In this section, the basic concepts of IVHFLSs are explained for understanding the proposed UTAE approach.

Definition 1 [58].

Let $S=\left\{s_i|i=0,1,\dots ,,2t\right\}$ be a discrete linguistic term set with odd cardinality, in which s_i is a possible value for a linguistic variable and t is a positive integer. Then, the linguistic term set has the following features:

(1) 

The set is ordered: $s_i>s_j$, if i > j;

(2) 

There is a negation operator: $s_i=neg\left(s_j\right)$ satisfying i + j = 2t.

Definition 2 [10].

Let $X=\left\{x_1,x_2,\dots ,x_n\right\}$ be a reference set and   $s_{\theta \left(x\right)}\in S$. An IVHFLS A in X is defined by

$$A=\left\{\langle x,s_{\theta \left(x\right)},{\Gamma }_A\left(x\right)\rangle |x\in X\right\}$$

(1)

 where ${\Gamma }_A\left(x\right)$ is a set of closed intervals belonging to (0, 1] and represents the interval-valued membership degrees that x belongs to $s_{\theta \left(x\right)}$.

When $X=\left\{x_1,x_2,\dots ,x_n\right\}$ has only one component, the IVHFLS A becomes $\alpha =\langle s_{\theta \left(x\right)},{\Gamma }_A\left(x\right)\rangle$ which is called an interval-valued hesitant fuzzy linguistic number (IVHFLN).

Definition 3 [10].

For the linguistic term set S, the relationship between s_i and its subscript i is strictly monotonically increasing. Then, a linguistic scale function f to map from s_i to θ_i $\left({\theta }_i\in R^{+},R^{+}=\left\{r|r\ge 0,r\in R\right\}\right)$ can be given as:

$$f:s_i\to {\theta }_i\hspace{0.17em}\hspace{0.17em}\left(i=0,1,2,\dots ,2t\right)$$

(2)

 where $0\le {\theta }_0<{\theta }_1<\dots <{\theta }_{2t}$.

Several functions have been developed for conducting the mapping, and the one frequently used is based on the subscript function as:

$$f\left(s_i\right)={\theta }_i=\frac{i}{2t}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left(i=0,1,2,\dots ,2t\right)$$

(3)

Here ${\theta }_i\in \left[0,1\right]$.

Definition 4 [10].

Assume that there are two IVHFLNs $\alpha =\langle s_{\theta \left(\alpha \right)},{\Gamma }_{\alpha }\rangle$ and $\beta =\langle s_{\theta \left(\beta \right)},{\Gamma }_{\beta }\rangle$. The basic operations of IVHFLNs are defined as follows:

(1) 

$\alpha \oplus \beta =\langle f^{*-1}\left(f^{*}\left(s_{\theta \left(\alpha \right)}\right)+f^{*}\left(s_{\theta \left(\beta \right)}\right)\right),{\cup }_{r_1=\left[r_1^L,r_1^U\right]\in {\Gamma }_{\alpha },r_2=\left[r_2^L,r_2^U\right]\in {\Gamma }_{\beta }}$

$$\left\{\left[\frac{f^{*}\left(s_{\theta \left(\alpha \right)}\right)r_1^L+f^{*}\left(s_{\theta \left(\beta \right)}\right)r_2^L}{f^{*}\left(s_{\theta \left(\alpha \right)}\right)+f^{*}\left(s_{\theta \left(\beta \right)}\right)},\frac{f^{*}\left(s_{\theta \left(\alpha \right)}\right)r_1^U+f^{*}\left(s_{\theta \left(\beta \right)}\right)r_2^U}{f^{*}\left(s_{\theta \left(\alpha \right)}\right)+f^{*}\left(s_{\theta \left(\beta \right)}\right)}\right]\right\}\rangle$$

(2)  

$\lambda \alpha =\langle f^{*-1}\left(\lambda f^{*}\left(s_{\theta \left(\alpha \right)}\right)\right),{\Gamma }_{\alpha }\rangle ,where\lambda \ge 0$;

(3)  

$\alpha \otimes \beta =\langle f^{*-1}\left(f^{*}\left(s_{\theta \left(\alpha \right)}\right)f^{*}\left(s_{\theta \left(\beta \right)}\right)\right),{\cup }_{r_1=\left[r_1^L,r_1^U\right]\in {\Gamma }_{\alpha },r_2=\left[r_2^L,r_2^U\right]\in {\Gamma }_{\beta }}\left\{r_1^L{r}_2^L,r_1^U{r}_2^U\right\}\rangle$.

Definition 5 [10].

Let $\alpha =\langle s_{\theta \left(\alpha \right)},{\Gamma }_{\alpha }\rangle =\langle s_{\theta \left(\alpha \right)},{\cup }_{r=\left[r^L,r^U\right]\in {\Gamma }_{\alpha }}\left\{\left[r^L,r^U\right]\right\}\rangle$ be an IVHFLN. An expectation function $E_r\left({\Gamma }_{\alpha }\right)$ of ${\Gamma }_{\alpha }$ can be denoted by $E_r\left({\Gamma }_{\alpha }\right)=\frac{{\displaystyle {\sum }_{\left[r^L,r^U\right]\in {\Gamma }_{\alpha }}\left(r^L+r^U\right)}}{2\times \#{\Gamma }_{\alpha }}$. Thus, the score function $E\left(\alpha \right)$ of $\#{\Gamma }_{\alpha }$ can be represented as follows:

$$E\left(\alpha \right)=f^{*}\left(s_{\theta \left(\alpha \right)}\right)\times E_r\left({\Gamma }_{\alpha }\right)$$

(4)

where $\#{\Gamma }_{\alpha }$ is the number of the interval values in ${\Gamma }_{\alpha }$.

Definition 6 [10].

Let $\alpha =\langle s_{\theta \left(\alpha \right)},{\Gamma }_{\alpha }\rangle =\langle s_{\theta \left(\alpha \right)},{\cup }_{r=\left[r^L,r^U\right]\in {\Gamma }_{\alpha }}\left\{\left[r^L,r^U\right]\right\}\rangle$ be an IVHFLN. A variance function $D_r\left({\Gamma }_{\alpha }\right)$ of ${\Gamma }_{\alpha }$ can be denoted by $D_r\left({\Gamma }_{\alpha }\right)=\frac{1}{\#{\Gamma }_{\alpha }}{\displaystyle {\sum }_{\left[r^L,r^U\right]\in {\Gamma }_{\alpha }}{\left[\frac{r^L+r^U}{2}-E_r\left({\Gamma }_{\alpha }\right)\right]}^2}$. Thus, the accuracy function $D\left(\alpha \right)$ of $\alpha$ is computed by:

$$D\left(\alpha \right)=f^{*}\left(s_{\theta \left(\alpha \right)}\right)\times \left[1-D_r\left({\Gamma }_{\alpha }\right)\right]$$

(5)

where $\#{\Gamma }_{\alpha }$ is the number of the interval values in ${\Gamma }_{\alpha }$.

Definition 7 [10].

Let ${\alpha }_1=\langle s_{\theta \left({\alpha }_1\right)},{\Gamma }_{{\alpha }_1}\rangle$ and ${\alpha }_2=\langle s_{\theta \left({\alpha }_2\right)},{\Gamma }_{{\alpha }_2}\rangle$ be any two IVHFLNs, then:

(1) 

If E(α₁) > E(α₂), then α₁ > α₂.

(2) 

If E(α₁) = E(α₂), then:

(a) 

if D(α₁) > D(α₂), then α₁ > α₂;

(b) 

if D(α₁) = D(α₂), then α₁ = α₂.

Definition 8 [10].

Let ${\alpha }_j=\langle s_{\theta \left({\alpha }_j\right)},{\Gamma }_{{\alpha }_j}\rangle \left(j=1,2,\dots ,n\right)$ be n IVHFLNs. Then, the interval-valued hesitant fuzzy linguistic prioritized weighted average (IVHFLPWA) operator is defined as:

$$\begin{array}{l}IVHFLPWA\left({\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n\right)=\frac{T_1}{{\displaystyle {\sum }_{i=1}^n{T}_i}}{\alpha }_1\oplus \frac{T_2}{{\displaystyle {\sum }_{i=1}^n{T}_i}}{\alpha }_2\oplus \cdots \oplus \frac{T_n}{{\displaystyle {\sum }_{i=1}^n{T}_i}}{\alpha }_n\\ =\langle f^{*}{}^{-1}\left({\displaystyle \sum _{j=1}^n\left(\frac{T_j}{{\displaystyle {\sum }_{i=1}^n{f}^{*}\left(s_{\theta \left({\alpha }_j\right)}\right)}}\right)}\right),{\cup }_{r_1=\left[r_1^L,r_1^U\right]\in {\Gamma }_{\alpha 1},\dots ,r_n=\left[r_n^L,r_n^U\right]\in {\Gamma }_{\alpha n}}\left\{\left[\frac{{\displaystyle {\sum }_{j=1}^n{f}^{*}\left(s_{\theta \left({\alpha }_j\right)}\right)r_j^L{T}_j}}{{\displaystyle {\sum }_{j=1}^n{f}^{*}\left(s_{\theta \left({\alpha }_j\right)}\right)T_j}},\frac{{\displaystyle {\sum }_{j=1}^n{f}^{*}\left(s_{\theta \left({\alpha }_j\right)}\right)r_j^U{T}_j}}{{\displaystyle {\sum }_{j=1}^n{f}^{*}\left(s_{\theta \left({\alpha }_j\right)}\right)T_j}}\right]\right\}\rangle \end{array}$$

(6)

where $T_1=1,T_j={\displaystyle \prod _{k=1}^{j-1}E\left({\alpha }_k\right)},\left(j=2,\dots ,n\right)$ and $E\left({\alpha }_k\right)$ is the score function of ${\alpha }_k$.

4. The Proposed UTAE Methodology

This section provides a new group decision approach by extending the ORESTE method with IVHFLSs for UTAE. The proposed UTAE approach comprises two phases: (1) assessing the teaching quality of universities by employing IVHFLSs; (2) obtaining the teaching quality ranking of universities with an extended ORESTE method. Figure 1 displays the detailed procedure of the proposed UTAE approach.

Assume that there are l education experts $E_k\left(k=1,2,\dots ,l\right)$ in an UTAE group responsible for assessing the teaching quality of m universities $U_i\left(i=1,2,\dots ,m\right)$ concerning n audit elements $A{E}_j\left(j=1,2,\dots ,n\right)$. Each expert is assigned a weight λ_k which satisfies λ_k > 0 (k = 1, 2, …, l) and ${\displaystyle \sum _{k=1}^l{\lambda }_k}=1$ for representing the relative importance of experts in the UTAE. In what follows, the proposed UTAE model to acquire the prioritization of university is given in detail.

Phase 1: Assess the teaching quality of universities based on IVHFLSs.

UTAE team members assess the teaching quality of universities according to their personal knowledge and experience. However, the linguistic information provided by experts is often ambiguous and uncertain. To tackle this problem, the IVHFLSs are adopted here for describing UTAE team members’ assessments.

Step 1: Gathering the team members’ linguistic information on candidate universities.

When implementing UTAE, the team members are invited to score the teaching quality of universities by using IVHFLSs. After each expert’s opinions on the teaching quality of universities have been acquired, an interval-valued hesitant fuzzy linguistic assessment matrix can be established as $R_k={\left[r_{ij}^k\right]}_{m\times n}\left(k=1,2,\dots ,l\right)$, where $r_{ij}^k=\langle s_{\theta \left(r_{ij}^k\right)},{\Gamma }_{r_{ij}^k}\rangle$ indicates the IVHFLN evaluation of the ith university concerning the ith audit element given by the expert E_k.

Step 2: Calculate the collective interval-valued hesitant fuzzy linguistic assessment matrix R.

By using the IVHFLPWA operator, the UTAE team members’ individual opinions are aggregated for computing the collective interval-valued hesitant fuzzy linguistic assessment matrix $R={\left[r_{ij}\right]}_{m\times n}$, i.e.,

$$\begin{array}{l}{r}_{ij}=IVHFLPWA\left(r_{ij}^1,r_{ij}^2,\dots ,r_{ij}^l\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{T_1}{{\displaystyle {\sum }_{t=1}^k{T}_t}}\langle s_{\theta \left(r_{ij}^1\right)},{\Gamma }_{r_{ij}^1}\rangle \oplus \frac{T_2}{{\displaystyle {\sum }_{t=1}^k{T}_t}}\langle s_{\theta \left(r_{ij}^2\right)},{\Gamma }_{r_{ij}^2}\rangle \oplus \cdots \frac{T_k}{{\displaystyle {\sum }_{t=1}^k{T}_t}}\langle s_{\theta \left(r_{ij}^k\right)},{\Gamma }_{r_{ij}^k}\rangle ,\end{array}$$

(7)

where $T_1=1,T_k={\displaystyle \prod _{t=1}^{k-1}E\left(\langle s_{\theta \left(r_{ij}^k\right)},{\Gamma }_{r_{ij}^k}\rangle \right)},\left(k=2,3,\dots ,l\right)$.

Example 1.

Consider an UTAE problem in which four experts are involved to evaluate the teaching quality of a university with a criterion. The interval-valued hesitant fuzzy linguistic assessments of the four experts are assumed to be <s₄,[0.5, 0.8]>, <s₃,[0.4, 0.8]>, <s₄,[0.4, 0.7]>, and <s₃,[0.5, 0.8]>. Then, we can obtain the collective interval-valued hesitant fuzzy linguistic assessment as <s_3.58,[0.46, 0.79>] by using Equation (7).

Phase 2: Acquire the teaching quality ranking of universities with the ORESTE method.

The ORESTE method is a general outranking MCDM method proposed by Roubens [18]. This method turns out to be a particularly useful tool for problems involving decision making and evaluation because of its simplicity and stability [19,21,25]. An expanded ORESTE technique is now being suggested for ranking universities’ teaching quality under the interval-valued hesitant fuzzy linguistic environment.

Step 3: Obtain the global preference scores for each university.

The Besson’s mean ranks ${\tilde{r}}_j$ and ${\tilde{r}}_j\left(U_i\right)$ denote a preference structure and a merit of the U_i on AE_j, respectively. ${\tilde{r}}_j$ can be calculated by the weights of audit elements and ${\tilde{r}}_j\left(U_i\right)$ can be determined with the pairwise comparisons between elements in the matrix $R={\left[r_{ij}\right]}_{m\times n}$. Then, the global preference score D(r_ij) of the U_i against the audit element AE_j can be calculated by:

$$D\left(r_{ij}\right)=\sqrt{\varsigma {\left({\tilde{r}}_j\right)}^2+\left(1-\varsigma \right){\left({\tilde{r}}_j\left(U_i\right)\right)}^2}$$

(8)

where $\varsigma \left(0\le \varsigma \le 1\right)$ is the coefficient determined by experts to show the relative importance between the ranking of audit elements and universities. Generally, $\varsigma$ is assumed as 0.5.

Step 4: Acquire the global weak ranking of universities.

If $D\left(r_{ij}\right)$ > $D\left(r_{pq}\right)$, then $\tilde{r}\left(r_{ij}\right)>\tilde{r}\left(r_{pq}\right)$; if $D\left(r_{ij}\right)$ = $D\left(r_{pq}\right)$, then $\tilde{r}\left(r_{ij}\right)=\tilde{r}\left(r_{pq}\right)$, where i, p = 1, 2, …, m and k, q = 1, 2, …, n. Then, the global weak ranking of each U_i is determined with

$$\tilde{R}\left(U_i\right)=\frac{1}{n}{\displaystyle \sum _{j=1}^{n}D\left(r_{ij}\right)}$$

(9)

Step 5: Determine the preference intensities among universities.

If two universities have the same global weak ranking, then they should be distinguished by computing their preference intensities. To distinguish the incomparability and indifference relations, the preference intensity should be calculated using average preference intensity and net preference intensity, respectively. The average preference intensity between U_i and U_p is defined by:

$$T\left(U_i,U_p\right)=\frac{{\displaystyle \sum _{j=1}^n\max \left[\tilde{R}\left(r_{pj}\right)-\tilde{R}\left(r_{ij}\right),0\right]}}{\left(m-1\right)n^2}$$

(10)

The net preference intensity between U_i and U_p can be computed by:

$$\Delta T\left(U_i,U_p\right)=T\left(U_i,U_p\right)-T\left(U_p,U_i\right)$$

(11)

Step 6: Set up the PIR relation among universities.

When universities are ranked in the same worldwide weak order, the PIR relation explains the priorities of the universities. Universities can be grouped into three categories based on these relationships: preference (P), indifference (I), and incomparability (R). The relations can be analyzed based on the following rules:

(1)

If $\left|\Delta T\left(U_i,U_p\right)\right|\le \sigma$, then

(a)

U_i I U_p, if $\left|T\left(U_i,U_p\right)\right|\le \eta$ and $\left|T\left(U_p,U_i\right)\right|\le \eta$;

(b)

U_i R U_p, if $\left|T\left(U_i,U_p\right)\right|>\eta$ or $\left|T\left(U_p,U_i\right)\right|>\eta$.

(2)

If $\left|\Delta T\left(U_i,U_p\right)\right|>\sigma$, then

(a)

U_i R U_p, if $\frac{\min \left(T\left(U_i,U_p\right),T\left(U_p,U_i\right)\right)}{\left|\Delta T\left(U_i,U_p\right)\right|}\ge \rho$

(b)

U_i P U_p, if $\frac{\min \left(T\left(U_i,U_p\right),T\left(U_p,U_i\right)\right)}{\left|\Delta T\left(U_i,U_p\right)\right|}<\rho$ and $\Delta T\left(U_i,U_p\right)>0$;

(c)

U_p P U_i, if $\frac{\min \left(T\left(U_i,U_p\right),T\left(U_p,U_i\right)\right)}{\left|\Delta T\left(U_i,U_p\right)\right|}<\rho$ and $\Delta T\left(U_p,U_i\right)>0$.

Note that σ, ρ, and η are three different thresholds to distinguish the PIR relations [21] and they are computed by:

$$\sigma <1/\left(m-1\right)n,\rho >\left(n-2\right)/4,\eta <\lambda /2\left(m-1\right)$$

(12)

where λ is determined based on experts’ knowledge to demonstrate the greatest ranking difference between two indifferent universities.

Step 7: Obtain the strong ranking of universities.

The strong ranking result for a university is a joint decision based on the global weak ranking and the PIR structure. The rank of universities is first established in accordance with the P and I relations in the PIR structure, and the full rank may then be calculated by combining the weak rank when the R relations exist among other universities.

The strong ranking $\tilde{r}\left(U_i\right)$ is obtained in line with the sorted $\tilde{R}\left(U_i\right)$ and its PIR relation. When global weak ranking orders of two universities $\stackrel{⌢}{R}\left(U_i\right)>\tilde{R}\left(U_p\right)$, then $\tilde{r}\left(U_i\right)>\tilde{r}\left(U_p\right)$ is a natural number from 1, 2, … and the like. In addition, when $\stackrel{⌢}{R}\left(U_i\right)=\tilde{R}\left(U_p\right)$, $\tilde{r}\left(U_i\right)=\tilde{r}\left(U_p\right)$ and the relationships between r(U_i) and r(U_p) are acquired according to the PIR rules, where i, p = 1, 2, …, m.

5. Illustrative Example

5.1. Application

In this section, a practical example is carried out to display the feasibility and effectiveness of the introduced UTAE methodology. For the case example, seven universities located in Shanghai, China, are involved in the UTAE program [1]. The seven universities are represented by U₁, U₂, …, and U₇. The UTAE team consists of four senior experts represented by E₁, E₂, E₃, and E₄ who work in different universities and teaching quality departments. In line with the review system of the UTAE, ten highly representative audit elements are determined by the expert team to measure the quality of universities from five respects, which are school-running orientation (AE1), training goal (AE2), education and teaching level (AE3), teaching facilities (AE4), classroom teaching (AE5), practical teaching (AE6), learning effect (AE7), employment and development (AE8), teaching quality assurance system (AE9), and quality improvement (AE10).

Next, the proposed UTAE method is adopted to estimate the teaching quality of the seven universities.

Step 1: The UTAE experts use the linguistic term set $S$ given below to evaluate the teaching quality of the universities against each audit element. We gather the UTAE team members’ linguistic information, expressed via the IVHFLNs. After four experts’ opinions on the teaching quality of the seven universities are acquired, the interval-valued hesitant fuzzy linguistic assessment matrixes $R_k={\left[r_{ij}^k\right]}_{7\times 10}\left(k=1,2,3,4\right)$ are established. Because of space constraints, the evaluation result of expert (E₁) is listed in

Table 1. Evaluation result provided by expert E₁.

	U1	U2	U3	U4	U5	U6	U7
AE1	<s4,[0.5, 0.8]>	<s4,[0.2, 0.8]>	<s3,[0.1, 0.3]>	<s3,[0.1, 0.6]>	<s4,[0.5, 0.8]>	<s5,[0.6, 0.8]>	<s5,[0.5, 0.7]>
AE2	<s3,[0.3, 0.6]>	<s5,[0.6, 0.8]>	<s3,[0.1, 0.5]>	<s2,[0.3, 0.6]>	<s5,[0.6, 0.8]>	<s4,[0.5, 0.8]>	<s3,[0.1, 0.4]>
AE3	<s4,[0.4, 0.8]>	<s4,[0.4, 0.7]>	<s4,[0.5, 0.8]>	<s2,[0.2, 0.6]>	<s3,[0.4, 0.7]>	<s6,[0.5, 0.9]>	<s4,[0.5, 0.8]>
AE4	<s3,[0.3, 0.9]>	<s5,[0.6, 0.8]>	<s4,[0.5, 0.8]>	<s5,[0.4, 0.6]>	<s2,[0.7, 0.8]>	<s3,[0.2, 0.6]>	<s4,[0.4, 0.8]>
AE5	<s5,[0.1, 0.5]>	<s6,[0.4, 0.8]>	<s3,[0.1, 0.6]>	<s2,[0.1, 0.4]>	<s3,[0.6, 0.8]>	<s4,[0.5, 0.8]>	<s4,[0.5, 0.8]>
AE6	<s1,[0.1, 0.7]>	<s4,[0.5, 0.9]>	<s2,[0.3, 0.5]>	<s1,[0.5, 0.6]>	<s3,[0.4, 0.6]>	<s5,[0.4, 0.6]>	<s3,[0.1, 0.5]>
AE7	<s3,[0.3, 0.6]>	<s5,[0.2, 0.8]>	<s2,[0.2, 0.6]>	<s3,[0.3, 0.6]>	<s4,[0.5, 0.8]>	<s5,[0.5, 0.6]>	<s4,[0.5, 0.6]>
AE8	<s3,[0.3, 0.6]>	<s4,[0.4, 0.7]>	<s3,[0.4, 0.8]>	<s2,[0.4, 0.9]>	<s4,[0.4, 0.8]>	<s5,[0.2, 0.8]>	<s3,[0.4, 0.7]>
AE9	<s2,[0.1, 0.6]>	<s5,[0.4, 0.8]>	<s5,[0.2, 0.5]>	<s3,[0.3, 0.8]>	<s2,[0.1, 0.3]>	<s3,[0.3, 0.6]>	<s5,[0.5, 0.7]>
AE10	<s3,[0.3, 0.8]>	<s6,[0.6, 0.7]>	<s4,[0.5, 0.8]>	<s2,[0.6, 0.9]>	<s2,[0.2, 0.3]>	<s3,[0.5, 0.6]>	<s2,[0.4, 0.5]>

.

$$S=\left\{\begin{array}{l}{s}_0=\mathrm{Very}\mathrm{poor}\left(\mathrm{VP}\right),\hspace{0.17em}{s}_1=\mathrm{Poor}\left(\mathrm{P}\right),s_2=\mathrm{Medium}\mathrm{poor}\left(\mathrm{MP}\right),s_3=\mathrm{Medium}\left(\mathrm{M}\right),\\ s_4=\mathrm{Medium}\mathrm{good}\left(\mathrm{MG}\right),s_5=\mathrm{Good}\left(\mathrm{G}\right),s_6=\mathrm{Very}\mathrm{good}\left(\mathrm{VG}\right)\end{array}\right\}$$

Step 2: Based on Equation (7), the individual linguistic evaluations of experts are aggregated to get the collective interval-valued hesitant fuzzy linguistic assessment matrix $R={\left[r_{ij}\right]}_{7\times 10}$ as shown in

Table 2. The collective interval-valued hesitant fuzzy linguistic assessment matrix R.

${\mathit{r}}_{\mathit{i}\mathit{j}}$	U1	U2	U3	U4	U5	U6	U7
AE1	<s3.58,[0.46, 0.79]>	<s3.43,[0.32, 0.78]>	<s2.94,[0.23, 0.39]>	<s3.29,[0.09, 0.68]>	<s3.98,[0.48, 0.85]>	<s5.38,[0.59, 0.74]>	<s4.89,[0.49, 0.73]>
AE2	<s2.13,[0.23, 0.72]>	<s4.12,[0.43, 0.79]>	<s2.69,[0.13, 0.51]>	<s1.99,[0.22, 0.63]>	<s5.18,[0.62, 0.84]>	<s3.46,[0.47, 0.86]>	<s3.79,[0.12, 0.46]>
AE3	<s3.79,[0.42, 0.80]>	<s3.80,[0.39, 0.72]>	<s4.12,[0.49, 0.76]>	<s2.35,[0.19, 0.61]>	<s3.39,[0.41, 0.74]>	<s5.66,[0.49, 0.87]>	<s3.87,[0.56, 0.85]>
AE4	<s4.52,[0.36, 0.60]>	<s5.36,[0.59 0.76]>	<s4.68,[0.57, 0.89]>	<s5.06,[0.41 0.55]>	<s2.63,[0.74, 0.81]>	<s3.11,[0.21, 0.57]>	<s4.03,[0.45, 0.84]>
AE5	<s5.21,[0.22, 0.62]>	<s5.97,[0.42, 0.79]>	<s3.08,[0.24, 0.71]>	<s2.48,[0.36, 0.87]>	<s3.46,[0.62, 0.77]>	<s4.18,[0.47, 0.80]>	<s3.96,[0.16, 0.83]>
AE6	<s0.89,[0.22, 0.53]>	<s3.69,[0.55, 0.86]>	<s2.92,[0.32, 0.57]>	<s1.24,[0.45, 0.62]>	<s3.25,[0.49, 0.55]>	<s5.03,[0.49, 0.63]>	<s3.07,[0.09, 0.43]>
AE7	<s2.78,[0.32, 0.55]>	<s5.01,[0.31, 0.87]>	<s1.85,[0.29, 0.81]>	<s3.31,[0.31, 0.79]>	<s3.86,[0.52, 0.84]>	<s4.99,[0.51, 0.62]>	<s3.80,[0.52, 0.66]>
AE8	<s3.31,[0.31, 0.65]>	<s3.69,[0.43, 0.76]>	<s5.09,[0.15, 0.59]>	<s2.27,[0.58, 0.69]>	<s3.58,[0.46, 0.79]>	<s5.36,[0.19, 0.77]>	<s3.14,[0.41, 0.73]>
AE9	<s2.54,[0.16, 0.58]>	<s5.45,[0.42, 0.86]>	<s3.58,[0.46, 0.79]>	<s3.96,[0.41, 0.72]>	<s1.85,[0.08 0.79]>	<s3.46,[0.28, 0.61]>	<s5.03,[0.56, 0.70]>
AE10	<s3.64,[0.34, 0.77]>	<s5.81,[0.56, 0.73]>	<s3.98,[0.45, 0.74]>	<s2.30,[0.49, 0.95]>	<s2.34,[0.17, 0.36]>	<s3.90,[0.47, 0.61]>	<s2.09,[0.43, 0.53]>

.

Step 3: According to the weights of audit elements given by the UTAE expert team, the Besson’s mean ranks ${\tilde{r}}_j$ are determined as ${\tilde{r}}_j={\left(5.5,5.5,1,7,10,9,8,2,3.5,3.5\right)}^{\mathrm{T}}$. By the pairwise comparisons between elements in the matrix R, the values of Besson’s mean rank ${\tilde{r}}_j\left(U_i\right)\left(i=1,2,\dots ,7,j=1,2,\dots ,10\right)$ are derived as presented in

Table 3. The values of Besson’s mean rank ${\tilde{r}}_j\left(U_i\right)$.

${\tilde{r}}_j\left(U_i\right)$	U1	U2	U3	U4	U5	U6	U7
AE1	27.5	42.5	63	54	16	5.5	10
AE2	59	22	64.5	64.5	2	27.5	57
AE3	25.5	36.5	19.5	61.5	41	1	14
AE4	34.5	5.5	5.5	23.5	38.5	55.5	19.5
AE5	34.5	5.5	51.5	50	23.5	16	40
AE6	70	19.5	53	68	46.5	12.5	66.5
AE7	55.5	11	59	44.5	16	12.5	31
AE8	48.5	31	42.5	51.5	31	19.5	44.5
AE9	61.5	8	31	31	66.5	48.5	9
AE10	38.5	3	25.5	46.5	69	36.5	59

. In accordance with Equation (8), the global preference scores for the seven universities D(r_ij) $\left(i=1,2,\dots ,7,j=1,2,\dots ,10\right)$ are obtained as shown in

Table 4. The global preference scores D(r_ij).

D(rij)	1	2	3	4	5	6	7
AE1	19.83	30.30	44.72	38.38	11.96	5.50	8.07
AE2	41.90	16.04	45.77	45.77	4.14	19.83	40.49
AE3	18.05	25.82	13.81	43.49	29.00	1.00	9.92
AE4	24.89	6.29	6.29	17.34	27.67	39.56	14.65
AE5	25.40	8.07	37.10	36.06	18.06	13.34	29.15
AE6	49.90	15.19	38.01	48.50	33.49	10.89	47.45
AE7	39.65	9.62	42.10	31.97	12.65	10.49	22.64
AE8	34.32	21.97	30.09	36.44	21.97	13.86	31.50
AE9	43.56	6.17	22.06	22.06	47.09	34.38	6.83
AE10	27.22	2.12	18.03	32.88	48.79	25.81	41.72

.

Step 4: By Equation (9), the global weak ranking of each university is obtained as:$\tilde{R}\left(U_1\right)=32.47,\tilde{R}\left(U_2\right)=14.16,\tilde{R}\left(U_3\right)=29.80,\tilde{R}\left(U_4\right)=35.29,\tilde{R}\left(U_5\right)=25.48,\tilde{R}\left(U_6\right)=17.47,$ $\tilde{R}\left(U_7\right)=25.24$.

Steps 5 and 6: We inspect the results of the global weak ranking of each university and find that none have the same global weak ranking order. Thus, the preference intensities among universities need not be computed.

Step 7: The strong ranking $\tilde{r}\left(U_i\right)$ is obtained according to the sorted $\tilde{R}\left(U_i\right)$. So, the ranking of the seven universities is determined as shown in

Table 5. The strong ranking r(U_i).

Strong Ranking	Universities
	U1	U2	U3	U4	U5	U6	U7
r(Ui)	6	1	5	7	4	2	3

. Therefore, U₂ is the best university which can be selected for benchmarking.

5.2. Comparative Analysis

In this section, a comparative analysis with current approaches is given to further illustrate the effectiveness and benefits of the suggested UTAE model. The above case study is solved by the q-rung orthopair fuzzy MABAC (q-ROF-MABAC) method [1], the classical ORESTE [18], the hesitant fuzzy linguistic ORESTE (HFL-ORESTE) [19], the IVHFL decision making (IVHFL-DM) [10], and the IVHFL-MULTIMOORA [14] methods. The ranking results of alternative universities derived by the listed approaches are tabulated in Figure 2.

From Figure 2, we can find that the ranking results of the seven universities solved by the listed methods are approximately the same, where U₂ ranks first and U₄ ranks the last. Moreover, the priority orders for the universities obtained by the q-ROF-MABAC and the HFL-ORESTE method are exactly the same; the ranking result of six universities (i.e., U₁, U₂, U₄, U₆, U₇) obtained via the ORESTE method is identical to the one by the proposed approach and the ranking orders of four universities (i.e., U₂, U₄, U₆, U₇) obtained by the IVHFL-MULTIMOORA are consistent with those derived by the proposed approach. This demonstrates the value of the UTAE strategy advocated in this study.

In addition, we can see that the ranking outcomes produced by the suggested UTAE approach and those produced by the ORESTE, IVHFL-DM, and IVHFL-MULTIMOORA methods differ slightly from one another. In line with the ORESTE and the IVHFL-MULTIMOORA methods, U₅ has a lower priority in comparison with U₃. In addition, the result of the proposed approach suggests U₇ has a higher priority compared with U₅. But their ranking order is reversed by the IVHFL-DM method. The following factors can be used to explain the inconsistencies. First, the ORESTE and HFL-ORESTE approaches, which are unable to adequately portray the ambiguous evaluations of the UTAE team members, use crisp values and cautious fuzzy sets, respectively, to express the expert opinions. Second, there may be inconsistencies in the ranking results since the algorithms used to determine the priority ranking of colleges using the methods stated above differ.

A further study of the comparative approaches is carried out utilizing the aggregation methodology [21] in order to further validate the suggested UTAE strategy. The most effective strategy ought to produce results that are most consistent with the aggregation ranking. The best aggregated probability priority of the tasks in the case example is judged to be U₇ > U₅ > U₃ > U₁ > U₄, which is the same as the outcome produced by the suggested UTAE approach. As a result, the suggested UTAE technique produces a more fair and reliable ranking of universities in the application.

The ranking results of universities acquired using the newly established UTAE approach are more acceptable and rational, according to the analysis above. Compared with other UTAE methods, the proposed approach using IVHFLSs and the ORESTE method has the following advantages:

(1)

The proposed approach can reduce information loss when aggregating multiple-expert evaluations and increase the flexibility of eliciting and displaying experts’ linguistic assessment information by utilizing the IVHFLSs. Decision makers can now communicate their opinions more clearly and realistically.

(2)

The suggested methodology, which is based on the ORESTE method, is more effective in the UTAE process and can help decision makers get rankings of alternative colleges that are fairer and more trustworthy. This improves the viability and realism of the IVHFLS-ORESTE technique.

6. Conclusions

In this study, we introduced a new approach for UTAE based on IVHFLSs and the ORESTE method. The IVHFLSs are used to describe the real evaluation information of experts and reflect their uncertainty, hesitancy, and inconsistency. An extended ORESTE method with the IVHFLSs is developed to rank the alternative universities and choose the best one for benchmarking. Additionally, an illustrative UTAE example is provided to illustrate the practicability and effectiveness of the proposed UTAE approach. The results demonstrate that the proposed IVHFLS-ORESTE methodology is efficient and can not only characterize the ambiguous and uncertain assessments of experts flexibly, but also obtain a more accurate and stable ranking result of universities.

Considering the findings revealed in this study, the proposed IVHFLS-ORESTE methodology for UTAE has the following benefits for higher education administrators. Firstly, the outcomes of the example illustrate the process of how to determine the ranking of teaching quality among a set of universities. Secondly, the IVHFLS-ORESTE method we proposed can be applied in the field of UTAE, which provides a more stable and reliable result. Lastly, the same ranking orders of the universities achieved by the IVHFLS-ORESTE method can be analyzed by PIR relation that assists administrators in adjusting education policies.

However, the proposed UTAE approach has several disadvantages which can be addressed in future research. First, the proposed UTAE approach is unable to handle incomplete information elaborately. Thus, we suggest adopting other uncertainty methods, such as the quasirung orthopair fuzzy sets [59,60], to express the ambiguity and impreciseness of experts’ assessment data. Second, the proposed UTAE methodology is limited to a small group of experts. In the future, it can be extended to a large group environment [61,62] to improve the evaluation efficiency. In addition, because of the involvement of objective data in the UTAE, subsequent research may focus on how to integrate the objective data and subjective evaluations for UTAE.