A Novel D–SCRI–EDAS Method and Its Application to the Evaluation of an Online Live Course Platform

Abstract

D number theory removes the constraints of mutual exclusion and completeness in the frame of discernment of DS evidence theory, and is therefore widely used to deal with uncertain and incomplete information. EDAS (evaluation based on distance from average solution) selects the optimal solution according to the distance from each solution to the average. This method is very suitable for solving issues of multiple attribute decision making (MADM) with conflicting attributes. In this study, we propose an evaluation method that combines D numbers with the EDAS method. D number theory is used to express the evaluation of the alternatives for experts. Then, we use the SCRI (stepwise comparison and replacement integration) method of D numbers for data integration, and finally the EDAS method is used to select the optimal one by sorting the alternatives. We apply this method to address the user experience evaluation problems of platforms that offer online live courses and compare the evaluation results with other methods to verify the applicability and practicability of the method.

1. Introduction

Multiple attribute decision making (MADM) is an important part of modern decision science. In the decision-making process, most MADMs are uncertain types due to the uncertainty of decision-making information and the limitations of decision-makers’ cognitive abilities. The main research methods include: grey correlation analysis [1], Dempster–Shafer evidence theory (DS evidence theory) [2,3,4], fuzzy set theory [5,6,7] and rough set theory [8,9,10], etc. Among them, DS evidence theory can deal with uncertain information more effectively than other methods, and overcome the shortcomings of classical probability theory in uncertainty modeling such as belief expression and subjective cognition, thus has been widely used. D number theory [11] is an extension of DS evidence theory. It removes the constraints of mutual exclusivity and completeness in the framework of discernment of evidence theory and fits a more general framework than DS evidence theory. When faced with actual decision-making problems, the form of language level is more favored by decision-makers, because it is closer to human language habits, so the multi-attribute decision-making problem based on language evaluation has also attracted much attention. However, the traditional language evaluation implies that the confidence of each language value is 1, which cannot describe the degree of hesitation of the evaluator. In addition, in life many semantic evaluation levels such as “very good”, “good”, “average”, “poor” and “very poor” overlap with each other, and evaluators are often forced to make choices in fields they are not familiar with, resulting in distorted evaluation results. The application of D number theory allows evaluators to make choices according to their actual situation when obtaining expert information, or even choose to abstain, which preserves the authenticity of language evaluation to the greatest extent. Therefore, the use of D number theory for language decision-making has significant advantages.

In recent years, some scholars have done a lot of research into D number theory. For example, Guan et al. [12] proposed a new combination rule for the problem that the combination rules had failed to satisfy the associativity; regarding the use of language terms as the evaluation set of D numbers, Xiao [13] proposed an MADM model based on D numbers and linguistic variables, namely a linguistic D numbers model. Seiti et al. [14] extended linguistic D numbers to an interval-valued belief structure. Mo and Deng [15] proposed a new method to deal with a MADA problem under both probabilistic and fuzzy uncertainties based on D numbers. In response to semantic intersection data, the distance between two D numbers is defined according to the distance definition between two BPAs in DS evidence theory, so that under the same frame of discernment, the distance of D numbers is smaller than that of DS evidence theory [16]. Regarding the reliability of D numbers, Xia et al. [17] designed an entropy function of D numbers, and based on this, they proposed a reliability index of D numbers. To deal with the conversion issue between D numbers and real numbers, Deng [11] proposed a D numbers integration representation, which can convert a D number into a real number. However, if the information is incomplete, the method will simply ignore the missing part of the evaluation, and the result may go against common sense. To remedy this shortcoming, Wang et al. [18] proposed an improved D numbers integration method, which proportionally assigns missing information to the decision-making according to the original value of D numbers, but this method does not consider the evaluation objects in a large image, thus leading to a counter-intuitive result as well. Hou and Zhao [19] proposed a new D numbers integration method updated from Wang et al. [18], which horizontally compares the data of multiple evaluation projects and can fill in the missing information properly.

D numbers can also be combined with other MADM classic methods to deal with decision-making problems in uncertain environments. We summarize the relevant literature in

Table 1. The review of the D numbers be combined with other MADM classic methods.

Year	Combined with Other MADM Method	Applications	Lierature Reference
2014	D-AHP	Supplier Selection	[20]
	D-VIKOR	Medicine Provider Selection	[21]
2016	D-TOPSIS	Human Resources Selection	[22]
2017	D-AHP	Human Reliability Analysis	[23]
		University Scientific Research Ability	[24]
2018	D-TOPSIS	Failure Mode and Effects Analysis	[25]
2019	D-TODIM-Choquet Integral	Performance of Motor Engine	[26]
2020	D-VIKOR-Fuzzy Entropy	Medicine Provider Selection	[27]
	D-Soft Likelihood Function	Performance of Automobiles	[28]
	D-BWM-COPRAS-WASPAS	Evaluation of Renewable Energy Resources	[29]
	D-FAHP	Promoting Quality Goals	[30]
	D-TOPSIS-BM Operator	Supplier Selection	[31]
2021	D-Soft Likelihood Function	Healthcare Waste Management	[32]
	D-DNMA-CRITIC	Blockchain Platform Evaluation	[33]
	D-MABAC-BWM	Healthcare Waste Management	[34]
	D-SWOT	Safety risk	[35]
	D-POWA Aggregation-Soft Likelihood Function	Car Performance Assessment	[36]
2022	D-BWM-EDAS	Battery Suppliers for New Energy Vehicles	[37]

. (The abbreviations used for the first time in the table: AHP (analytic hierarchy process), VIKOR (Vlsekriterijumska Optimizacija I Kompromisno Resenje method), TOPSIS (technique for order preference by similarity to ideal solution), TODIM (an acronym in Portuguese for interactive and multi-criteria decision making), the best–worst method (BWM), COPRAS (complex proportional assessment), WASPAS (weighted aggregates sum product assessment), FAHP (fuzzy analytic hierarchy process), BM (Bonferroni mean) operator, DNMA (double normalization-based multiple aggregation), CRITIC (criteria importance through inter-criteria correlation), MABAC (multi-attributive border approximation area comparison), SWOT (strengths–weaknesses–opportunities–threats), POWA (power ordered weighted averaging))

Keshavarz Ghorabaee et al. [38] proposed the EDAS method and studied the problems of multi-criteria inventory classification. Keshavarz Ghorabaee et al. [39] extended the EDAS approach to a fuzzy information environment and illustrated the applicability of the proposed approach with a case study of supplier selection. Keshavarz Ghorabaee et al. [40] proposed a stochastic extension of the EDAS method to handle MCDM problems with normally distributed data. In recent years, many scholars have extended the EDAS method to different information environments and combined it with other MCDM classical methods to solve problems in different fields, such as the supplier selection problem in the environment of interval type-2 fuzzy sets [41]; the construction contractor selection problem in the interval grey numbers environment [42]; combining with the fuzzy AHP method to solve a third-party logistics provider selection problem [43]; prioritization of sustainable development goals in an interval-valued neutrosophic environment [44]; evaluation of autonomous vehicles under Minkowski space [45]; combined with the MULTIMOORA approach to address a renewable energy adoption problem [46]; green supplier selection in the context of the probabilistic linguistic term sets (PLTSs) [47], and so on.

However, there is still much room to research into the EDAS model. The D numbers method has an advantage in collecting expert opinions, while the EDAS method does well in the selection of solutions with conflicting attributes. In the process of solving a multi-stage evaluation problem, we can comprehensively adopt advantageous methods in various phases to select the optimal solution for the problem. In the process of evaluating the user experience of the online live course platform, we found that the evaluation attributes are qualitative indicators with ambiguity and uncertainty, and it is difficult to carry out quantitative analysis. D numbers have certain advantages in dealing with uncertain information and individual preferences, and combined with the SCRI method, the incomplete evaluation information can also be quantified. At the same time, there is a certain conflict between the multi-criteria of online live courses. Compared with the common multi-attribute decision-making method, the EDAS method is more suitable for dealing with multi-attribute problems with conflict. Therefore, this paper attempts to evaluate the user experience of the online live course platform based on D number theory combined with the EDAS method.

Fan et al. [37] proposed an evaluation method that combines D numbers with BWM and the EDAS method to solve the problem of selecting battery suppliers for new energy vehicles. Firstly, in order to express the uncertainty of expert decision-making, D numbers are used to describe the evaluation information. Then the BWM method is applied to determine the weight of a given criterion, and finally the EDAS method is applied to rank the battery suppliers of new energy vehicles. We note that in the first stage of obtaining expert information, this method represents each expert’s opinion as a D number, and then applies the D number combination method proposed by Xiao [13] to obtain the comprehensive evaluation D number of each criterion, and then transforms each D number into a real number according to the integration method of Deng [11]. In the above-mentioned D number combination and integration process, the evaluation information will be distorted and discarded. When the number of experts is relatively large, after several D number combinations, the final comprehensive evaluation value may no longer effectively reflect the actual situation. For the evaluation problem that requires a large number of experts, this paper proposes another way of thinking. We use the form of language grading to let each expert choose the most suitable grade for each criterion of the alternative. If individual experts cannot give a choice for some reason, they are also allowed to abstain. After the evaluation, the choices of all experts will be converted into D number form. This D number form may be incomplete information. In order to avoid the loss of information as much as possible, we use the SCRI method of D numbers for data integration, and finally use the EDAS method to sort and evaluate the obtained real number evaluation results.

The structure of this paper is organized as follows: Section 2 briefly reviews the relevant definitions of DS evidence theory and D numbers. Section 3 gives the idea and calculation steps of the D–SCRI–EDAS method. Section 4 presents a specific case of user experience evaluation in the platform that offers online live broadcast courses to verify the feasibility of the proposed method. Section 5 presents the conclusion.

2. Preliminaries

2.1. Dempster–Shafer Theory

DS evidence theory is a numerical reasoning method for evidence reasoning. It was first proposed by Dempster in 1967 and later popularized by Shafer in 1976, hence named as Dempster–Shafer evidence theory. The essence of DS evidence theory is a generalization of classical probability theory. It expands the basic event space in probability theory into the power set space, and then establishes a basic probability assignment function to overcome the flaws of classical probability theory in uncertainty modeling such as incomprehensibility, subjective cognition, and so on. In addition, DS evidence theory also provides Dempster fusion rules for information fusion, which features a simple fusion algorithm structure and high fusion accuracy, and obeys the commutative and associative laws.

Definition 1.

Refs. [2,3] (basic probability assignment, BPA) Let $\Theta =\left\{{\theta }_1,{\theta }_2,\cdots ,{\theta }_n\right\}$ be a nonempty finite set of hypotheses or propositions, which is called the frame of discernment. A basic probability assignment (BPA) is a function $m:2^{\Theta }\to \left[0, 1\right]$, which is called a mass function, satisfying:

$$m\left(\varphi \right)=0;$$

(1)

$$0\le m\left(A\right)\le 1, \mathit{\forall }A\subseteq \Theta ;$$

(2)

$${\sum }_{A\subseteq \Theta }m\left(A\right)=1;$$

(3)

where$\varphi$is an empty set and$2^{\Theta }$is the power set of$\Theta$.

Shafer interprets the basic probability assignment function as a subjective representation of evidence, that is, the degree of trust in Proposition A represents the degree of support for it. The advantage of DS evidence theory’s BPA is that uncertainty is directly expressed by assigning basic probability numbers to subsets that consist of n objects rather than individual objects. $A$ is regarded as a focal element when $m\left(A\right)>0$, and the union of all focal elements is called the core of the mass function.

Definition 2.

Refs. [2,3] (Dempster’s combination rule) Suppose $m_1$ and $m_2$ are two independent BPAs from different sources, the Dempster combination rule, denoted by $m=m_1\oplus m_2$, is defined as follows:

$$m\left(A\right)=\{\begin{array}{c}0, A=\varphi \\ \begin{array}{c}\\ \frac{1}{1-k}{\displaystyle {\displaystyle \sum }_{B{\displaystyle \cap }C=A}}{m}_1\left(B\right)m_2\left(C\right), \forall A\subseteq \Theta ,A\ne \varphi \end{array}\end{array}.$$

(4)

With

$$k={\displaystyle \sum }_{B{\displaystyle \cap }C=\varphi }{m}_1\left(B\right)m_2\left(C\right)$$

(5)

where$k$is a normalization constant, called the conflict coefficient of two BPAs. When$k=1$, it means that$m_1$and$m_2$completely conflict, and they cannot be combined by Dempster’s combination rule.

2.2. D Number Theory

DS theory expands the basic event space in probability theory into the power-set space. Compared with traditional probability theory, DS evidence theory not only effectively expresses random uncertainty, but also illustrates incomplete information and subjective uncertainty information. However, some premises and strict constraints of DS evidence theory have caused DS evidence theory to be greatly restricted when dealing with fuzzy and uncertain information: First, the frame of discernment must be a mutually exclusive set, that is, the elements in the frame must be mutually exclusive, and it is often difficult to meet these conditions in the language environment of fuzzy evaluation, such as “satisfied”, “very satisfied”, “extremely satisfied”, all of which are personal discourses that are evaluated according to their very own set of reference objects. Therefore, it will inevitably contain many fuzzy areas that intersect with each other, meaning the mutual exclusivity of elements cannot be guaranteed. Second, in general, the use of BPA is subject to completeness, that is, the sum of the reliability of all focal elements must be equal to 1 (${\sum }_{A\subseteq \Theta }m\left(A\right)=1$), which is difficult to satisfy in reality. As many evaluations and assessments are based on partial information, a complete BPA may not be obtained.

Deng improved the evidence theory to obtain D number theory [11] that overcomes the defects of DS evidence theory and expands the scope of application, so that D number theory can also be used in fuzzy contexts and under the condition that completeness constraints are not satisfied. D number theory is defined as follows:

Definition 3.

Ref. [11] Let $\Omega$ be a finite nonempty set, D number is a mapping $D:\Omega \to \left[0, 1\right]$, such that

$${\sum }_{B\subseteq \Omega }D\left(B\right)\le 1\mathrm{and}D\left(\varphi \right)=0$$

(6)

where$\varphi$is an empty set and$B$is a subset of$\Omega$. Note that, different from the concept of frame of discernment in Dempster–Shafer theory, in set$\Omega$the elements do not require mutual exclusivity.

There is a special situation of D numbers.

Definition 4.

Ref. [11] For a discrete set $\Omega =\left(b_1,b_2,\cdots ,b_n\right)$, where $b_i\in {\mathrm{N}}^{+}$ ($i=1,2,\cdots ,n$), and $b_i\ne b_j$ if $i\ne j$, a special form of D numbers can be expressed by

$$D\left(\left\{b_1\right\}\right)=v_1$$

$$D\left(\left\{b_2\right\}\right)=v_2$$

$$\cdots$$

$$D\left(\left\{b_n\right\}\right)=v_n$$

Simple notes for $D=\left\{\left(b_1,v_1\right),\left(b_2,v_2\right),\cdots ,\left(b_n,v_n\right)\right\}$, where $v_i>0$ and $\sum _{i=1}^n{v}_i\le 1$. If $\sum _{i=1}^n{v}_i=1$, the information is said to be complete; If $\sum _{i=1}^n{v}_i<1$, the information is said to be incomplete.

Definition 5.

Ref. [11] (D number theory combination rule) Suppose $D_1$ and $D_2$ are two D numbers, denoted by

$$\{\begin{array}{c}{D}_1=\left\{\left(b_1^1,v_1^1\right),\cdots ,\left(b_i^1,v_i^1\right),\cdots ,\left(b_n^1,v_n^1\right)\right\}\\ D_2=\left\{\left(b_1^2,v_1^2\right),\cdots ,\left(b_j^2,v_j^2\right),\cdots ,\left(b_m^2,v_m^2\right)\right\}\end{array}\begin{array}{c}\\ \end{array}$$

The combination of $D_1$ and $D_2$, indicated by $D=D_1\oplus D_2$, is defined by

$$D\left(b\right)=v$$

(7)

with

$$b=\frac{b_i^1+b_j^2}{2}$$

(8)

$$v=\frac{v_i^1+v_j^2}{2}/C$$

(9)

$$C=\{\begin{array}{cc}{\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_j^2}{2}),}}& {\displaystyle \sum _{i=1}^n{v}_i^1}=1and{\displaystyle \sum _{j=1}^m{v}_j^2}=1;\\ {\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_j^2}{2})+{\displaystyle \sum _{j=1}^m(\frac{v_c^1+v_j^2}{2})},}}& {\displaystyle \sum _{i=1}^n{v}_i^1}<1and{\displaystyle \sum _{j=1}^m{v}_j^2}=1;\\ {\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_j^2}{2})+{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_c^2}{2})},}}& {\displaystyle \sum _{i=1}^n{v}_i^1}=1and{\displaystyle \sum _{j=1}^m{v}_j^2}<1;\\ {\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_j^2}{2})+{\displaystyle \sum _{j=1}^m(\frac{v_c^1+v_j^2}{2})}+{\displaystyle \sum _{i=1}^n(\frac{v_i^1+v_c^2}{2})}+\frac{v_c^1+v_c^2}{2},}}& {\displaystyle \sum _{i=1}^n{v}_i^1}<1and{\displaystyle \sum _{j=1}^m{v}_j^2}<1;\end{array}$$

(10)

where

$$V_c^1=1-{{\displaystyle \sum }}_{i=1}^n{v}_i^1\mathrm{and}{V}_c^2=1-{{\displaystyle \sum }}_{j=1}^m{v}_j^2$$

Definition 6.

Ref. [11] (D numbers’ Integration) Let $D=\left\{\left(b_1,v_1\right),\left(b_2,v_2\right),\cdots ,\left(b_i,v_i\right),\cdots \left(b_n,v_n\right)\right\}$ be a D number, the integration representation of D is defined as

$$I\left(D\right)={\displaystyle \sum }_{i=1}^n{b}_i{v}_i$$

(11)

where$I\left(D\right)$is real number.

Example 1.

Suppose there is a D number: $D=\left\{\left(9, 0.4\right),\left(8, 0.2\right),\left(7, 0.3\right)\right\}$, then $I\left(D\right)=9\times 0.4+8\times 0.2+7\times 0.3=7.3$.

3. Proposed Method

In this section, we propose the D–SCRI–EDAS method, which takes into account incomplete information. D–SCRI method [19] is a new method proposed in 2021. This method can effectively avoid the problem of information distortion in the process of multiple combination and integration of D numbers, and solves the problem when multiple experts evaluate an evaluation object and some experts are unable to give accurate evaluation of all criteria due to the lack of knowledge and information, which leads to the problem of missing information, so D–SCRI is selected for expert opinion integration. In the process of program evaluation information integration, because the evaluation criteria of the program have certain conflicts, the EDAS method, which is effective for conflicting multi-attribute decision-making, is used for integration.

3.1. D Numbers Stepwise Comparison and Replacement Integration (SCRI) Method

In view of the incomplete information in the D numbers integration process, Wang et al. [18] put forward an improved D numbers integration method, but its result was biased due to the fact that the evaluation object was not considered globally. Based on Wang et al. [18], Hou and Zhao [19] proposed a SCRI integration method that not only considered the evaluation results of individual project, but also analyzed them horizontally with those of other projects. It can handle missing information in a more reasonable way, which is more efficient than previous integration methods. In this article, in order to supplement the D numbers of incomplete information into one with complete information, we have slightly modified the SCRI method and rephrased it as follows:

Suppose there are M Objects $O_1,O_2,\cdots ,O_M$ to be evaluated, and K experts are participating in the evaluation.

Step 1, As for the Object $O_i$, sort the evaluation level from each expert in descending order, and record the non-evaluation ones as 0, that is to say, the evaluation results of the experts on the Object $O_i$ are arranged as a row vector in descending order, which constitutes the original evaluation matrix $R$ with the order $M\times K$;

Step 2, observe whether there is a column in which elements are all 0. If so, go to step 3, otherwise, jump to step 4;

Step 3, Starting from the first column, check in turn whether element 0 emerges in any column. Here let us assume that 0 appears in the s + 1 column for the first time, that is, the elements corresponding to the previous s columns are initial integration value calculated from confirmed evaluations made by the experts, then the previous column elements in each row are added, and then average it, that is, the sum of each row is divided by s, which is called the basic evaluation reference value. As for the columns where all the column elements are 0, replace the 0 elements in this column with the basic evaluation reference value of the corresponding row. This is also equivalent to distributing this part of incomplete information proportionally to the results of the evaluation that have been clearly given in the preceding s columns.

Step 4, Check whether there is element 0 in each column. If yes, compare the non-zero elements of this column, and replace element 0 with the smallest value, That is to say, the incomplete information is replaced with the minimum evaluation value corresponding to the evaluation object through horizontal comparison, until all elements in the matrix are nonzero, which is called the modified integrated value matrix $R_{Int}^{*}$;

Step 5, According to the integration value matrix $R_{Int}^{*}$, write the D numbers of the complete information, and then calculate the final integration value according to the integration formula (11).

Example 2.

Assume that 10 experts are invited to evaluate the performance of 3 alternatives (A, B, C), and also suppose the evaluation system as a set $\Omega =\left\{1,2,3,4,5,6,7,8,9\right\}$. When these 10 experts are evaluating the performance of A, 4 persons choose “9”, 2 persons choose “8”, 3 persons choose “7” and 1 person does not give any evaluation since he is unfamiliar with A. Thus, A’s evaluation is expressed as D numbers $D_A=\left\{\left(9, 0.4\right),\left(8, 0.2\right),\left(7, 0.3\right)\right\}$. When the same 10 experts are evaluating the performance of B, 3 of them choose “9”, 5 choose “8”, and 2 do not evaluate as they are not familiar with B, which is then expressed in the form of D numbers $D_B=\left\{\left(9, 0.3\right),\left(8,0.5\right)\right\}$. When the same 10 experts are evaluating the performance of C, 4 choose “9” and 3 choose “8”, and 3 conduct no evaluations because they are not familiar with B, which is then expressed as in the form of D numbers $D_C=\left\{\left(9, 0.4\right),\left(8,0.3\right)\right\}$.

Step 1, Build an original evaluation matrix R= $\left(\begin{array}{cccccccccc}9& 9& 9& 9& 8& 8& 7& 7& 7& 0\\ 9& 9& 9& 8& 8& 8& 8& 8& 0& 0\\ 9& 9& 9& 9& 8& 8& 8& 0& 0& 0\end{array}\right)$

Step 2, Check whether there is a column whose column elements are all 0, where the last column is all 0, enter step 3;

Step 3, 0 appears in the eighth column for the first time, that is, the elements corresponding to the previous 7 columns are the interval integrated values calculated based on the clear evaluation given by the experts. Add the elements in each row of the first 7 columns, and then calculate the arithmetic mean, $59/7$, $59/7$, $60/7$, respectively, and replace the last column element of the interval integrated value matrix with these values, thus:

$$\left(\begin{array}{cccccccccc}9& 9& 9& 9& 8& 8& 7& 7& 7& 59/7\\ 9& 9& 9& 8& 8& 8& 8& 8& 0& 59/7\\ 9& 9& 9& 9& 8& 8& 8& 0& 0& 60/7\end{array}\right)$$

Step 4, Check whether element 0 appears in each column. If there is, compare the non-zero elements of this column, and replace element 0 with the smallest value. In the eighth column, $\min \left(7,8\right)=7$, replace 0 with 7, and in the ninth column, replace 0 with 7 to obtain the modified integrated value matrix $R_{Int}^{*}$:

$$R_{Int}^{*}=\left(\begin{array}{cccccccccc}9& 9& 9& 9& 8& 8& 7& 7& 7& 59/7\\ 9& 9& 9& 8& 8& 8& 8& 8& 7& 59/7\\ 9& 9& 9& 9& 8& 8& 8& 7& 7& 60/7\end{array}\right)$$

Step 5, According to the modified integrated value matrix $R_{Int}^{*}$, write the D numbers of complete information. For example, there are four 9s, three 8s, two 7s, and one 60/7 in the third row of the matrix. We can convert this row into complete informative D numbers ${\tilde{\mathrm{D}}}_{\mathrm{C}}=\left\{\left(9,0.4\right),\left(8,0.3\right),\left(7,0.2\right),\left(60/7,0.1\right)\right\}$, and according to the Formula (11), the integrated value $I\left({\tilde{\mathrm{D}}}_C\right)=9\times 0.4+8\times 0.3+7\times 0.2+60/7\times 0.1\approx 8.2571$.

3.2. D–SCRI–EDAS Method

The EDAS method was raised by Keshavarz Ghorabaee et al. [37] in 2015 to solve the MADM problem with conflicting attributes. The method is to calculate the average solution of each solution, firstly, and then measure the difference between each solution and the average value, which will be recorded as either the positive distance from average or the negative distance from average. Generally speaking, the optimal solution is the one with the largest positive distance from the average and the smallest negative distance from the average. In this section, we propose the D–SCRI–EDAS method to solve the decision-making problem in the context of incomplete information D numbers. After obtaining the D numbers evaluation matrix of each scheme, use the SCRI method to convert all D numbers in the evaluation matrix into the form of complete information and obtain the integrated value, which is called the score value of each attribute in each scheme. Do an arithmetic mean of these scores to obtain the average score value of each attribute, and then calculate the positive distance and negative distance of each attribute of each scheme from the average score value, hence the weighted positive distance and negative distance of each scheme is obtained according to the weight of each attribute. Finally, sort the normalized score values and make the best choice.

Suppose there are m alternatives $\{A_1,A_2,\cdots ,A_m\}$, n attributes $\{X_1,X_2,\cdots ,X_n\}$ and k experts $\{E_1,E_2,\cdots ,E_k\}$. Let $\{{\omega }_1,{\omega }_2,\cdots ,{\omega }_n\}$. be the attribute’s weighting vector which satisfy ${\omega }_i\in [0,1]$, and ${\displaystyle \sum _{i=1}^n{\omega }_i}=1$.

The D–SCRI–EDAS method works in the following seven steps:

Step 1. Construct the D numbers initial evaluation matrix $D={[D_{ij}]}_{m\times n}$; of which $D_{ij}$ represents the evaluation D numbers of the Attribute j of the Scheme i.

Step 2. Use the SCRI method to convert all the D numbers in the evaluation matrix into the form of complete information to obtain the D numbers matrix $\tilde{D}$ of complete information evaluation, and use the integrated Formula (11) to convert $\tilde{D}$ into a real number evaluation matrix $I={[I_{ij}]}_{m\times n}$;

Step 3. Compute the value of average integration value matrix $ASI={[AV{I}_j]}_{1\times n}$ of the n attributes:

$$ASI={[AV{I}_j]}_{1\times n}={[\frac{{\displaystyle {\sum }_{i=1}^m{I}_{ij}}}{m}]}_{1\times n}$$

(12)

Step 4. Calculate the positive distance matrix $PDA={[PD{A}_{ij}]}_{m\times n}$ and negative distance matrix $NDA={[ND{A}_{ij}]}_{m\times n}$ according to Equations (13)–(16):

If jth attribute is beneficial type,

$$PD{A}_{ij}=\frac{\max (0,I_{ij}^{}-AV{I}_j)}{AV{I}_j}$$

(13)

$$ND{A}_{ij}=\frac{\max (0,AV{I}_j-I_{ij}^{})}{AV{I}_j}$$

(14)

If jth attribute is cost type,

$$PD{A}_{ij}=\frac{\max (0,AV{I}_j-I_{ij}^{})}{AV{I}_j}$$

(15)

$$ND{A}_{ij}=\frac{\max (0,I_{ij}^{}-AV{I}_j)}{AV{I}_j}$$

(16)

Step 5. Calculate the values of $WS{P}_i$ and $WS{N}_i$ which denotes the weighted sum of PDA and NDA, the computing formulae are provided as follows:

$$WS{P}_i={\displaystyle \sum _{j=1}^n{\omega }_j}\cdot PD{A}_{ij}, WS{N}_i={\displaystyle \sum _{j=1}^n{\omega }_j}\cdot ND{A}_{ij}$$

(17)

Step 6. The results of Equation (17) can be normalized as:

$$NWS{P}_i=\frac{WS{P}_i}{\max (WS{P}_i)}, NWS{N}_i=1-\frac{WS{N}_i}{\max (WS{N}_i)}$$

(18)

Step 7. Calculate the final evaluation score $FE{S}_i$ according to $NWS{P}_i$ and $NWS{N}_i$ The larger the value is, the better the alternative is.

$$FE{S}_i=\frac{1}{2}(NWS{P}_i+NWS{N}_i)$$

(19)

The stepwise representation of the proposed method is given as follows. The flowchart of the proposed method is shown in Figure 1.

4. The Numerical Example and Comparative Analysis

4.1. A Numerical Example

During the rapid spreading time of COVID-19, online learning has emerged as a new way to prevent people from gathering and to contain the spread of the virus. Online live course platforms play an important role in online study, and they rely heavily on some remote office and conference software. However, these software cannot fully meet the needs of schools’ online live classes due to the original design purpose and the nature of the software. Because of this, the user experience evaluation and optimization of the online live course platform is particularly important [48,49]. For example, Wang et al. [49] adopted a combination of qualitative and quantitative methods, and from the perspective of user experience, integrated the data obtained through literature research, interviews, and network research to set evaluation indicators. Delphi method and AHP method are used to determine the weight of each index, and to set the rating scale. The t-test method is used to establish a comparative analysis method of model application, design a standardized process, and build a complete evaluation system. However, there is a major problem that is not well addressed in Wang et al. [49]: when evaluating different platforms, some experts may not be able to give clear ratings because they seldom know the rating items, and in such cases a reluctantly determined rating may greatly affect the final evaluation result. Instead, our proposed D numbers–SCRI method can solve this problem very well, and can deal with the uncertainty, ignorance and missing information that may occur, in a simple and effective way. In Wang et al. [49], the main factors affecting the user experience of the online live course platform are divided into four first-level indicators, namely ease of operation, functional completeness, interface rationality, and technical reliability. Each of the first-grade indicators can be divided into several secondary ones. The hierarchical structure and corresponding weights in the user experience evaluation of the online live course platform are given in Figure 2.

Step 1. Construct the initial D numbers initial evaluation matrix $D={[D_{ij}]}_{m\times n}$ (see

Table 2. Assessment data for Ding Talk, Tencent Classroom and Tencent QQ are represented by D numbers.

Basic Attribute	Ding Talk	Tencent Classroom	Tencent QQ
Y1	{(9, 0.2), (8, 0.5), (7, 0.3)}	{(8, 0.5), (7,0.2), (6, 0.2), (5, 0.1)}	{(7, 0.5), (6, 0.2), (5, 0.2), (4,0.1)}
Y2	{(9, 0.2), (8, 0.5), (6, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.2), (5, 0.2)}	{(8, 0.2), (7, 0.3), (6, 0.1), (5, 0.1)}
Y3	{(9, 0.2), (8, 0.2), (7, 0.3), (6, 0.1)}	{(8, 0.4), (7, 0.3), (6, 0.2)}	{(8, 0.2), (7, 0.4), (5, 0.1)}
Y4	{(8, 0.2), (7, 0.2), (6, 0.4)}	{(8, 0.2), (7, 0.2), (6, 0.6)}	{(7, 0.2), (6, 0.4), (5, 0.2), (4, 0.1)}
Y5	{(9, 0.6), (8, 0.2), (7, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.1), (5, 0.4)}	{(8, 0.4), (7, 0.2), (5, 0.2)}
Y6	{(7, 0.2), (6, 0.2), (5, 0.1), (4, 0.2)}	{(7, 0.2), (5, 0.4), (4, 0.2)}	{(9, 0.2), (8, 0.2), (7, 0.3), (5, 0.1)}
Y7	{(8, 0.2), (6, 0.2), (5, 0.4)}	{(9, 0.2), (7, 0.2), (6, 0.4)}	{(7, 0.4), (6, 0.2), (5, 0.2)}
Y8	{(9, 0.4), (8, 0.4), (6, 0.2)}	{(9, 0.2), (8, 0.3), (6, 0.2), (5, 0.2)}	{(8, 0.4), (7, 0.1), (5, 0.2), (4, 0.1)}
Y9	{(9, 0.1), (8, 0.4), (6, 0.3)}	{(8, 0.1), (7, 0.6), (6, 0.1)}	{(8, 0.4), (7, 0.3), (4, 0.2)}
Y10	{(9, 0.3), (7, 0.2), (6, 0.4)}	{(8, 0.5), (7, 0.2), (6, 0.2)}	{(9, 0.2), (8, 0.6)}
Y11	{(6, 0.6), (5, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.2),(4, 0.1), (3, 0.1)}	{(8, 0.4), (6, 0.2), (4, 0.1)}
Y12	{(8, 0.3), (7, 0.4), (5, 0.1)}	{(9, 0.2), (8, 0.2), (6, 0.3), (5, 0.1)}	{(8, 0.3), (7, 0.2), (6, 0.2), (5, 0.1)}
Y13	{(9, 0.4), (8, 0.2), (7, 0.2), (5, 0.2)}	{(8, 0.4), (7, 0.2), (6, 0.2), (5, 0.2)}	{(9, 0.2), (7, 0.4), (6, 0.2)}
Y14	{(9, 0.2), (8, 0.4), (6, 0.3)}	{(7, 0.4), (6, 0.4), (5, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.2), (5, 0.1), (4, 0.1)}
Y15	{(9, 0.2), (8, 0.4), (7, 0.2), (5, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.6)}	{(8, 0.2), (7, 0.2), (6, 0.2), (5, 0.4)}
Y16	{(8, 0.4), (7, 0.1), (6, 0.4)}	{(8, 0.3), (7, 0.2), (6, 0.4)}	{(7, 0.5), (6, 0.1), (5, 0.3)}

).

Step 2. Use the SCRI method to convert all the D numbers in the evaluation matrix into the form of complete information to obtain the D numbers matrix $\tilde{D}$ of complete information evaluation (see

Table 3. Assessment data for Ding Talk, Tencent Classroom and Tencent QQ are represented by complete information D numbers.

Basic Attribute	Ding Talk	Tencent Classroom	Tencent QQ
Y1	{(9, 0.2), (8, 0.5), (7, 0.3)}	{(8, 0.5), (7,0.2), (6, 0.2), (5, 0.1)}	{(7, 0.5), (6, 0.2), (5, 0.2), (4,0.1)}
Y2	{(9, 0.2), (8, 0.5), (6, 0.2), (58/7, 0.1)}	{(8, 0.2), (7, 0.2), (6, 0.3), (5, 0.2), (47/7, 0.1)}	{(8, 0.2), (7, 0.3), (6, 0.2), (5, 0.2), (48/7, 0.1)}
Y3	{(9, 0.2), (8, 0.2), (7, 0.3), (6, 0.2), (55/7,0.1)}	{(8, 0.4), (7, 0.3), (6, 0.2), (53/7, 0.1)}	{(8, 0.2), (7, 0.4), (6, 0.2), (5, 0.1), (49/7, 0.1)}
Y4	{(8, 0.2), (7, 0.2), (6, 0.5), (4, 0.1)}	{(8, 0.2), (7, 0.2), (6, 0.6)}	{(7, 0.2), (6, 0.5), (5, 0.2), (4, 0.1)}
Y5	{(9, 0.6), (8, 0.2), (7, 0.2)}	{(8, 0.2), (7, 0.3), (6, 0.1), (5, 0.4)}	{(8, 0.4), (7, 0.3), (5, 0.3)}
Y6	{(7, 0.2), (6, 0.2), (5, 0.1), (4, 0.3), (39/7, 0.2)}	{(7, 0.2), (5, 0.4), (4, 0.2), (38/7, 0.2)}	{(9, 0.2), (8, 0.2), (7, 0.3), (5, 0.1), (55/7, 0.2)}
Y7	{(8, 0.2), (6, 0.4), (5, 0.4)}	{(9, 0.2), (7, 0.4), (6, 0.4)}	{(7, 0.4), (6, 0.2), (5, 0.2), (50/8, 0.2)}
Y8	{(9, 0.4), (8, 0.4), (6, 0.2)}	{(9, 0.2), (8, 0.3), (6, 0.3), (5, 0.2)}	{(8, 0.4), (7, 0.1), (6, 0.1), (5, 0.3), (4, 0.1)}
Y9	{(9, 0.1), (8, 0.4), (6, 0.3), (4, 0.1), (59/8, 0.1)}	{(8, 0.1), (7, 0.7), (6, 0.1), (4, 0.1)}	{(8, 0.4), (7, 0.3), (4, 0.2), (57/8, 0.1)}
Y10	{(9, 0.3), (7, 0.2), (6, 0.4), (59/8, 0.1)}	{(8, 0.5), (7, 0.2), (6, 0.2), (60/8, 0.1)}	{(9, 0.2), (8, 0.6), (6, 0.1), (66/8, 0.1)}
Y11	{(6, 0.6), (5, 0.2), (41/7, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.2),(4, 0.1), (3, 0.1), (46/7, 0.2)}	{(8, 0.4), (6, 0.2), (4, 0.1), (3, 0.1), (48/7, 0.2)}
Y12	{(8, 0.3), (7, 0.4), (5, 0.1), (57/8, 0.2)}	{(9, 0.2), (8, 0.2), (6, 0.3), (5, 0.1), (57/8, 0.2)}	{(8, 0.3), (7, 0.2), (6, 0.2), (5, 0.1), (55/8, 0.2)}
Y13	{(9, 0.4), (8, 0.2), (7, 0.2), (5, 0.2)}	{(8, 0.4), (7, 0.2), (6, 0.2), (5, 0.2)}	{(9, 0.2), (7, 0.4), (6, 0.2), (5, 0.2)}
Y14	{(9, 0.2), (8, 0.4), (6, 0.3), (5, 0.1)}	{(7, 0.4), (6, 0.4), (5, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.2), (5, 0.3), (4, 0.1)}
Y15	{(9, 0.2), (8, 0.4), (7, 0.2), (5, 0.2)}	{(8, 0.2), (7, 0.2), (6, 0.6)}	{(8, 0.2), (7, 0.2), (6, 0.2), (5, 0.4)}
Y16	{(8, 0.4), (7, 0.1), (6, 0.4), (63/9, 0.1)}	{(8, 0.3), (7, 0.2), (6, 0.4), (62/9, 0.1)}	{(7, 0.5), (6, 0.1), (5, 0.3), (56/9, 0.1)}

), and convert $\tilde{D}$ into a real number evaluation matrix $I={[I_{ij}]}_{m\times n}$ (see

Table 4. Integration of the assessment results.

Basic Attribute	Ding Talk	Tencent Classroom	Tencent QQ
Y1	7.9000	7.1000	6.1000
Y2	7.8286	6.4714	6.5857
Y3	7.4857	7.2571	6.8000
Y4	6.4000	6.6000	5.8000
Y5	8.4000	6.3000	6.8000
Y6	5.4143	5.2857	7.5714
Y7	6.0000	7.0000	6.2500
Y8	8.0000	7.0000	6.4000
Y9	7.0375	6.7000	6.8125
Y10	7.2375	7.3500	8.0250
Y11	5.7714	6.2143	6.4714
Y12	7.1250	7.1250	6.8750
Y13	7.6000	6.8000	6.8000
Y14	7.3000	6.2000	6.1000
Y15	7.4000	6.6000	6.2000
Y16	7.0000	6.8889	6.2222

).

Step 3. Compute the value of average integration value matrix $ASI={[AV{I}_j]}_{1\times n}$ of the n attributes (see

Table 5. Average integration value of each attribute.

Basic Attribute	Average Integration Value	Basic Attribute	Average Integration Value
Y1	7.0333	Y9	6.8500
Y2	6.9619	Y10	7.5375
Y3	7.1809	Y11	6.1524
Y4	6.2667	Y12	7.0417
Y5	7.1667	Y13	7.0667
Y6	6.0905	Y14	6.5333
Y7	6.4167	Y15	6.7333
Y8	7.1333	Y16	6.7037

).

Step 4. Because the attributes are both beneficial types, calculate the positive distance matrix $PDA={[PD{A}_{ij}]}_{m\times n}$ and negative distance matrix $NDA={[ND{A}_{ij}]}_{m\times n}$ according to Equations (13) and (14):

$$PDA=\left(\begin{array}{cccccccccccccccc}0.1232& 0.1245& 0.0424& 0.0213& 0.1721& 0& 0& 0.1215& 0.0274& 0& 0& 0.0118& 0.0755& 0.1174& 0.0990& 0.0442\\ 0.0095& 0& 0.0106& 0.0532& 0& 0& 0.0909& 0& 0& 0& 0.0101& 0.0118& 0& 0& 0& 0.0276\\ 0& 0& 0& 0& 0& 0.2431& 0& 0& 0& 0.0647& 0.0518& 0& 0& 0& 0& 0\end{array}\right)$$

$$NDA=\left(\begin{array}{cccccccccccccccc}0& 0& 0& 0& 0& 0.1110& 0.0649& 0& 0& 0.0398& 0.0619& 0& 0& 0& 0& 0\\ 0& 0.0705& 0& 0& 0.1209& 0.1321& 0& 0.0187& 0.0219& 0.0249& 0& 0& 0.0377& 0.0510& 0.0198& 0\\ 0.1327& 0.0540& 0.0530& 0.0745& 0.0512& 0& 0.0260& 0.1028& 0.0055& 0& 0& 0.0237& 0.0377& 0.0663& 0.0792& 0.0718\end{array}\right)$$

Step 5. Calculate the values of $WS{P}_i$ and $WS{N}_i$ (see

Table 6. WSP and WSN in three platforms.

	Ding Talk	Tencent Classroom	Tencent QQ
WSP	WSP1 = 0.0741	WSP2 = 0.0122	WSP3 = 0.0145
WSN	WSN1 = 0.0102	WSN2 = 0.0297	WSN3 = 0.0609

) according to Equation (17).

Step 6. The results of Table 6 can be normalized as follows (see

Table 7. NWSP and NWSN in three platforms.

	Ding Talk	Tencent Classroom	Tencent QQ
NWSP	NWSP1 = 1	NWSP2 = 0.1646	NWSP3 = 0.1957
NWSN	NWSN1 = 0.8325	NWSN2 = 0.5123	NWSN3 = 0

) according to Equation (18).

Step 7. Calculate the final evaluation score $FE{S}_i$ according to Equation (19). The larger the value is, the better the alternative is for the final evaluation score:

$$FE{S}_1=0.9163,FE{S}_2=0.3385,FE{S}_3=0.0979$$

Thereby, we consider that object $\mathrm{Ding}\mathrm{Talk}\gg \mathrm{Tencent}\mathrm{Classroom}\gg \mathrm{Tencent}\mathrm{QQ}$, in which the symbol ‘≫’ means ‘superior’. Hou and Zhao [19] used D numbers to collect evaluation information, and used the SCRI method to convert the evaluation information into real numbers to calculate the final evaluation scores of the three live broadcast platforms with the weight of each attribute considered. In order to compare it with the D–EDAS method in this paper, we normalized relevant scores by using the maximum normalization method and listed these in

Table 8. Evaluation scores and normalized scores of the three platforms with two methods.

Score Type	Method	Ding Talk	Tencent Classroom	Tencent QQ
Evaluation scores	Hou and Zhao [19]	7.2508	6.6856	6.4891
	D-EDAS (this paper)	0.9163	0.3385	0.0979
Normalized scores	Hou and Zhao [19]	1	0.9220	0.8949
	D-EDAS (this paper)	1	0.3694	0.1068

and Figure 3.

In order to verify the effectiveness of the method, we adopted the following means of sensitivity analysis: the weights of 16 basic attributes were randomly changed (see

Table 9. The weight of the evaluation attributes obtained randomly.

Basic Attribute	The Weight of the First Random Change	The Weight of the Second Random Change
Y1	0.1027	0.0787
Y2	0.1511	0.0319
Y3	0.0719	0.0526
Y4	0.0222	0.0985
Y5	0.0227	0.0679
Y6	0.0289	0.0625
Y7	0.0017	0.0901
Y8	0.1213	0.0767
Y9	0.0797	0.0891
Y10	0.0823	0.0432
Y11	0.0476	0.0117
Y12	0.0285	0.0356
Y13	0.1237	0.0732
Y14	0.0317	0.0657
Y15	0.0571	0.0892
Y16	0.0269	0.0334

), the D–SCRI–EDAS method and the D–SCRI method proposed in [19] were, respectively, compared. The evaluation order of the two methods was the same, and the results obtained by the D–SCRI–EDAS method were more sensitive than those obtained by the D–SCRI method (see

Table 10. Normalized evaluation results after two weight adjustments.

Evaluation Method	Ding Talk	Tencent Classroom	Tencent QQ
D-SCRI/1	1.0000	0.9125	0.8981
D-SCRI-EDAS/1	1.0000	0.2763	0.1064
D-SCRI/2	1.0000	0.9354	0.9150
D-SCRI-EDAS/2	1.0000	0.3905	0.1655

).

As the gap between the ranks increases without changing the sort, it suggests that the second method is more sensitive to data differences and can handle more precise evaluation problems. This also indicates that the D–EDAS method is more effective than simply using D numbers in a MADM evaluation of incomplete information, showing more method advantages.

4.2. Comparative Analysis

Refs. [21,27] used the combination of D number theory and the VIKOR method to solve the problem of medicine provider selection. In this paper, the method is extended to D–SCRI–VIKOR method to evaluate the user experience of the online course live platform. In order to further demonstrate the effectiveness of the D–SCRI–EDAS method in dealing with multi-attribute decision-making problems, we compared and analyzed the evaluation results obtained by two methods.

The D–SCRI–VIKOR method is briefly described as follows:

Step 1~Step 2, The same as D–SCRI–EDAS method, a D-number evaluation matrix is transformed into a complete information D-number matrix, and then a real number evaluation matrix is obtained. In order to be consistent with the expression habits of the VIKOR method, the real number evaluation matrix is denoted as ${[f_{ij}]}_{m\times n}$;

Step 3, Identify the best values $f_j^{*}$ and worst values $f_j^{-}$ of each positive attribute by using Equations (20) and (21):

$$f_j^{*}=\underset{i}{\max }{f}_{ij};i=1,\cdots ,m,j=1,\cdots ,n$$

(20)

$$f_j^{-}=\underset{i}{\min }{f}_{ij};i=1,\cdots ,m,j=1,\cdots ,n$$

(21)

For the negative attribute, the indexes are computed using Equations (22) and (23):

$$f_j^{*}=\underset{i}{\min }{f}_{ij};i=1,\cdots ,m,j=1,\cdots ,n$$

(22)

$$f_j^{-}=\underset{i}{\max }{f}_{ij};i=1,\cdots ,m,j=1,\cdots ,n$$

(23)

Step 4, Let $\{{\omega }_1,{\omega }_2,\cdots ,{\omega }_n\}$ be the attribute’s weighting vector which satisfies ${\omega }_i\in [0,1]$, and ${\displaystyle \sum _{i=1}^n{\omega }_i}=1$. The $S_i$ and $R_i$ indexes are obtained for each alternative using Equations (24) and (25):

$$S_i={\displaystyle \sum _{j=1}^n{w}_j\cdot }\frac{(f_j^{*}-f_{ij})}{(f_j^{*}-f_j^{-})};i=1,\cdots ,m$$

(24)

$$R_i=\underset{j}{\max }[w_j\frac{(f_j^{*}-f_{ij})}{(f_j^{*}-f_j^{-})}];i=1,\cdots ,m,j=1,\cdots ,n$$

(25)

Step 5, The index $Q_i$ is calculated for each alternative as in Equation (26):

$$Q_i=v\cdot \frac{(S_i-S^{*})}{(S^{-}-S^{*})}+(1-v)\cdot \frac{(R_i-R^{*})}{(R^{-}-R^{*})};i=1,\cdots ,m$$

(26)

where $S^{*}=\underset{i}{\min }{S}_i,S^{-}=\underset{i}{\max }{S}_i,R^{*}=\underset{i}{\min }{R}_i,R^{-}=\underset{i}{\max }{R}_i$, $v=0.5$ indicates the strategy of maximum group utility.

Step 6, Rank the alternatives sorting by values S, R and Q in an ascending order.

The final evaluation results are shown in

Table 11. The values of S, R and Q for each alternative.

The Ranking Index	Ding Talk	Tencent Classroom	Tencent QQ
S	0.1819	0.5619	0.8236
R	0.0756	0.1233	0.1466
Q	0	0.6298	1

and

Table 12. The ascending ranks of the alternatives.

The Ranking Index	The Ascending Ranks
S	Ding Talk ≫ Tencent Classroom ≫ Tencent QQ
R	Ding Talk ≫ Tencent Classroom ≫ Tencent QQ
Q	Ding Talk ≫ Tencent Classroom ≫ Tencent QQ

.

A compromise solution is obtained based on the three ranking lists, that is, Ding Talk $\gg$ Tencent Classroom $\gg$ Tencent QQ.

It can be seen from the results that the combination of D–SCRI and EDAS or VIKOR all obtain the same ranking result. However, the VIKOR method needs to comprehensively consider the comprehensive ranking results of the three indicators S, R and Q, especially when the ranking results of the three indicators are inconsistent; this may cause confusion in the comprehensive evaluation, and the VIKOR method requires a relatively large amount of calculation. Therefore, the combination of D–SCRI and EDAS has better applicability to the evaluation problems that require the participation of more evaluation experts and conflicting attributes.

5. Conclusions

This paper proposes an extended EDAS method to solve multi-attribute decision-making problems with incomplete evaluation information. D numbers are used to collect experts’ evaluation information, while the extended D–EDAS method is used to rank the alternatives. Finally, it is applied to solve the user experience evaluation problem in the platforms that offers online course live broadcasts. The implementation steps of the proposed method and the realization of specific situations are introduced in detail. To verify the stability and practicability of the method, we compare the computational results with the D numbers method of Hou and Zhao [19], and the experimental results demonstrate the good feasibility and superiority of the method. Admittedly, the limitation of this study is that the information filled by the SCRI method is only an inference of the missing part, which may not be strictly correct, and may not fully consider the weight of information of experts as well.

In future work, we can apply this method to other fields, such as green supplier selection, new energy sustainability assessment, and more. Furthermore, the simple operation of the EDAS method allows further research into uncertain and incomplete information environments, such as two-dimensional belief function [50] and Z-probabilistic linguistic information [51], amongst others.