Applying the DEMATEL−ANP Fuzzy Comprehensive Model to Evaluate Public Opinion Events

Abstract

Network public opinion is a mirror reflecting people’s will, and evaluating its urgency can help to find hidden social crises. Research on public opinion in the field of machine learning usually focuses on micro-sentiment judgment, which is unable to offer support for the evaluation of public opinion events without additional data, and research from the perspective of artificial weighting has the disadvantage of the confusion of explanation. Judging the urgency of public opinion events is usually based on human perception, which is fuzzy and conforms to the attribute of fuzzy mathematics. Therefore, the index system in this paper was constructed in line with five principles, from which the weights were scientifically evaluated by integrating the DEMATEL and ANP model, and fuzzy mathematics was applied to determine the urgency level of public opinion. The result has three-fold significance. First, the index system constructed was more closely linked. Second, the integration of the DEMATEL and ANP weight calculating model took the interdependence of indicators fully into account. Third, fuzzy mathematics provided support for determining the public opinion crisis level, especially in the absence of immediate dissemination data.

1. Introduction

Leaders of nations, managers of big companies, and even heads of small families all hope to prevent small problems from developing into big ones, and the best way is to pay close attention to public opinion. Public opinion is a mirror reflecting people’s will, and its development is often driven by the inner demands of netizens, triggering many public opinion incidents when mishandled, which inevitably give rise to adverse reactions from the public. The evaluation of the urgency of online public opinion can help to discover hidden social crises, quickly focus on the topics most concerning the public, understand people’s demands, and quell public grievances in a timely manner. The judgment of public opinion events is usually based on human perception, the core of which is guessing and analyzing the development trend of public opinion and evaluating its degree of urgency.

Network public opinion has been studied by many researchers to discern its law of dissemination, evolution, forecast, and propagation influence, and some mathematical and machine learning models have been discussed. The LDA-ARMA deep neural network for the dynamic presentation of public opinion events was proposed in [1], and the back propagation neural network based on the genetic algorithm (GA-BP) was employed to establish a network public opinion early warning model in [2] and an SIR model in [3]; however, the LDA-ARMA model, GA-BP, SIR model, and the Logistic model [4] utilized to classify the sentiments are too microcosmic to reflect the full picture of public opinion events, which demand a large amount of data and usually take hundreds or even thousands of learning sessions to converge even for a very simple question, and it is unable to offer support to the evaluation of public opinion events without data. The G-GERT network model of online public opinion reversal based on kernel and grey degree was discussed in [5]; however, using the grey degree might limit the samples. The MIMIC-CUB model was proposed to estimate citizens’ perception of the state of European economies in [6]; however, the formative part of the MIMIC model could cause confusion in interpretation. The AHP artificial weighting in [7,8] ignores the interdependence of indicators. It was found that research on public opinion in the field of machine learning usually focuses on micro-sentiment judgment, which is unable to offer support for the evaluation of public opinion events without data and cannot evaluate events integrally. Current research from the perspective of mathematical statistics and artificial weighting has the disadvantages of sample limitation and confusion in interpretation.

Given the above, the present paper took the fuzzy mathematical method, which cannot be limited by the number of samples and can be easily interpreted, to construct a public opinion evaluation model integrated with a decision-making model. This paper has the following structure: we first describe the construction of the index system of network public opinion based on the principles of it being easy to obtain, quantifiable, and correlated. Then, we describe the combination of the DEMATEL and ANP, which determine the weights of indicators, in which the analysis results of the DEMATEL were taken as the input of the ANP; thus, the interrelationship of the indicators was taken into consideration completely. Then, we describe how fuzzy mathematics was applied, based on which, a DEMATEL−ANP fuzzy comprehensive evaluation model was constructed and organized, as shown in Figure 1.

2. Construction of a Network Public Opinion Evaluation Index System

Establishing a reasonable index system is the prerequisite for judging the degree of urgency of public opinion, but the current ones are cumbersome. Some of them lack a consideration of feasibility, making it difficult to extract a certain indicator from a large amount of data and affecting the overall assessment accuracy; some rely on human subjective assignments, which may cause one-sided results; and some rely too much on the complete data of the whole process of a certain public opinion event, making it difficult to be applied when the amount of collected data is insufficient at the early stage of public opinion fermentation.

This paper takes the laws and changes related to network public opinion transmission fully into account when emergent events occur. By referring to research [9,10,11,12,13,14,15,16,17,18] and consulting with relevant experts who have published papers in the field of public opinion, an index system of 5 first-level indicators and 13 secondary-level indicators was eventually established for emergent events after repeated rectification and selection. All the indicators in the index system are within the acquired capability and meet the construction principles of being important, professional, easy to achieve, quantifiable, and correlated with each other, helping to compose a scientific network public opinion index system. The principles of constructing the index system are shown in

Table 1. Principles for constructing a public opinion evaluation index system.

Principles	Description
Principle of profession	The screening and selection should be consistent with the research topic and should conform to the workflow of public opinion.
Principle of practicality	Instead of being limited to a theoretical scope, the indicators should be able to be quantified and easy to obtain.
Principle of quantification	Conceptualized indicators are difficult to quantify when converted into data. Thus, the indicators screened should be quantified or able to be converted to quantified data.
Principle of correlation	The indicators should have internal connections to a certain extent to make sure the built index system is systematical.
Principle of importance	According to the existing research, scholars have constructed their public opinion index systems with different goals, resulting in a huge number of influencing factors and causing the problem of redundancy and overlapping, based on which, selected indicators should be precise and important.

, and the whole network public opinion evaluation index system for emergent events is shown in

Table 2. The index system of public opinion evaluation.

The First Level	The Second Level
Degree of heat U1	Number of issuances U11
	Duration of concern U12
Degree of quantity U2	Number of clicks U21
	Number of shares U22
	Number of comments U23
Degree of strength U3	Number of opinion leaders U31
	Network area distribution U32
Degree of focus U4	Negative opinion holding rate U41
	Neutral opinion holding rate U42
	Positive opinion holding rate U43
Degree of variation U5	Click growth rate U51
	Share growth rate U52
	Comment growth rate U53

.

As is shown in Table 2, the index system has a two-tier hierarchy. The indicators at the second level are the individual elements of the overarching indicators set at the first level. The first level consists of five degrees referring to five dimensions of public opinion events: degree of heat, degree of quantity, degree of strength, degree of focus, and degree of variation. The degree of heat is measured by the volume of media issuance and the length of time netizens are concerned about the issue. The degree of quantity represents the degree of participation of netizens in the discussion of public opinion topics, which is mainly reflected in the aspects of the number of clicks, shares, and comments. The degree of strength characterizes the diffusion trend of public opinion in the transmission process, which is explained by the number of opinion leaders and the network area distribution. The degree of focus concerns the emotion and attitude of netizens, which can be counted as different opinion holding rates. The degree of variation reflects the dynamic change in the transmission process explained as the click growth rate, the share growth rate, and the comment growth rate.

3. Research Method

3.1. Calculate Indicator Weights Using the DEMATEL−ANP Model

In some of the existing articles, the AHP used by researchers to determine the weights often ignores the interdependence between indicators. Although using ANP to determine the weights is more scientific, using the ANP alone may result in the problem of the relationship between indicators being highly ambiguous. Research using the DEMATEL−ANP model often obtains the weights individually and then derives their final mixed weights, which requires experts to repeat the evaluation of the same system twice, which does not take full advantage of the method.

The DEMATEL can solve complex problems and sort out the relationships among various influencing factors [19]. The ANP method is used to normalize the matrix relationship generated by the influencing factors to test the weights among influencing factors [20]. Therefore, in this study, we used DEMATEL to enhance our understanding of the relationships between indicators whose purpose was to determine the causality of the influencing factors using the traditional DEMATEL method; at the same time, we used the ANP to calculate the weights of various aspects and influencing factors [19]. The DEMATEL is designed on a scoring scale of 0 to 4 (0: none; 1: very weak; 2: normal; 3: strong; 4: very strong), and 10 experts who have published papers in the field of communication and the study of public opinion were asked to score the importance of factors related to the dimensions in the questionnaire, from which the sum was used to analyze the degree of association between the indicators. The main procedures were as the follows.

(1)

Generate the direct relation matrix.

(i)

$a_{ij}$ = impacting factor, in which $a_{ij}$ stands for the degree of the impacting criteria.

(ii)

Impact range = (0: none; 1: very weak; 2: normal; 3: strong; 4: very strong).

(iii)

$A=\left[\begin{array}{ccc}0& a_{1j}& a_{1n}\\ a_{i1}& 0& a_{in}\\ a_{n1}& a_{nj}& 0\end{array}\right]$, where $a_{ij}$ denotes the average value.

(iv)

As there are n number of criteria, the direct relation matrix is marked as

$$A={\left(a_{ij}\right)}_{n\times n}.$$

(2)

Calculate the normalized matrix.

Matrix N is obtained by normalizing the direct relation matrix A.

$$N=A/(\underset{1\le i\le n}{max}\sum _{j=1}^n{a}_{ij})$$

(1)

(3)

Calculate the total relation matrix.

The total relation matrix T is calculated with the help of the MATLAB software tool, where N is the normalized matrix, I is the unit matrix, and ${\left(I-N\right)}^{-1}$ is the inverse matrix of $\left(I-N\right)$.

$$T=N+N^2+\cdots N^{\mathrm{K}}=\sum _{K=1}^{\mathrm{\infty }}{N}^{\mathrm{K}}=N{\left(I-N\right)}^{-1}$$

(2)

This study applied the 5-level scale to label the relationship between factors (0: none; 1: very weak; 2: normal; 3: strong; 4: very strong) by means of expert scoring to construct the direct relation matrix A. With the help of MATLAB software, the direct relation matrix was normalized Formula (1), from which the total relation matrix T was calculated using Formula (2), as displayed in

Table 3. Direct relation matrix and total relation matrix (A/T).

A/T	U11	U12	U21	U22	U23	U31	U32	U41	U42	U43	U51	U52	U53
U11	0/0.17	4/0.24	4/0.24	3/0.23	4/0.28	3/0.21	3/0.17	3/0.26	3/0.24	3/0.24	4/0.27	3/0.23	3/0.22
U12	4/0.22	0/0.11	3/0.18	3/0.19	2/0.19	1/0.13	1/0.10	2/0.19	2/0.17	2/0.17	3/0.20	2/0.17	2/0.16
U21	4/0.27	3/0.22	0/0.15	4/0.26	4/0.28	3/0.21	2/0.15	2/0.24	2/0.21	2/0.21	4/0.28	4/0.26	4/0.24
U22	4/0.25	3/0.20	3/0.2	0/0.15	4/0.26	3/0.20	1/0.12	3/0.24	1/0.17	1/0.17	3/0.24	4/0.24	2/0.19
U23	2/0.18	1/0.13	0/0.11	3/0.19	0/0.14	4/0.20	3/0.14	4/0.24	3/0.20	3/0.20	3/0.20	2/0.17	4/0.2
U31	2/0.16	0/0.10	1/0.12	3/0.17	4/0.22	0/0.09	1/0.09	4/0.22	2/0.16	2/0.16	3/0.19	2/0.15	1/0.12
U32	1/0.07	1/0.06	2/0.08	1/0.07	1/0.07	2/0.08	0/0.03	0/0.05	0/0.04	0/0.04	0/0.05	0/0.04	0/0.04
U41	3/0.21	3/0.19	3/0.18	2/0.18	4/0.24	2/0.16	0/0.08	0/0.16	4/0.23	4/0.23	3/0.22	3/0.20	2/0.17
U42	0/0.02	0/0.02	0/0.02	0/0.02	0/0.03	0/0.02	0/0.01	4/0.13	0/0.04	4/0.13	0/0.02	0/0.02	0/0.02
U43	0/0.02	0/0.02	0/0.02	0/0.02	0/0.03	0/0.02	0/0.01	4/0.13	4/0.13	0/0.04	0/0.02	0/0.02	0/0.02
U51	4/0.23	3/0.19	4/0.21	2/0.18	2/0.20	1/0.14	2/0.13	2/0.20	2/0.18	2/0.18	0/0.15	3/0.20	3/0.19
U52	3/0.20	3/0.19	2/0.16	4/0.22	3/0.22	1/0.13	2/0.13	2/0.20	2/0.18	2/0.18	3/0.21	0/0.13	3/0.19
U53	2/0.19	3/0.19	2/0.16	2/0.18	4/0.25	3/0.18	1/0.11	2/0.20	2/0.18	2/0.18	4/0.24	4/0.22	0/0.12

.

(4)

ANP network relationship diagram.

Based on the correlation analysis of the indicators in the DEMATEL model, the ANP network structure was drawn, as shown in Figure 2. The ANP divided the factors in the system into two parts: the control layer formed by the target layer and criterion layer, and the network layer constituted by the second level indicators, which are interdependent.

(5)

Generate the weighted matrix.

(i)

Let Tm = Um, where Um represents the unweighted matrix and Tm denotes the total relation matrix.

(ii)

Wm = NTm, passing Tm as the argument, NTm represents the normalized total relation matrix.

(6)

Generate the limit hyper matrix.

LW_m (limit hyper matrix) = $W_m^n$, where n denotes the W_mth power. The total relation matrix (similar to the ANP unweighted matrix) in DEMATEL was used as the input to the ANP model, which is normalized by Formula (3) to obtain the weighted hyper matrix ω. In the ANP, to reflect the stable influence relationship among the indicator factors, the weighted hyper matrix needed to be stabilized, which was solved for the limit relative ranking vector by using the power method (

Table 4. Weighted hyper matrix.

Weighted Hyper Matrix	U11	U12	U21	U22	U23	U31	U32	U41	U42	U43	U51	U52	U53	Limit Sorting Vectors
U11	0.08	0.13	0.13	0.11	0.12	0.12	0.14	0.11	0.11	0.11	0.12	0.11	0.12	0.11
U12	0.10	0.06	0.10	0.09	0.08	0.07	0.08	0.08	0.08	0.08	0.09	0.08	0.09	0.08
U21	0.12	0.12	0.08	0.13	0.12	0.12	0.12	0.10	0.10	0.10	0.12	0.13	0.13	0.11
U22	0.11	0.11	0.11	0.07	0.11	0.11	0.09	0.10	0.08	0.08	0.10	0.12	0.10	0.10
U23	0.08	0.07	0.06	0.09	0.06	0.11	0.11	0.10	0.09	0.09	0.09	0.08	0.11	0.09
U31	0.07	0.05	0.06	0.09	0.09	0.05	0.07	0.10	0.08	0.08	0.08	0.07	0.07	0.07
U32	0.03	0.03	0.04	0.03	0.03	0.05	0.02	0.02	0.02	0.02	0.02	0.02	0.02	0.03
U41	0.10	0.10	0.10	0.09	0.10	0.90	0.07	0.07	0.11	0.11	0.10	0.10	0.09	0.09
U42	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.05	0.02	0.06	0.01	0.01	0.01	0.02
U43	0.01	0.01	0.01	0.01	0.01	0.01	0.01	0.05	0.06	0.02	0.01	0.01	0.01	0.02
U51	0.11	0.10	0.12	0.09	0.08	0.08	0.10	0.08	0.08	0.08	0.06	0.10	0.10	0.09
U52	0.11	0.10	0.09	0.11	0.09	0.08	0.10	0.08	0.08	0.08	0.09	0.10	0.10	0.09
U53	0.09	0.10	0.09	0.09	0.10	0.10	0.09	0.08	0.09	0.09	0.11	0.11	0.07	0.09

). The nth power of the weighted hyper matrix was solved for the vectors of the columns of the matrix, resulting in the comprehensive weights of each indicator, from which the weights of the network layer indicators could be obtained, as shown in

Table 5. Weight of each indicator.

Control Layer	Weight	Network Layer	Weight
U1	0.2	U11	0.57
		U12	0.43
U2	0.3	U21	0.38
		U22	0.33
		U23	0.29
U3	0.1	U31	0.73
		U32	0.27
U4	0.12	U41	0.76
		U42	0.12
		U43	0.12
U5	0.28	U51	0.34
		U52	0.32
		U53	0.34

.

The normalization formula:

$$\frac{x}{{\sum }_{i=1}^n{x}_i}$$

(3)

3.2. Fuzzy Comprehensive Evaluation of Public Opinion Events

The core of the evaluation of public opinion is to guess the developing trend and the degree of urgency of the emergent events, which is in accordance with the principles of fuzzy mathematics. The fuzzy comprehensive evaluation is based on fuzzy mathematics and applies the principle of fuzzy relationship synthesis to quantify some factors with unclear boundaries, on which it then conducts a comprehensive evaluation. Based on the indicators of the network public opinion established in the previous section and the principle of fuzzy comprehensive evaluation, the specific steps for constructing the fuzzy comprehensive evaluation model of online public opinion are as follows.

Determine the set of indicators. All indicator factors together constitute the factor set written as $U=\left\{U_1,U_2\dots U_n\right\}$. In the ANP network structure analysis, the indicator factors of the control layer were determined as the primary evaluation indicator factors, given as $U_m=\left\{U_1,U_2,U_3,U_4,U_5\right\}$, and the indicator factors of the network layer were the secondary evaluation indicator factors, given as $U_n=\left\{U_{11},U_{12},U_{21},U_{22},U_{23},U_{31},U_{32},U_{41},U_{42},U_{43},U_{51},U_{52},U_{53}\right\}$.

Determine the weight sets. Since the indicators have different degrees of importance to the whole evaluation system, it is necessary to assign different weights to each indicator factor to distinguish its importance. According to the calculation results of the DE-ANP model, the indicator weight set was achieved, of which the weights of indicator factors at the control level were noted as m and the weights of indicator factors at the network level were noted as n, and the corresponding matrix is as follows.

$$w_n=\left[\begin{array}{ccc}{w}_{11}& \cdots & w_{1n}\\ ⋮& \cdots & ⋮\\ w_{m1}& \cdots & w_{mn}\end{array}\right]$$

Determine the evaluation set. There are numerous topics of online public opinion information, including people’s livelihood, accidents and disasters, public health, culture, education, etc. The semantic affective tendency of information has a fuzzy attribute with a clear connotation but an unclear boundary, which is in line with the basic principles of fuzzy mathematics. Regardless of the topics, the semantic affective tendencies carried by the information can be roughly summarized as positive, neutral, and negative. It is necessary to further refine the information semantic affective tendency rubric and determine the fine-grained rubric set V, V = {micro, small, medium, large, extreme}, which are shown in Figure 3.

Construct the fuzzy comprehensive evaluation sets. The advantage of fuzzy evaluation is that it can evaluate both a single factor and the total factors. If the affiliation of the i element in the factor set U corresponding to the m element in the evaluation set V was r_im, then the single-factor evaluation result for the i element was noted as a fuzzy set: $R_i=\left\{R_{i1},R_{i2},\dots R_{im}\right\}$, with m single-factor evaluation sets R₁, R₂,… R_m as rows to form the matrix $R_{m\times n}$. After determining the single-factor evaluation matrix R and the factor weight vector ${\omega }_{\mathrm{n}}$, the fuzzy weight set at ${\omega }_n$ in U became a fuzzy vector B on V through fuzzy changes, denoted as $B={\omega }_n\times R_{m\times n}=\left\{b_1,b_2,\dots b_n\right\}$, where the fuzzy algorithm used the weighted average method.

The second-level fuzzy evaluating model is:

$$B_i={\omega }_n\times R_i$$

(4)

The first-level fuzzy evaluating model is:

$$B=\omega \times R,\mathrm{where}R=B_i$$

(5)

${\omega }_n$ is the weight of network layer indicators, R is the second-level fuzzy judgment matrix B_i, and $\omega$ is the first-level fuzzy judgment matrix weight. Finally, based on the principle of maximum affiliation, the network public opinion events were evaluated.

4. Empirical Analysis

4.1. Case Data Collection and Communication Power Evaluation

This study selected the public opinion events that occurred in 2022, whose topics covered education, entertainment, public health, and business. With the help of Python crawler software, this paper collected the dissemination data of four major public opinion events, namely a math textbook with inappropriate illustrations, the CNKI monopoly, shifts in the COVID-19 policy, and the crash of flight MU5735, within ten days from the date of occurrence. In March, flight MU5735 of the CEA Holding company crashed causing the deaths of the crew and all the passengers. In May, the Chinese PEP version math textbook was exposed for its inappropriate character illustrations with protruding genitals, and subsequently, more picture books affecting children’s physical and mental health were exposed, which created an uproar on the Internet. In December, China’s largest academic website, CNKI, was investigated by the relevant government departments for a suspected monopoly, and China ended its dynamic zero-COVID-19 case policy.

The data were collected from four major communication platforms, namely Sina Weibo, Netease News, Tencent News, and Sohu News. Due to the space limitations, this paper only introduces the analysis process of the incident regarding the math textbook with inappropriate illustrations in detail, whose statistical processing of the data were captured using Python tracking, and the specific data statistical results are shown in

Table 6. Dissemination data of the math textbook with inappropriate illustrations.

Control Layer	Network Layer	Weibo	Netease	Tecent	Souhu
Degree of heat	Number of issuances (piece)	33,678	1456	2877	1782
	Duration of concern (day)	10	7	9	6
Degree of quantity	Number of clicks (million)	67.31	1.93	2.89	1.72
	Number of shares (thousand)	310.4	10.08	8.53	12.23
	Number of comments (thousand)	16.06	2.7	1.83	6.87
Degree of strength	Number of opinion leaders (person)	23	13	9	11
	Network area distribution (province)	24	26	23	27
Degree of focus	Negative opinion holding rate	0.91	0.82	0.93	0.85
	Neutral opinion holding rate	0.07	0.09	0.07	0.08
	Positive opinion holding rate	0.02	0.09	0	0.07
Degree of variation	Click growth rate	0.63	0.34	0.28	0.37
	Share growth rate	0.54	0.29	0.33	0.31
	Comment growth rate	0.59	0.41	0.29	0.34

.

Online questionnaires were distributed to the teachers and students of two universities via WeChat, in which they were asked to evaluate the communication power of the events based on their communication data. A total of 198 valid questionnaires were finally obtained, after which the statistical analysis of the questionnaire was conducted, and the value was taken as the weight of the Tth evaluation result under the Sth item, $P_{ST}=n_{ST}/N$ (Formula (6)). The specific data results are presented in

Table 7. Evaluation of the communication power of the math textbook illustrations.

Control Layer	Network Layer	Extreme	Large	Medium	Small	Light
Degree of heat	Number of issuances	0.93	0.06	0.03	0	0
	Duration of concern	0.85	0.1	0.03	0.01	0
Degree of quantity	Number of clicks	0.84	0.09	0.06	0.01	0
	Number of shares	0.86	0.06	0.04	0.03	0.01
	Number of comments	0.74	0.21	0.02	0.03	0
Degree of strength	Number of opinion leaders	0.72	0.18	0.09	0.01	0
	Network area distribution	0.68	0.3	0.02	0	0
Degree of focus	Negative opinion holding rate	0.82	0.11	0.07	0	0
	Neutral opinion holding rate	0.44	0.17	0.23	0.04	0.12
	Positive opinion holding rate	0.28	0.12	0.46	0.1	0.04
Degree of variation	Click growth rate	0.81	0.16	0	0.02	0
	Share growth rate	0.78	0.12	0.04	0.03	0.03
	Comment growth rate	0.72	0.16	0.1	0.02	0

. The affiliation degree of the quantitative ranking of the network public opinion indicators according to their probability distribution was determined, based on which a fuzzy subset of the affiliation degree was constructed.

4.2. Fuzzy Comprehensive Evaluation

The set of fuzzy comprehensive evaluation weights was constructed in accordance with the calculation results of the DEMATEL−ANP model, where the control layer weights were $\omega =\left\{0.2,0.3,0.1,0.12,0.28\right\}$, and the network layer weights were ${\omega }_1=\left\{0.57,0.43\right\}$, ${\omega }_2=\left\{0.38,0.33,0.29\right\}$, ${\omega }_3=\left\{0.73,0.27\right\}$, ${\omega }_4=\left\{0.76,0.12,0.12\right\}$, and ${\omega }_5=\left\{0.34,0.32,0.34\right\}$.

Following the affiliation principle and according to the determined rubric set V = {micro, small, medium, large, extreme}, the affiliation of the rubric set took the value V = (0.2, 0.4, 0.6, 0.8, 1.0). Based on the determined set of factors, a single-factor evaluation matrix R_i was constructed.

$$\begin{gathered} R_1=\left[\begin{array}{ccccc}0.93& 0.06& 0.03& 0& 0\\ 0.85& 0.10& 0.03& 0.01& 0\end{array}\right] \\ {R}_2=\left[\begin{array}{ccccc}0.84& 0.09& 0.06& 0.01& 0\\ 0.86& 0.06& 0.04& 0.03& 0.01\\ 0.74& 0.21& 0.02& 0.03& 0\end{array}\right] \\ {R}_3=\left[\begin{array}{ccccc}0.72& 0.18& 0.09& 0.01& 0\\ 0.68& 0.30& 0.02& 0& 0\end{array}\right] \\ {R}_4=\left[\begin{array}{ccccc}0.82& 0.11& 0.07& 0& 0\\ 0.44& 0.17& 0.23& 0.04& 0.12\\ 0.28& 0.12& 0.46& 0.10& 0.04\end{array}\right] \\ {R}_5=\left[\begin{array}{ccccc}0.81& 0.16& 0.01& 0.02& 0\\ 0.78& 0.12& 0.04& 0.03& 0.03\\ 0.72& 0.16& 0.10& 0.02& 0\end{array}\right] \end{gathered}$$

The new B_i fuzzy evaluation matrix was obtained using the fuzzy synthesis of each single-factor evaluation matrix in Formula (6), where $B_1=\left\{0.90,0.08,0.03,0,0\right\}$, $B_2=\left\{0.82,0.12,0.04,0.02,0\right\}$, $B_3=\left\{0.71,0.21,0.07,0,0\right\}$, $B_4=\left\{0.71,0.12,0.14,0.02,0.02\right\}$, and $B_5=\left\{0.77,0.15,0.05,0.02,0.01\right\}$. According to the second-level fuzzy calculation method, the single-factor evaluation results of the next level were used to form the judgment matrix R of the previous level, where R is the second-level fuzzy judgment matrix B_i.

$$R=\left[\begin{array}{ccccc}0.90& 0.08& 0.03& 0& 0\\ 0.82& 0.12& 0.04& 0.02& 0\\ 0.71& 0.21& 0.07& 0& 0\\ 0.71& 0.12& 0.14& 0.02& 0.02\\ 0.77& 0.15& 0.15& 0.02& 0.01\end{array}\right]$$

(6)

The fuzzy comprehensive evaluation of the math textbook with inappropriate illustrations was $B=\omega \times R=\left\{0.80,0.13,0.06,0.01,0\right\}$. Using Equation (5), and following the principle of maximum subordination in fuzzy evaluation, it can be found that the math textbook with inappropriate illustrations was at the level of “extreme”. The same method was applied to quantitatively evaluate the CNKI monopoly, shifts in the COVID-19 policy, and the crash of flight MU5735, achieving evaluation values of 0.52, 0.87, and 0.81, respectively, the specific results of which are shown in

Table 8. Evaluation results of four public opinion events.

Events	Social Field	Evaluation Value	Level
Math textbook with inappropriate illustrations	Education	0.80	Extreme
CNKI monopoly	Business	0.52	Medium
Shifts in the COVID-19 policy	Public health	0.87	Extreme
Flight MU5735 crash	Security	0.81	Extreme

.

The math textbook with inappropriate illustrations reached an extreme level because it is closely related to every family and every child’s physical and mental health, which required an immediate response and crisis management by government education departments to change the status quo and conform to public aesthetics. COVID-19 cast a shadow on people’s health, and the strict lockdown policy in China brought inconvenience to people’s lives, any shift in which would undoubtedly draw public attention. The crash of flight MU5735 resulted in people’s sympathy and condolences for the victims and their families and raised concerns about the safety of public transportation. CNKI is the biggest academic search platform, and it was suspected of monopoly, triggering discussion by netizens; however, the discussion on this issue was limited to specific groups such as college students, university teachers, and scholars; thus, the network public opinion evaluation of it was regarded as medium.

5. Conclusions

Public opinion is the most direct way to understand the influence of events, as people can gather online to share information, knowledge, and opinions. Therefore, it is necessary to predict and evaluate the urgency of public opinion events.

In this study, the index system indicator weights of the evaluation of public opinion for emergent events were determined using the DEMATEL−ANP method, and the fuzzy comprehensive evaluation model was constructed to evaluate the degree of emergent events’ urgency; in addition, empirical analysis was conducted on four public events. The following conclusions can be drawn.

First, an index system consisting of 5 first-level indicators and 13 second-level indicators was established for emergent events after repeated rectification and selection under the principles of profession, practicality, quantification, correlation, and importance via an extensive literature review to improve the existing public opinion index systems, which have the problem of being cumbersome, redundant, and difficult to obtain or to quantify.

Second, the DEMTEL−ANP model used in some research papers often obtains the individual weights and then derives their final mixed weights, which requires experts to repeat the evaluation of the same system twice that does not take advantage of it fully. This study improved it by taking the results of the DEMATEL as the input of the ANP to attain the weights of the indicators scientifically.

Third, the core of the evaluation of public opinion is to guess the developing trend and the degree of urgency of the emergent events, which is in accordance with the principles of fuzzy mathematics. Therefore, this study applies the principle of fuzzy relationship synthesis to quantify some factors with unclear boundaries, on which it then conducted a comprehensive evaluation.

In summary, the DEMATEL−ANP fuzzy comprehensive model has the following merits: it derives the weights of indicators scientifically by taking their interdependent relationship fully into account; fuzzy evaluation deals with fuzzy objects using more accurate numerical means, and it can conduct a more scientific, reasonable quantitative evaluation of the information presenting fuzziness; finally, the evaluation result is a vector containing richer information, not a point value, which not only portrays the evaluated events more accurately, but also can be further processed to obtain more reference information.

In this study, from a mathematical statistical perspective, the DEMATEL−ANP fuzzy comprehensive evaluation model was developed to assess the urgency of public opinion events, though it still has some limitations. In future research, machine learning models should be incorporated to improve its accuracy and dynamics.