*2.3. Assessment Model Construction*

The goal of theMEEM is to use the degree of association of the extension set to determine the assessment level of the matter element feature. The MEEM is a method for comprehensive multi-index assessments, and it is mainly based on the extrinsic matter-element model, extension set and correlation function theory [22]. This method can judge the subordinate level of items according to their different characteristics with low data requirements and can avoid the randomness and subjectivity of the evaluation process to a certain extent. The MEEM combines qualitative and quantitative analyses and has achieved good results in risk assessment in many fields, such as oil exploitation [23], tailings pond [24], and building fire [25]. Based on the above advantages, this method is applied to research CBM development risk assessments.

The MEEM includes the following steps [44,45]: (1) According to the development of things and relevant reference materials, the characteristics of things are analyzed, the things are divided into several grades according to certain rules, the numerical range of each level is clarified, and a multi-index MEEM is established; (2) Using the correlation function to calculate the degree of association between the things to be evaluated and each assessment level; (3) Things have the highest degree of relevance to one of the levels, indicating that they are most consistent with that level. Next, the calculation process of the CBM development risk assessment model will be described in detail.

## 2.3.1. Determination of the Classical Domain, Joint Domain and Matter-Element Evaluation

The matter element uses the ordered triplet *M* = {*C, R, V*} as the basic element to describe things, where *C* is the name of the thing, *R* is the name of the feature, and *V* is the value taken by *C* for *R* [25].

(1) Determining the classical domain *Mj*

Let *Mj* be the classic domain of matter-element *M*:

$$\mathcal{M}\_{\dot{j}} = \begin{pmatrix} \mathcal{U}\_{\dot{j}}, \mathcal{R}, \mathcal{V}\_{\dot{j}} \end{pmatrix} = \begin{pmatrix} \mathcal{U}\_{\dot{j}} & r\_1 & \begin{pmatrix} a\_{j1}, b\_{j1} \\ a\_{j2}, b\_{j2} \end{pmatrix} \\ & \vdots & \vdots \\ & r\_i & \begin{pmatrix} a\_{j i}, b\_{j i} \end{pmatrix} \end{pmatrix} \tag{9}$$

where *Uj* is the *j* risk level in the risk level domain *U*, *Vj* is the range of assessment index set *R* about the risk level *Uj*, and *aji* and *bji* are the lower and upper limits of the index *ri* at the *j*th risk level, respectively.

(2) Determining the joint domain *Mc*

Let *Mc* be the joint domain of matter-element *M*:

$$M\_{\mathcal{E}} = (\mathcal{U}, \mathcal{R}, V\_{\mathcal{E}}) = \begin{pmatrix} \mathcal{U} & r\_1 & (a\_{c1}, b\_{c1}) \\ & r\_2 & (a\_{c2}, b\_{c2}) \\ & \vdots & \vdots \\ & & r\_i & (a\_{ci}, b\_{ci}) \end{pmatrix} \tag{10}$$

where *Vc* is the range of evaluation index set *R* about the risk level domain *U*, and *aci* and *bci* are the lower and upper limits of the index *ri* at all risk levels, respectively.

(3) Determining the matter-element evaluation *Mi*

Let *Mi* be the matter-element evaluation of matter-element *M*:

$$M\_i = \left( S\_{i\nu} R\_{i\nu} V\_i \right) = \begin{pmatrix} S\_j & r\_{i1} & v\_{i1} \\ & r\_{i2} & v\_{i2} \\ & & \vdots & \vdots \\ & & r\_{ip} & v\_{ip} \end{pmatrix} \tag{11}$$

where *Si* is the *i*th first grade index to be evaluated, *Ri*={ *ri*1, *ri*2, ... , *rip*} is the second grade assessment index set for *Si*, *rip* is the *p*th second grade assessment index of the *i*th first grade index, and *Vi* is the value of the second grade index *Ri* for *Si*.

#### 2.3.2. Calculating the Correlation Degree

By introducing the concept of distance in classical mathematics, the correlation function of the second grade index *rik* of the CBM development risk assessment on the risk level *Uj* is established. Therefore, the correlation degree *Kj*(*rik*) of the *k*th second grade index in the *i*th first grade index with respect to the risk level *Uj* is determined.

$$K\_{\dot{j}}(r\_{\text{ik}}) = \begin{cases} \frac{\rho(\upsilon\_{\text{ik}}, V\_{\dot{j}})}{\rho(\upsilon\_{\text{ik}}, V\_{\cir}) - \rho(\upsilon\_{\text{ik}}, V\_{\dot{j}})} & \rho(\upsilon\_{\text{ik}}, V\_{\cir}) - \rho(\upsilon\_{\text{ik}}, V\_{\dot{j}}) \neq 0\\ -\rho(\upsilon\_{\text{ik}}, V\_{\dot{j}}) - 1 & \rho(\upsilon\_{\text{ik}}, V\_{\cir}) - \rho(\upsilon\_{\text{ik}}, V\_{\dot{j}}) = 0 \end{cases} \tag{12}$$

where ρ *vik*, *Vj* = *vik* <sup>−</sup> *aji*+*bji* 2 − 1 2 *bij* <sup>−</sup> *aji* , ρ(*vik*, *Vc*) = *vik* <sup>−</sup> *aci*+*bci* 2 − 1 2 *bcj* <sup>−</sup> *aci* , *aci* and *bci* are the lower and upper limits of the index *ri* at all risk levels, respectively, and *vik* is the expert score for the second grade index *rik*.

#### 2.3.3. Multi-Level Extension Assessment

#### (1) Primary assessment

Calculate the correlation matrix *K*(*ri*) of the first indexes for each risk level:

$$K(r\_i) = \begin{pmatrix} \omega\_{i1}, \omega\_{i2}, \cdots, \omega\_{ip} \end{pmatrix} \begin{bmatrix} k\_1(r\_{i1}) & k\_2(r\_{i1}) & \cdots & k\_m(r\_{i1}) \\ k\_1(r\_{i2}) & k\_2(r\_{i2}) & \cdots & k\_m(r\_{i2}) \\ \vdots & \vdots & & \vdots \\ k\_1(r\_{ip}) & k\_2(r\_{ip}) & \cdots & k\_m(r\_{ip}) \end{bmatrix} = (k\_j(r\_i)),\tag{13}$$

where ω*<sup>i</sup>* = (ω*ik*) is the weight vector of the second grade indexes, and the calculation method is shown in Formulas (1)~(8); and *K*(*rik*) = (*kj*(*rik*)) is the correlation degree matrix of the second grade indexes for each risk level.

(2) Secondary assessment

Determine the correlation degree matrix *K*(*S*) of the CBM development safety for each risk level.

$$K(S) = \begin{pmatrix} \omega\_1, \omega\_2, \dots, \omega\_j \end{pmatrix} \begin{bmatrix} k\_1(r\_1) & k\_2(r\_1) & \cdots & k\_m(r\_1) \\ k\_1(r\_2) & k\_2(r\_2) & \cdots & k\_m(r\_2) \\ \vdots & \vdots & & \vdots \\ k\_1(r\_p) & k\_2(r\_p) & \cdots & k\_m(r\_p) \end{bmatrix} = (k\_j(S)),\tag{14}$$

where ω = (ω*j*) is the weight vector of the first grade indexes, and the calculation method is shown in formulas (1)~(8); a®*K*(*r*) = (*k*(*ri*)) is the correlation degree matrix of the first grade indexes for each risk level.

(3) Determining the risk level

According to the principle of maximum membership degree, the risk level corresponding to the maximum correlation degree in the correlation degree matrix *K*(*S*) of the CBM development for each risk level is the risk level of the assessment object. That is, when *maxkj*(*S*) = *kj* (*S*), with *j* = (1,2,3 ... *n*), the risk level of the assessment object is at level *j*.

#### 2.3.4. Risk Classification

To scientifically measure the risk of CBM development and ensure the systematic nature and accuracy of the assessment results, this paper divides the risk level of CBM development into five grades according to the actual CBM development situation and the risk classification rules in the literature [25,46], as shown in Table 5. Notably, when the established risk assessment model has been tested by a large number of empirical tests, the risk classification criteria can be corrected by data feedback to make the criteria more sensitive.


#### **Table 5.** Risk levels.

## **3. Case Study**

The southern part of the Qinshui Basin is one of the earliest areas for CBM exploration and development in China, and it has also attracted the most investment and research in CBM exploration and development in China [47]. The CBM storage conditions in this area are stable and have good development potential [48,49]. As a key breakthrough area for CBM exploration and development, many experts have carried out exploration and research work here, leading to the accumulation of a wealth of test and production data [50]. The experts have a deeper understanding of the characteristics of reservoir CBM accumulation, reservoir geological conditions and gas layer distribution. Therefore, this paper takes a CBM development project in this area as the research object and invited 5 experts, including 1 CBM exploration expert, 2 CBM mining technical experts, 1 energy policy expert and 1 project manager, to participate. Based on the engineering practice data of the project, the experts anonymously scored the actual operation status of each second grade index. The total score for each index is 100. In this paper, the actual scores of the second grade indexes of the CBM development project are obtained by calculating the average value, as shown in Table 4.

#### *3.1. Determination of the Classical Domain, Joint Domain and Matter-Element Evaluation*

Take the first grade index laws, regulations and policies (*r*1) as an example to establish the matter-element *M* and determine its classical domain *Mj*, joint domain *Mc* and matter-element evaluation *Mi*. Similarly, classical domains, joint domains and matter-element evaluations of other first grade indexes can be obtained.

(1) Determining the classical domain *Mj*

The classical domains for each risk level are determined by Formula (9):

*M*<sup>I</sup> = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ *U*<sup>I</sup> *r*<sup>11</sup> (85, 100) *r*<sup>12</sup> (85, 100) *r*<sup>13</sup> (85, 100) *r*<sup>14</sup> (85, 100) *r*<sup>15</sup> (85, 100) *r*<sup>16</sup> (85, 100) ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠ *M*II = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ *U*II *r*<sup>11</sup> (70, 85) *r*<sup>12</sup> (70, 85) *r*<sup>13</sup> (70, 85) *r*<sup>14</sup> (70, 85) *r*<sup>15</sup> (70, 85) *r*<sup>16</sup> (70, 85) ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠ , *M*III = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ *U*III *r*<sup>11</sup> (50, 70) *r*<sup>12</sup> (50, 70) *r*<sup>13</sup> (50, 70) *r*<sup>14</sup> (50, 70) *r*<sup>15</sup> (50, 70) *r*<sup>16</sup> (50, 70) ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠ *M*IV = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ *U*IV *r*<sup>11</sup> (25, 50) *r*<sup>12</sup> (25, 50) *r*<sup>13</sup> (25, 50) *r*<sup>14</sup> (25, 50) *r*<sup>15</sup> (25, 50) *r*<sup>16</sup> (25, 50) ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠ , *M*<sup>V</sup> = ⎛ ⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝ *U*<sup>V</sup> *r*<sup>11</sup> (0, 25) *r*<sup>12</sup> (0, 25) *r*<sup>13</sup> (0, 25) *r*<sup>14</sup> (0, 25) *r*<sup>15</sup> (0, 25) *r*<sup>16</sup> (0, 25) ⎞ ⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠ ,

(2) Determining the joint domain *Mc*

The joint domain *Mc* is determined by Formula (10):

$$M\_{\mathbb{C}} = \begin{pmatrix} \mathcal{U} & r\_{11} & (0,100) \\ & r\_{12} & (0,100) \\ & r\_{13} & (0,100) \\ & r\_{14} & (0,100) \\ & r\_{15} & (0,100) \\ & r\_{16} & (0,100) \end{pmatrix}$$

.

.

(3) Determining the matter-element evaluation *Mi*

The matter-element evaluation *M*<sup>1</sup> of indexes *r*1*<sup>i</sup>* is determined by Formula (11):

$$M\_1 = \begin{pmatrix} S\_1 & r\_{11} & v\_{11} \\ & r\_{12} & v\_{12} \\ & \vdots & \vdots \\ & & r\_{1p} & v\_{1p} \end{pmatrix} = \begin{pmatrix} R\_1 & r\_{11} & 91 \\ & r\_{12} & 88 \\ & & r\_{13} & 48 \\ & & r\_{14} & 83 \\ & & r\_{15} & 86 \\ & & r\_{16} & 84 \end{pmatrix}$$

#### *3.2. Calculating the Correlation Degree*

Taking the correlation degree of the second grade index *r*<sup>11</sup> of the first grade index *r*<sup>1</sup> as an example, the calculation steps of the correlation degrees of the second grade indexes are obtained by Formula (12).

ρ(*v*11,*V*1)=|91 − (85 + 100)/2| − (100 − 85)/2 = − 6; ρ(*v*11,*V*2) = |91 − (70 + 85)/2| − (85 − 70)/2 = 6; ρ(*v*11,*V*3) = |91 − (50 + 70)/2| − (70 − 50)/2 = 21; ρ(*v*11,*V*4)=|91 − (25 + 50)/2| − (50 − 25)/2 = 41; ρ(*v*11,*V*5) = |91 − (25 + 0)/2| − (25 − 0)/2 = 66; ρ(*v*11,*Vc*) = |91 − (0 + 100)/2| − (100 − 0)/2 = − 9; *k*1(*r*11) = ρ(*v*11,*V*1)/[ ρ(*v*11,*Vc*)− ρ(*v*11,*V*1)] = −6/(−9 + 6) = 2; *k*2(*r*11) = ρ(*v*11,*V*2)/[ ρ(*v*11,*Vc*) − ρ(*v*11,*V*2)] = 6/(−9−6) = −0.4; *k*3(*r*11) = ρ(*v*11,*V*3)/[ ρ(*v*11,*Vc*) − ρ(*v*11,*V*3)] = 21/(−9−21) = −0.7; *k*4(*r*11) = ρ(*v*11,*V*4)/[ ρ(*v*11,*Vc*) − ρ(*v*11,*V*4)] = 41/(−9−41) = −0.82; *k*4(*r*11) = ρ(*v*11,*V*5)/[ ρ(*v*11,*Vc*) − ρ(*v*11,*V*5)] = 66/(−9−66) = −0.88;

Similarly, the correlation degrees of the second grade indexes under other first grade indexes can be calculated. The calculation results are shown in Table 6. The calculation process is shown in Appendix C.


**Table 6.** The relevance of second grade indexes.

**First Grade Indexes Second Grade Indexes Relevance of Second Grade Indexes Risk Level I II III IV V** Organizational management (*r*5) Organizational structure adaptability (*r*51) 0.000 0.000 −0.500 −0.700 −0.800 I, II Management coordination and communication (*r*52) −0.222 0.400 −0.300 −0.580 −0.720 II Process management (*r*53) −0.308 0.125 −0.100 −0.460 −0.640 II Organizational management quality (*r*54) −0.142 0.200 −0.400 −0.640 −0.760 II Resource allocation capability (*r*55) −0.297 0.182 −0.133 −0.480 −0.653 II Safety and emergency protection (*r*6) Safety technology and equipment (*r*61) −0.286 0.250 −0.167 −0.500 −0.667 II Hidden danger investigation and treatment (*r*62) −0.241 0.467 −0.267 −0.560 −0.707 II Safety training and education (*r*63) −0.397 −0.241 0.158 −0.120 −0.413 III Safety culture (*r*64) −0.317 0.077 −0.067 −0.440 −0.627 II Safety investment (*r*65) −0.143 0.200 −0.400 −0.640 −0.760 II Emergency plan (*r*66) −0.297 0.182 −0.133 −0.480 −0.653 II Emergency drill frequency (*r*67) −0.403 −0.258 0.095 −0.080 −0.387 III Emergency supplies reserve (*r*68) −0.373 −0.159 0.233 −0.260 −0.507 III Emergency rescue team (*r*69) −0.273 0.333 −0.200 −0.520 −0.680 II

**Table 6.** *Cont.*

*3.3. Multi-Level Extension Assessment*

(1) Primary assessment

Taking the first grade index *r*<sup>1</sup> as an example, the correlation degree matrix *K*(*r*1) is determined according to the weight calculation result in Table 4 and Formula (13).

$$\begin{aligned} K(r\_1) &= (\omega\_{11}, \omega\_{12}, \omega\_{13}, \omega\_{14}, \omega\_{15}, \omega\_{16}) (k\_j(r\_1)) = \\ &(0.226, 0.235, 0.113, 0.188, 0.118, 0.120) \begin{bmatrix} 0.200 & -0.400 & -0.700 & -0.820 & -0.880 \\ 0.333 & -0.200 & 0.600 & -0.760 & -0.840 \\ -0.435 & -0.314 & -0.040 & -0.043 & -0.324 \\ -0.105 & 0.133 & -0.433 & -0.660 & -0.773 \\ 0.077 & -0.067 & -0.533 & -0.720 & -0.813 \\ -0.059 & 0.067 & -0.467 & -0.680 & -0.787 \end{bmatrix} \\ &= (0.463, -0.148, -0.504, -0.650, -0.769) \end{aligned}$$

In the same way, the correlation degree matrixes of other first grade indexes are calculated. The detailed calculation processes are shown in Appendix B.

*K*(*r*2)=(−0.006, 0.105, −0.411, −0.647, −0.765); *K*(*r*3)=(−0.075, 0.205, −0.276, −0.566, −0.711); *K*(*r*4)=(−0.204, 0.200, −0.260, −0.540, −0.694); *K*(*r*5)=(−0.188, 0.163, −0.286, −0.572, −0.714); *K*(*r*6)=(−0.295, 0.114, −0.103, −0.414, −0.610).

(2) Secondary assessment

According to formula (14), the comprehensive correlation degree matrix of the research object for each risk level is determined.

$$\begin{aligned} K(S) &= (\omega\_1, \omega\_2, \omega\_3, \omega\_4, \omega\_5, \omega\_6) K\_i(r\_i) = \\ K\_1(0.165, 0.214, 0.209, 0.184, 0.094, 0.134) \begin{bmatrix} 0.463 & -0.148 & -0.504 & -0.650 & -0.769 \\ -0.066 & 0.105 & -0.411 & -0.647 & -0.765 \\ -0.075 & 0.205 & -0.276 & -0.566 & -0.711 \\ -0.204 & 0.200 & -0.260 & -0.540 & -0.694 \\ -0.188 & 0.163 & -0.286 & -0.572 & -0.714 \\ -0.295 & 0.114 & -0.103 & -0.414 & -0.610 \end{bmatrix} \\ &= (-0.035, 0.108, -0.317, -0.573, -0.716) \end{aligned}$$

(3) Determining the risk level

According to the principle of maximum membership degree, the risk level corresponding to the maximum correlation degree in correlation degree matrix *K*(*S*) of CBM development for each risk level is the risk level of the assessment object. Because *maxkj*(*S*)=0.108=*K*2(*S*), the risk level of the CBM development project is Grade II.
