Knowledge Granularity Attribute Reduction Algorithm for Incomplete Systems in a Clustering Context

Abstract

The phenomenon of missing data can be seen everywhere in reality. Most typical attribute reduction models are only suitable for complete systems. But for incomplete systems, we cannot obtain the effective reduction rules. Even if there are a few reduction approaches, the classification accuracy of their reduction sets still needs to be improved. In order to overcome these shortcomings, this paper first defines the similarities of intra-cluster objects and inter-cluster objects based on the tolerance principle and the mechanism of knowledge granularity. Secondly, attributes are selected on the principle that the similarity of inter-cluster objects is small and the similarity of intra-cluster objects is large, and then the knowledge granularity attribute model is proposed under the background of clustering; then, the IKAR algorithm program is designed. Finally, a series of comparative experiments about reduction size, running time, and classification accuracy are conducted with twelve UCI datasets to evaluate the performance of IKAR algorithms; then, the stability of the Friedman test and Bonferroni–Dunn tests are conducted. The experimental results indicate that the proposed algorithms are efficient and feasible.

1. Introduction

Rough set theory (RST) [1], initiated by Pawlak, is an effectively mathematical tool to deal with imprecise, fuzzy, and incomplete data. RST has been successfully applied in machine learning [2,3,4], knowledge discovery [5,6], expert system [7], disease diagnostics [8,9,10], decision support [11,12,13], and other areas [14,15]. Attribute reduction is one of the research hotspots in RST. As an important technology in the process of data preprocessing, attribute reduction has captured researchers’ attention in big data and knowledge discovery [16,17,18]. The main objective of attribute reduction is to remove some irrelevant or non-important attributes while keeping the original distinguishing ability unchanged. In this way, the effect of data dimension reduction can be achieved, and a lot of time and space resources can be saved for the process of knowledge discovery and rule extraction.

With the rapid development of network and information technology, we have gradually entered the era of big data. The datasets have the characteristics of large volume, rapid change, and diverse data forms [19]. At the same time, due to the influence of the collection method and environment during the data collection process, there are a large number of missing data or wrong data in the datasets. The existence of these disturbed data will seriously affect the decision making and judgement of big data, and even mislead decision makers. After a long period of unremitting endeavor, scholars have achieved outstanding results in attribute reduction [20,21,22,23,24,25,26]. For example, Dai [20] proposed a semi-supervised attribute reduction based on attribute indiscernibility. Cao [21] put forward a three-way approximate reduction approach by using information-theoretic measure. Yang [22] presented a novel incremental attribute reduction method via quantitative dominance-based neighborhood self-information. Lin [16] et al. developed a feature selection way by using neighborhood multi-granulation fusion. In a variable precision rough set model, Yu [24] et al. raised a novel attribute reduction based on local attribute significance. In the literature [27], Devi proposed a new dimension reduce technology by considering the picture of fuzzy soft matrices in the decision-making process. Wen [28] et al. raised an unsupervised attribute reduction algorithm for mixed data based on fuzzy optimal approximation set. These above-mentioned classic reduction models are suitable only for complete systems.

As with most of the datasets in various real-world applications, the classical rough set model defined with equivalence relation leads to the limitation in handling data in incomplete systems. The important attributes cannot be selected correctly, which leads to a decrease in the classification accuracy of the reduction set. In order to reduce the incomplete system, Zhang and Chen proposed a lambda-reduction approach based on the similarity degree respect to a conditional attribute subset for incomplete set-valued information systems [25]. For incomplete interval-valued information systems, Li [29] proposed the concept of similarity degree and tolerance relation between two information values of a given attribute. Then, three reduction algorithms based on theta-discernibility matrix, theta-information entropy, and theta-significance were designed. Liu introduced a new attribute reduction approach by using conditional entropy based on the fuzzy alpha-similarity relation [30]. Subsequently, Dai [31] proposed interval-valued fuzzy min–max similarity relations and designed two attribute reduction algorithms based on interval-valued fuzzy discernibility pairs model. Song [32] put forward the similarity degree between information values on each attribute and an attribute reduction method was designed by using information granulation and information entropy. Zhou presented a heuristic attribute reduction algorithm with a binary similarity matrix and attribute significance as heuristic knowledge under incomplete information systems [33]. Zhang [34] presented a novel approach for knowledge reduction by using the discernibility techniques in multi-granulation rough set model. He and Qu [35] put forward the fuzzy-rough iterative computation model based on symmetry relations for an incomplete categorical decision information system. Srirekha et al. proposed an attribute reduction in SE-ISI concept lattice based on the concept of object ranking for incomplete information systems [36]. Cornelis et al. put forward a generalized model of attribute reduction using fuzzy tolerance relation within the context of fuzzy rough set theory [37]. Liu applied the concept of accurate reduction and reduced invariant matrix for reducing attribute under information systems [38]. To reduce unnecessary tolerance classes for the original cover, Nguyen [39] introduced a new concept of stripped neighborhood covers and proposed an efficient heuristic algorithm in mixed and incomplete decision tables.

Although the above reduction algorithms can effectively reduce incomplete information systems, the classification accuracy of the reduction set is not ideal. The main reason is that only the importance of attributes is considered when selecting attributes, and the impact of attributes on classification is not considered. Usually, people take the best result of clustering as a reference standard for classification work and classify similar samples into the same cluster. In order to solve the problems mentioned above, this paper proposes an attribute reduction method from the perspective of clustering.

At present, the studies of attribute reduction on using the clustering idea to construct a feature selection model are relatively infrequent. In order to avoid or reduce the loss of some original information after discretizing continuous values, Zhang [40] proposed a feature selection method based on fuzzy clustering, but this method has no reliable theoretical support. Jia [41] proposed a spectral clustering method based on neighborhood information entropy feature selection, which uses a feature selection method to remove redundant features before clustering. In order to take the classification effect of the dataset into consideration when selecting features, Zhao proposed a fuzzy C-Means clustering fuzzy rough feature selection method [42], which can improve the classification accuracy of the reduction set to a certain degree, but the effect is not obvious. Jia proposed a similarity attribute reduction in the context of clustering in the literature [43], which can greatly improve the classification accuracy of the reduction set but needs to continuously adjust the parameters to achieve the best classification effect. Such a feature set has certain random limitations and increases the time consumption of the system. Therefore, it is necessary to design a stable model with high classification accuracy for data preprocessing.

Although these existing approaches can effectively reduce incomplete systems, they only consider the importance of the attributes themselves, and do not consider the correlation between attributes. The influence of conditional attributes on decision classification is not considered. In order to improve the classification accuracy of a reduction set, we apply the idea of clustering.

Based on the principle that the similarity of samples within a cluster is as large as possible and the similarity of samples between clusters is as small as possible, an attribute reduction algorithm for an incomplete system is designed under the background of clustering. First, according to the principle of tolerance and the theory of knowledge granularity, we define the similarity of intra-cluster and inter-cluster for an incomplete system. Secondly, a formula for calculating the similarity of intra-cluster and inter-cluster objects is designed. After normalizing the two similarities, we define the similarity of objects. Then, according to the corresponding similarity mechanism, a new attribute reduction algorithm for an incomplete system is proposed. Finally, a series of experiments have verified that the proposed algorithm in this paper is significantly better than other similar algorithms in terms of running time, accuracy, and the stability of algorithm was analyzed by using Friedman test and Bonferroni–Dunn test in statistics.

The contribution of this paper is embodied in the following four aspects:

(1)

A tolerance class calculation in incomplete information systems is proposed and applied to knowledge granularity calculation.

(2)

Knowledge granules are used as a measure of sample similarity to measure the similarity of inter-cluster samples and intra-cluster samples.

(3)

A knowledge granularity reduction algorithm based on clustering context is designed in incomplete information systems.

(4)

Lots of experiments are conducted to verify the validity of the algorithm proposed in this paper, and the stability of the algorithm is verified by mathematical statistics.

The other parts of this paper are constructed as follows. The principle of tolerance and related concepts of knowledge granularity are recalled in Section 2. In Section 3, we propose a similarity measure of intra-cluster and inter-cluster objects and discuss the reduction mechanism according to the clustering background for missing dataset. We normalize the similarity of inter-cluster and intra-cluster, and then design the corresponding reduction model in Section 4. In Section 5, a series of experiments are conducted and the performance of the algorithm is evaluated from the reduction size, running time, classification accuracy, and stability. Then, the feasibility and effectiveness of the algorithm are verified. Finally, the advantages and disadvantages of the algorithm proposed in this paper are concluded and unfolded in the future work.

2. Preliminaries

In this section, we review some basic concepts in rough set theory, the definitions of tolerance class, knowledge granularity, clustering metrics, and the significance of attribute for incomplete decision systems.

2.1. Basic Concept of RST

A decision information system is a quadruple $DS=(U,A,$ $V,f)$, where $U$ is a non-empty finite set of objects and $A$ is a finite nonempty attribute sets; if $A=C\cup D$, where $C$ is the conditional attribute sets and $D$ is the decision attribute set; then, $V$ is the union of attribute domains, $V={\cup }_{a\in A}{V}_a,V_a$ is the value set of attribute $a$, called the domain of $a$; $f:U\times$ $A\to V$ is an information function with $f(x,a)=V_a$ for each $a\in A$ and $x\in U$. For every attribute subset $B\subseteq C$, a indiscernibility relation is defined as follows:

$$IND(B)=\{(x,y)\in U\times U|\forall a\in B,f(x,a)=f(y,a)\}.$$

(1)

By the relation $IND(B)$, we can obtain the partition of $U$ denoted by $U/IND(B)$ or $U/B$. If $B\subseteq A\wedge X\subseteq U$, the upper approximation is denoted as

$$\overline{B}(X)=\{x\in U|{[x]}_P\cap X\ne \varnothing \}.$$

(2)

The lower approximation of X can be denoted as

$$\underset{¯}{B}(X)=\{x\in U|{[x]}_P\subset X\}$$

(3)

where the objects in $\overline{B}(X)$ may belong to X, while the objects in $\underset{¯}{B}(X)$ must belong to X.

Definition 1. 

In a decision system $DS=\left(U,C,D,V,f\right)$, if $\exists a\in C and \exists x\in U$ that $f\left(x,a\right)=\ast$, then we call the decision system an incomplete system (IDS). In an incomplete decision system, if $P\subseteq C\cup D$ the tolerance relation is as follows:

$$\mathrm{T}\left(\mathrm{P}\right)=\{\left(\mathrm{x},\mathrm{y}\right)\in \mathrm{U}\times \mathrm{U}|\forall \mathrm{a}\in \mathrm{P},\mathrm{f}\left(\mathrm{x},\mathrm{a}\right)=\mathrm{f}\left(\mathrm{y},\mathrm{a}\right)\vee \mathrm{f}\left(\mathrm{x},\mathrm{a}\right)=\ast \vee \mathrm{f}\left(\mathrm{y},\mathrm{a}\right)=\ast \}$$

(4)

where * represents missing value. $T\left(P\right)$ is symmetric and reflexive, but not transitive.

Definition 2. 

Given an incomplete decision system $IDS=(U,C\cup D,V,f),\forall P\subseteq C\cup D$, $T(P)=\underset{a\in B}{\cap }T(a)$, and $T_P(o)$ is the tolerance class determined by o with respect to P, which is defined as follows:

$$T_P(o)=\{y\in U|(o,y)\in T(P)\}.$$

(5)

Let $U/T(P)$ denote the family set of $T_P(o)$, which is the classification included by P. If $P\subseteq B,$ then $T_B(o)\subseteq T_P(o)$, which is the monotonicity of the tolerance class.

2.2. Basic Concept of Knowledge Granularity

Definition 3. 

Suppose $IDS=(U,C\cup D,V,f)$ is an incomplete decision system, $\forall P\subseteq C$. $T_P(o_i)$ is the tolerance class of object $o_i$ with respect to P. The knowledge granularity of P on U is defined as follows:

$$G{K}_U(P)=\frac{1}{|U{|}^2}{\displaystyle \sum _{i=1}^{\left|U\right|}|T_P(o_i)}|$$

(6)

where $\left|U\right|$ represents the number of objects in dataset U. Since the reflexivity and symmetry of $T_P(o_i)$, there are lot of repeated calculations when calculating ${\displaystyle \sum _{i=1}^{\left|U\right|}|T_P(o_i)}|$. In order to reduce the amount of calculation, we propose Definition 4 as follows:

Definition 4. 

Suppose $IDS=(U,C\cup D,V,f)$ is an incomplete decision system, $\forall P\subseteq C$. $C{T}_P(o_i)$ is the simplified tolerance relation of object $o_i$ with respect to P, which is defined as follows:

$$\begin{array}{ll}C{T}_P(o_i)& =\{o_j|\forall a\in P ,(f(a,o_i)=f(a,o_j) \vee ( f(a,o_i)\\ & =*\wedge f(a,o_j)=*)\wedge (i<j)) \vee (f(a,o_i)=*\wedge f(a,o_j)\ne *)\}\end{array}.$$

(7)

Definition 5. 

Given an incomplete decision system IDS=$(U,C\cup D,V,f)$, $\forall P\subseteq C$ and $o\in U$, the simplified tolerance class of object $o$ with respect to attribute P is defined as follows:

$$C{T}_P(o)=\{y|(o,y)\in CT(P)\}.$$

(8)

We can delete the symmetric element pair and reflexive element pair from Definition 3 and obtain Definition 5, so that Definition 5 does not have the characteristics of symmetry and reflexivity.

Definition 6. 

Given an incomplete decision system $IDS=$$(U,C\cup D,V,f),\forall P\subseteq C,o\in U,$$C{T}_P(o)$, is the simplified tolerance class of object $o$ with respect to attribute P. The equal knowledge granularity of P on U is defined as follows:

$$EG{K}_U(P)=\frac{1}{|U{|}^2}{\displaystyle \sum _{o\in U}^{}|C{T}_P(o)}|.$$

(9)

Theorem 1. 

Given an incomplete decision system $IDS=$$(U,C\cup D,V,f)$,$\forall P\subseteq C$. Let $U/P=\{X_1,X_2,\cdots ,X_l\},X_i\subseteq$ $U/P$, $\left|X_i\right|=n_i$ where $1\prec i\prec l$. All objects in subdivision $X_l$ are missing a value on attribute P, $\left|X_l\right|=n_{\ast }$. For the convenience of the following description, we mark $X_l$ as $X_{\ast }$. Objects with all non-missing values on attribute P are marked with $\overline{X_{\ast }}$. $EG{K}_U(P)$ represents the equal knowledge granularity of P on U, we have:

$$EG{K}_U(P)=\frac{1}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^l{C}_{n_i}^2}+\left|X_{\ast }\right|\left|\overline{X_{\ast }}\right|\right)$$

(10)

where $C_n^2=\frac{n(n-1)}{2}$.

Proof. 

Suppose that $\forall o\in X_i$ and $\forall o_{\ast }\in X_{\ast }$. According to Definition 4, we can obtain ${\displaystyle \sum _{o\in X_i}\left|C{T}_P(o)\right|}$ $=\frac{n_i(n_i-1)}{2}+n_i{n}_{\ast }$ and ${\displaystyle \sum _{o_{\ast }\in X_{\ast }}\left|C{T}_P(o_{\ast })\right|}=\frac{n_{\ast }(n_{\ast }-1)}{2}$. Suppose $\forall o\in U$, according to Definition 5, then we can obtain $EG{K}_U(P)=\frac{1}{|U{|}^2}{\displaystyle \sum _{o\in U}\left|C{T}_P(o)\right|}=\frac{1}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^{l-1}{\displaystyle \sum _{o\in X_i}\left|C{T}_P(o)\right|}}\right.+$ $\left.{\displaystyle \sum _{o_{\ast }\in X_{\ast }}\left|C{T}_P(o_{\ast })\right|}\right)=\frac{1}{{\left|U\right|}^2}·$$\left({\displaystyle \sum _{i=1}^{l-1}\frac{n_i(n_i-1)}{2}}+n_{\ast }{\displaystyle \sum _{i=1}^{l-1}{n}_i}+\frac{n_{\ast }(n_{\ast }-1)}{2}\right)$$=\frac{1}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^l\frac{n_i(n_i-1)}{2}}+n_{\ast }\left(\left|U\right|-n_{\ast }\right)\right)$. Since $C_{n_i}^2=\frac{n_i(n_i-1)}{2}$, $C_{n_{\ast }}^2=\frac{n_{\ast }(n_{\ast }-1)}{2}$ and $\left|\overline{X_{\ast }}\right|=\left|U\right|-n_{\ast }$, we can obtain $EG{K}_U(P)==\frac{1}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^l{C}_{n_i}^2}+\left|X_{\ast }\right|\left|\overline{X_{\ast }}\right|)\right).$ $\square$

Property 1. 

Given an incomplete decision system IDS= $(U,C\cup D,V,f)$, $\forall P\subseteq C$. If the knowledge granularity of P is $G{K}_U(P)$ on U and the equal knowledge granularity of P is $EG{K}_U(P)$, then we have:

$$EG{K}_U(P)=(G{K}_U(P)-\frac{1}{\left|U\right|})/2.$$

(11)

Proof. 

Let $U/P=\{X_1,X_2,\cdots ,X_{l-1},X_{\ast }\}$, $\left|X_i\right|=n_i,\left|X_{\ast }\right|$ $=n_{\ast }$ where $X_i$ is the i-th subdivision of $U/P$ and $X_{\ast }$ stands for the subdivision of missing values on attribute P, $\left|X_i\right|$ stands for the number of object in subdivision $X_i$. We can obtain $\left|U\right|-{\displaystyle \sum _{i=1}^{l-1}\left|X_i\right|}=$ $\left|X_{\ast }\right|$. According to Definition 2, suppose that $\forall o\in X_i$ and $T_P(o)=\{x|x\in X_i\vee x\in X_{\ast }\}$, we can obtain $\left|T_P(o)\right|$ = $n_i+n_{\ast }$. Since the $\left|T_P(o)\right|$ value of each object in $X_i$ is $n_i+n_{\ast }$, we can obtain ${\displaystyle \sum _{o\in X_i}\left|T_P(o)\right|}=n_i(n_i+n_{\ast })$. In the same way, we can obtain ${\displaystyle \sum _{o\in X_{\ast }}\left|T_P(x)\right|}=n_{\ast }\left|U\right|$. According to Definition 3, we can obtain $G{K}_U(P)=\frac{1}{{\left|U\right|}^2}({\displaystyle \sum _{i=1}^{l-1}{\displaystyle \sum _{o\in X_i}^{}|T_P(o)}|}+$ ${\displaystyle \sum _{o\in X_{\ast }}\left|T_P(o)\right|})=\frac{1}{{\left|U\right|}^2}·$ $\left[{\displaystyle \sum _{i=1}^{l-1}{n}_i(n_i+n_{\ast })}+n_{\ast }\left|U\right|\right]$, then $G{K}_U(P)-\frac{1}{\left|U\right|}=\frac{1}{{\left|U\right|}^2}({\displaystyle \sum _{i=1}^{l-1}(n_i^2-n_i)+{\displaystyle \sum _{i=1}^{l-1}{n}_i}}+n_{\ast }{\displaystyle \sum _{i=1}^{l-1}{n}_i}+n_{\ast }\left|U\right|-\left|U\right|)=$ $\frac{1}{{\left|U\right|}^2}({\displaystyle \sum _{i=1}^{l-1}(n_i^2-n_i)}-n_{\ast }^2-n_{\ast }+2n_{\ast }$ $\left|U\right|)=\frac{2}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^{l-1}\frac{(n_i^2-n_i)}{2}}+\frac{n_{\ast }^2-n_{\ast }}{2}+n_{\ast }(\left|U\right|-n_{\ast })\right)=2EG{K}_U(P)$. Let $\left|X_l\right|$ = $\left|X_{\ast }\right|$, we can obtain $G{K}_U(P)-$ $\frac{1}{\left|U\right|}$ = $\frac{2}{{\left|U\right|}^2}\left({\displaystyle \sum _{i=1}^l\frac{(n_i^2-n_i)}{2}}+n_{\ast }(\left|U\right|-n_{\ast })\right)=2EG{K}_U(P).$ □

Due to the time complexity of calculating $T_P(o)$ is $\left|U\right|$, we know that the time complexity of calculating $G{K}_U(P)$ is ${\left|U\right|}^2$. However, the time complexity of $C{T}_P(o)$ is $\left|U/P\right|$, the time complexity of calculating $EG{K}_U(P)$ is ${\left|U/P\right|}^2$, and ${\left|U/P\right|}^2\ll {\left|U\right|}^2$. In addition, in the process of calculating $EG{K}_U(P)$, the sub-division with a cardinality of 1 is constantly pruned, which further speeds up the calculation. Therefore, the time of calculating $EG{K}_U(P)$ is less than $G{K}_U(P)$ for the same data set.

Example 1. 

Example of computing equivalent knowledgegranularity. Let $IDS=(U,C\cup D,V,$$f)$, $U=\{o_1,o_2,o_3,o_4,o_5,o_6,o_7,o_8,o_9\}$, $C=\{a,b,c,e\},D=\left\{d\right\}$. The detailed data are shown in

Table 1. Incomplete information system.

U	a	b	c	e	d
o1	0	0	0	1	0
o2	0	0	1	*	1
o3	0	1	0	1	0
o4	0	1	*	0	1
o5	0	1	0	*	1
o6	*	*	1	1	1
o7	1	0	*	0	2
o8	1	0	1	0	2
o9	*	*	0	1	2

‘*’ represents the missing value about some attributes.

. Let $P=\{a,b\}$, we can obtain $f(o_6,P)=*,$ $f(o_9,P)=*$. We use the following two methods to calculate $EG{K}_U(P)$.

(1)

According to Definition 5, we obtain that $C{T}_P(o_1)=\{o_2,o_6,o_9\},C{T}_P(o_2)=\{o_6,o_9\},C{T}_P$$(o_3)=\{o_4,o_5,o_6,o_9\},C{T}_P(o_4)=\{o_5,o_6,o_9\},C{T}_P(o_5)=\{o_6,o_9\},C{T}_P(o_6)=\left\{o_9\right\},C{T}_P(o_7)=\{o_6,o_8,o_9\}$$,C{T}_P(o_8)$$=\{o_6,o_9\},C{T}_P(o_9)=\{\varnothing \}$. According to Definition 6, we can obtain $EG{K}_U(P)=\frac{1}{{\left|U\right|}^2}·$${\displaystyle \sum _{i=1}^9\left|C{T}_P(o_i)\right|}$$\frac{1}{9^2}(3+2+4+3+2+3+2+1+0)=\frac{20}{81}$.

(2)

Since $U/P=\left\{\right\{o_1,o_2\},\{o_3,o_4,o_5\},\{o_7,o_8\},\{o_6,$$o_9\left\}\right\}$, let $X_1=\{o_1,o_2\},X_2=\{o_3,o_4,o_5\},X_3=$$\{o_7,o_8\}$, $X_{\ast }=$$\{o_6,o_9\}$, then $\left|X_{\ast }\right|=2,\left|\overline{X_{\ast }}\right|=\left|U\right|-2=7$. According to Theorem 1, we can obtain $EG{K}_U(P)=\frac{1}{9^2}$$·[C_2^2+C_3^2+C_2^2+C_2^2+2·(9-2)]=\frac{20}{81}$.

Although the above two methods achieve the same results, the calculation time is different. Since method 1 needs to scan the dataset multiple times, it consumes more time. However, method 2 only needs to scan the dataset one time and obtains each subdivision of U/P. According to the number of objects in each subdivision, we can acquire the combination value quickly, and then the value of equivalent knowledge granularity is calculated.

3. The Mechanism of Knowledge Granularity Attribute Reduction in the Background of Clustering

Most traditional attribute reduction models use equivalence class relation to compute the importance of conditional attributes. Although these methods can effectively deal with complete decision-making information systems, they cannot obtain correct reduction rules in incomplete ones. In order to deal with the loss of information effectively, this paper focuses on the reduction in incomplete decision systems.

The traditional reduction model does not consider the impact on the classification of the dataset when deleting redundant attributes. If there are inconsistent objects in the dataset, the classification accuracy of the reduced set will be affected. In order to improve the data quality, this paper uses the idea of clustering. Clustering is to divide all objects in the dataset into different clusters according to a certain standard when the target category is unknown. Objects within a cluster are as similar as possible, and objects between clusters are as dissimilar as possible. Classification is to classify all objects in the dataset according to a certain nature and level when the object category is known. Good clustering results can be used as a reference standard for accurate classification. The desired results of clustering involve the objects of the same class being gathered in intra-clustering, otherwise they will be gathered in different inter-clustering. This paper studies the labeled data objects decision information system. Therefore, we use the results of the classification to guide the process of clustering the data objects. When the data objects are clustered, they follow the principle that the objects of intra-clustering are as close as possible and the objects of inter-clustering are as far away as possible. Next, we discuss how to measure the distance of intra-clustering and inter-clustering objects.

3.1. The Intra-Cluster Similarity for Incomplete Systems

Generally, there are two approaches for clustering calculations: distance and similarity. The closer the distance between two different objects is, the weaker their ability to distinguish is. On the contrary, the farther the distance is, the stronger the ability to distinguish is. In this paper, the similarity method is used to measure the distinguishing ability of objects. Since the knowledge granularity can measure the similarity between objects, the coarser the knowledge granularity is, the stronger the distinguishing ability is. The better the knowledge granularity is, the weaker the distinguishing ability is. Next, we discuss how to use knowledge granularity information to measure the similarity of objects in an incomplete system.

Definition 7 

(The similarity of intra-cluster objects). Given an incomplete decision system $IDS=(U,C\cup D,V$$,f)$, $U/D=\{D_1,D_2,\cdots ,D_n\}$. For the sake of convenience, let $U/D={\pi }_D,$ $D_i\subseteq {\pi }_D$, $P\subseteq C$. Suppose the equivalence division relationship of $D_i$ under attribute set P is ${ℝ}_P=\{X_1,X_2,\cdots ,X_m\}$; then, the similarity of objects in the cluster of $D_i$ about attribute P is defined as follows (where $o\in D_i$):

$$SIntr{a}_{D_i}(P)=EG{K}_{D_i}(P)=\frac{1}{|D_i{|}^2}{\displaystyle \sum _{o\in D_i}\left|C{T}_P(o)\right|}.$$

(12)

Definition 8 

(The average similarity of intra-cluster objects). Given an incomplete decision system $IDS=(U,C\cup D,V,f),$ $U/D=\{D_1,D_2,\cdots ,D_n\}$. $P\subseteq C$, $o\in D_i$. The knowledge granularity similarity of intra-clustering objects for subdivision $D_i$ with respect to attribute P is $SIntr{a}_{D_i}(P)$, then the average intra-clustering similarity is defined as follows:

$$ASIntr{a}_{{\pi }_D}(P)=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}SIntr{a}_{D_i}(P)}.$$

(13)

The desired effect of clustering is that the similarity of intra-clustering is high, and the similarity of inter-clustering is low.

Property 2. 

Given an incomplete system $IDS=(U,C\cup D,V,f),U/D={\pi }_D,D_i\subseteq {\pi }_D,P,Q\subseteq C$. If $P\subseteq Q$, and we have

$$ASIntr{a}_{{\pi }_D}(P)\ge ASIntr{a}_{{\pi }_D}(Q).$$

(14)

Proof. 

Let $D_i/P=\{X_1,X_2,\cdots ,X_k\cup X_{k+1},X_{k+2},\cdots ,X_n,$ $X_{\ast }\cup Y\}$, where $X_{\ast },Y$ represents the object sets with missing values on attribute set P. Since $P\subseteq Q$, we can obtain $D_i/Q\subseteq D_i/P$. This to say, each subdivision of $D_i/Q$ is a subset of some subdivision of $D_i/P$. Let $D_i/Q=\{X_1,$ $X_2,\cdots ,X_k,X_{k+1},X_{k+2},\cdots ,X_n,Y,X_{\ast }\}$. According to Theorem 1 and Definition 7, we can obtain: $SIntr{a}_{D_i}(P)=\frac{2}{{\left|D_i\right|}^2}({\displaystyle \sum _{j=1}^{k-1}{C}_{\left|X_j\right|}^2}+C_{\left|X_k\right|+\left|X_{k+1}\right|}^2+{\displaystyle \sum _{j=k+2}^n{C}_{\left|X_j\right|}^2+C_{\left|X_{\ast }\right|+\left|Y\right|}^2}+(\left|X_{\ast }\right|+\left|Y\right|)(\left|D_i\right|-\left|X_{\ast }\right|-\left|Y\right|))$ $=\frac{2}{{\left|D_i\right|}^2}({\displaystyle \sum _{j=1}^{k-1}{C}_{\left|X_j\right|}^2}+C_{\left|X_k\right|}^2+C_{\left|X_{k+1}\right|}^2+\left|X_k\right|\left|X_{k+1}\right|+{\displaystyle \sum _{j=k+2}^n{C}_{\left|X_j\right|}^2+}{C}_{\left|X_{\ast }\right|}^2+C_{\left|Y\right|}^2+\left|X_{\ast }\right|\left|Y\right|+(\left|X_{\ast }\right|+\left|Y\right|)(\left|D_i\right|-\left|X_{\ast }\right|$ $-\left|Y\right|))=\frac{2}{{\left|D_i\right|}^2}({\displaystyle \sum _{j=1}^n{C}_{\left|X_j\right|}^2}+\left|X_k\right|\left|X_{k+1}\right|+C_{\left|X_{\ast }\right|}^2+C_{\left|Y\right|}^2$ $+\left|X_{\ast }\right|\left|Y\right|+(\left|X_{\ast }\right|+\left|Y\right|)(\left|D_i\right|-\left|X_{\ast }\right|-\left|Y\right|))$. Since $SIntr{a}_{D_i}(Q)=\frac{2}{{\left|D_i\right|}^2}({\displaystyle \sum _{j=1}^n{C}_{\left|X_j\right|}^2}+C_{\left|X_{\ast }\right|}^2+C_{\left|Y\right|}^2+\left|X_{\ast }\right|(\left|D_i\right|-\left|X_{\ast }\right|))$, then $\frac{2}{{\left|D_i\right|}^2}(\left|X_k\right|\left|X_{k+1}\right|+\left|Y\right|\left|D_i\right|-$ $\left|X_{\ast }\right|\left|Y\right|-{\left|Y\right|}^2)=\frac{2}{{\left|D_i\right|}^2}(\left|X_k\right|\left|X_{k+1}\right|+\left|Y\right|(\left|D_i\right|-\left|X_{\ast }\right|-\left|Y\right|))$. Since $\left|D_i\right|>=$ $\left|X_{\ast }\right|+\left|Y\right|$ $,\left|X_k\right|\left|X_{k+1}\right|\ge 0$; therefore, the results of $SIntr{a}_{D_i}(P)\ge$ $SIntr{a}_{D_i}(Q)$ and ${\displaystyle \sum _{i=1}^{n}SIntr{a}_{D_i}(P)}\ge {\displaystyle \sum _{i=1}^{n}SIntr{a}_{D_i}(Q)}$ are obtained. Above all, we can obtain $ASIntr{a}_{{\pi }_D}(P)\ge ASIntr{a}_{{\pi }_D}(Q)$. □

According to Property 2, we conclude that the intra-cluster similarity is monotonic when the conditional attribute set changes.

Example 2. 

In Table 1, let $P=\{a,b\}$; we have $D_1=\{o_1,o_3\},D_2=\{o_2,o_4,o_5,o_6\},D_3=\{o_7,o_8,o_9\},$$SIntr{a}_{D_1}(P)=0,$$SIntr{a}_{D_2}(P)=\frac{0+C_2^2+1\times 3}{4^2}=\frac{1}{4},SIntr{a}_{D_3}(P)=\frac{C_2^2+1\times 2}{3^2}=\frac{1}{3}$, $ASIntr{a}_{{\pi }_D}(P)=$$\frac{1}{3}{\displaystyle \sum _{i=1}^{3}SIntr{a}_{D_i}(P)}=\frac{7}{36}$.

3.2. The Inter-Cluster Similarity for Incomplete Systems

Definition 9 

(The inter-cluster similarity for incomplete systems). Given $IDS=(U,C\cup D,V,f)$ is an incomplete decision system, let ${\pi }_D=\{D_1,D_2,\cdots ,D_n\}$. Suppose $P\subseteq C$, $D_i,D_j\subset U/D$, then the inter-cluster similarity of $D_i$ and $D_j$ with respect to attribute set P for incomplete systems is defined as the following:

$$SInte{r}_{D_i,D_j}(P)=\frac{1}{{(\left|D_i\right|+\left|D_j\right|)}^2}\left({\displaystyle \sum _{o\in D_i\cup D_j}\left|C{T}_P(o)\right|}-\right.\left.{\displaystyle \sum _{o\in D_i}\left|C{T}_P(o)\right|}-{\displaystyle \sum _{o\in \cup D_j}\left|C{T}_P(o)\right|}\right).$$

(15)

Assuming that $D_i$ and $D_j$ are objects of two different clusters, the inter-cluster similarity between $D_i$ and $D_j$ is calculated in two steps. The first step is to calculate the similarity after the two clusters are merged, and the second step is to remove the similarity information of the same cluster. The rest is the similarity information between objects in different clusters.

Property 3. 

Given an incomplete decision system $IDS=(U,C\cup D,V,f),{\pi }_D=\{D_1,D_2,\cdots ,D_n\}$$P\subseteq C$. Let $D_i/P$$=\{X_1,X_2,\cdots ,X_k,X_{k+1},\cdots ,X_m,X_{\ast }\},\overline{X_{\ast }}=D_i-X_{\ast },D_j/P=\{Y_1,Y_2,\cdots ,Y_k,Y_{k+1},$$\cdots ,$$Y_n,Y_{\ast }\},\overline{Y_{\ast }}=D_j-Y_{\ast },D_i\cup D_j/P=\{X_1\cup Y_1,X_2\cup Y_2,\cdots ,X_k\cup Y_k,X_{\ast }\cup Y_{\ast },X_{k+1},\cdots ,X_m,Y_{k+1},$$\cdots ,Y_n\}$, where $\forall o\in X_{\ast }\cup Y_{\ast },f(o,P)=*$, ‘*’ is the flag of missing value. We have:

$$SInte{r}_{D_i,D_j}(P)=\frac{1}{{(\left|D_i\right|+\left|D_j\right|)}^2}\left({\displaystyle \sum _{l=1}^k\left|X_l\right|\left|Y_l\right|}++\left|X_{\ast }\right|\left|Y_{\ast }\right|+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|\right).$$

(16)

Proof. 

According to Definition 5 and Theorem 1, we can obtain ${\displaystyle \sum _{o\in D_i\cup D_j}\left|C{T}_P(o)\right|}={\displaystyle \sum _{l=1}^k{C}_{\left|X_l\right|+\left|Y_l\right|}^2}+$ $C_{\left|X_{\ast }\right|+\left|Y_{\ast }\right|}^2+{\displaystyle \sum _{l=k+1}^m{C}_{\left|X_i\right|}^2+{\displaystyle \sum _{l=k+1}^n{C}_{\left|Y_i\right|}^2+}}(\left|X_{\ast }\right|+\left|Y_{\ast }\right|)(\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|),{\displaystyle \sum _{o\in D_i}\left|C{T}_P(o)\right|}={\displaystyle \sum _{l=1}^m{C}_{\left|X_l\right|}^2}+C_{\left|X_{\ast }\right|}^2+\left|X_{\ast }\right|\left|\overline{X_{\ast }}\right|$ $,{\displaystyle \sum _{o\in D_j}\left|C{T}_P(o)\right|}=$ ${\displaystyle \sum _{l=1}^n{C}_{\left|Y_l\right|}^2}+C_{\left|Y_{\ast }\right|}^2+\left|Y_{\ast }\right|\left|\overline{Y_{\ast }}\right|$. Since ${\displaystyle \sum _{l=1}^k{C}_{\left|X_l\right|+\left|Y_l\right|}^2}={\displaystyle \sum _{l=1}^k\left(\frac{{\left|X_l\right|}^2-\left|X_l\right|+{\left|Y_l\right|}^2-\left|Y_l\right|}{2}+\left|X_l\right|\left|Y_l\right|\right)}=$ ${\displaystyle \sum _{l=1}^k{C}_{\left|X_l\right|}^2}+{\displaystyle \sum _{l=1}^k{C}_{\left|Y_l\right|}^2}$ $+{\displaystyle \sum _{l=1}^k\left|X_l\right|\left|Y_l\right|}$, according to Definition 9, we can conclude that ${(\left|D_i\right|+\left|D_j\right|)}^2$ $SInte{r}_{D_i,D_j}(P)=$ ${\displaystyle \sum _{o\in D_i\cup D_j}\left|C{T}_P(o)\right|}-{\displaystyle \sum _{o\in D_i}\left|C{T}_P(o)\right|}-$ ${\displaystyle \sum _{o\in \cup D_j}\left|C{T}_P(o)\right|}={\displaystyle \sum _{l=1}^k\left|X_l\right|\left|Y_l\right|}+\left|X_{\ast }\right|\left|Y_{\ast }\right|+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|$ $+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|$. Then, we have:

$$SInte{r}_{D_i,D_j}(P)=\frac{1}{{(\left|D_i\right|+\left|D_j\right|)}^2}\left({\displaystyle \sum _{l=1}^k\left|X_l\right|\left|Y_l\right|}+\left|X_{\ast }\right|\left|Y_{\ast }\right|+\right.\left.\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|\right).\square$$

Property 4. 

Given an incomplete decision system $IDS=$$(U,C\cup D,V,f),{\pi }_D=\{D_1,D_2,\cdots ,D_n\}$, $P\subseteq Q\subseteq C,D_i,D_j\subset U/D$, then we have $SInte{r}_{D_i,D_j}(P)\ge SInte{r}_{D_i,D_j}(Q)$.

Proof. 

Let $D_i/Q=\{X_1,X_2,\cdots ,X_k,X_{k+1},\cdots ,X_m,X_{△},$ $X_{\ast }-X_{△}\},\overline{X_{\ast }-X_{△}}=D_i-X_{\ast }+X_{△},D_j/Q$ $=\{Y_1,Y_2,\cdots ,$ $Y_k,Y_{k+1},\cdots ,Y_n,Y_{△},Y_{\ast }-Y_{△}\},\overline{Y_{\ast }-Y_{△}}=D_j-Y_{\ast }+Y_{△}$. Since $P\subseteq Q\subseteq C$, then $D_i/Q$ is a refinement of $D_i/P,D_j/Q$ is a refinement of $D_j/P$. Let $D_i/P=\{X_1,X_2,\cdots ,X_k\cup X_{k+1},\cdots ,$ $X_m,X_{\ast }\},\overline{X_{\ast }}=D_i-X_{\ast },D_j/P=\{Y_1,Y_2,\cdots ,Y_k\cup Y_{k+1},\cdots ,Y_n,Y_{\ast }\},\overline{Y_{\ast }}=D_j-Y_{\ast }$. Suppose $D_i\cup D_j/Q$ $=\{$ $X_1\cup Y_1,X_2\cup Y_2,\cdots ,X_k\cup Y_k,X_{k+1}\cup Y_{k+1},X_{\Delta }\cup Y_{\Delta },(X_{\ast }-X_{\Delta })\cup (Y_{\ast }-Y_{\Delta }),X_{k+2},\cdots ,X_m,Y_{k+2},\cdots$ $,Y_n\},D_i\cup D_j/P=\{X_1\cup Y_1,X_2\cup Y_2,\cdots ,X_k\cup Y_k,X_{k+1}\cup Y_{k+1},X_{\ast }\cup Y_{\ast },X_{k+2},\cdots ,X_m,Y_{k+2},\cdots ,Y_n\}$, then $SInte{r}_{D_i,D_j}(P)={\displaystyle \sum _{l=1}^{k-1}\left|X_l\right|\left|Y_l\right|}+(\left|X_k\right|+\left|X_{k+1}\right|)(\left|Y_k\right|+\left|Y_{k+1}\right|)+\left|X_{\ast }\right|\left|Y_{\ast }\right|+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|,$ $SInte{r}_{D_i,D_j}$ $(Q)={\displaystyle \sum _{l=1}^{k-1}\left|X_l\right|\left|Y_l\right|}+\left|X_k\right|\left|Y_k\right|+\left|X_{k+1}\right|\left|Y_{k+1}\right|+\left|X_{\Delta }\right|\left|Y_{\Delta }\right|+\left|X-X_{\Delta }\right|\left|Y-Y_{\Delta }\right|+\left|X_{\ast }-X_{\Delta }\right|\left|{\overline{Y_{\ast }-Y}}_{\Delta }\right|+\left|\overline{X_{\ast }-X_{\Delta }}\right|$ $\left|Y_{\ast }-Y_{\Delta }\right|$. $SInte{r}_{D_i,D_j}(P)-SInte{r}_{D_i,D_j}(Q)=\left|X_k\right|\left|Y_{k+1}\right|+\left|X_{k+1}\right|\left|Y_k\right|+\left|X_{\ast }\right|\left|Y_{\ast }\right|+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|-\left|X_{\Delta }\right|$ $\left|Y_{\Delta }\right|-\left|X-\right.\left.X_{\Delta }\right|\left.\left|Y-Y_{\Delta }\right|-\left|X_{\ast }-\right.X_{\Delta }\right|\left|{\overline{Y_{\ast }-Y}}_{\Delta }\right|+\left|\overline{X_{\ast }-X_{\Delta }}\right|\left|Y_{\ast }-Y_{\Delta }\right|$. Since $\left|\overline{X_{\ast }}\right|=\left|D_i\right|-\left|X_{\ast }\right|,\left|\overline{Y_{\ast }}\right|=\left|D_j\right|-$ $\left|Y_{\ast }\right|,\left|\overline{X_{\ast }-X_{\Delta }}\right|=\left|D_i\right|-\left|X_{\ast }\right|+\left|X_{\Delta }\right|,\left|\overline{Y_{\ast }-Y_{\Delta }}\right|=\left|D_j\right|-\left|Y_{\ast }\right|+\left|Y_{\Delta }\right|,\left|X_k\right|\left|Y_{k+1}\right|+\left|X_{k+1}\right|\left|Y_k\right|\ge 0$, then $SInte{r}_{D_i,D_j}(P)$ $-SInte{r}_{D_i,D_j}(Q)\ge \left|X_{\ast }\right|\left|Y_{\ast }\right|+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|-\left|X_{\Delta }\right|\left|Y_{\Delta }\right|-\left|X-\right.\left.X_{\Delta }\right|\left|Y-Y_{\Delta }\right|-\left|X_{\ast }-X_{\Delta }\right|\left|{\overline{Y_{\ast }-Y}}_{\Delta }\right|+$ $\left|\overline{X_{\ast }-X_{\Delta }}\right|\left|Y_{\ast }-\right.$ $\left.Y_{\Delta }\right|$ $=\left|X_{△}\right|·\left|D_j\right|+\left|Y_{\Delta }\right|\left|D_i\right|-\left|X_{\ast }\right|\left|Y_{\Delta }\right|-\left|X_{\Delta }\right|\left|Y_{\ast }\right|=\left|X_{\Delta }\right|(\left|D_j\right|-\left|Y_{\ast }\right|)+\left|Y_{\Delta }\right|·(\left|D_i\right|-\left|X_{\ast }\right|)$. Since $\left|D_j\right|$ $\ge \left|Y_{\ast }\right|,\left|D_i\right|$ $\ge \left|X_{\ast }\right|$, then we have $SInte{r}_{D_i,D_j}(P)-SInte{r}_{D_i,D_j}(Q)\ge 0$. □

From Property 4, it can be concluded that the similarity of inter-clusters has mono- tonicity with the change in conditional attributes.

Definition 10 

(The average similarity of inter-cluster objects). Given an incomplete decision system $IDS=(U,$ $C\cup D,V,f)$,${\pi }_D=\{D_1,D_2,\cdots ,D_n\}$, $P\subseteq C$. In n different clusters, the inter-cluster similarity is calculated between every two clusters and the time of comparisons is $\frac{1}{2}n(n-1)$, then the average knowledge granularity similarity of inter-cluster in dataset U respect of attribute set P is defined as the following:

$$ASInte{r}_{{\pi }_D}(P)=\frac{2}{n(n-1)}{\displaystyle \sum _{i=1}^{n-1}{\displaystyle \sum _{j=i+1}^{n}SInte{r}_{D_i,D_j}(P)}}.$$

(17)

Example 3. 

In Table 1, with the same conditions as Example 2, let $P=\{a,b\}$, we have $D_1=\{o_1,o_3\},D_2=\{o_2,o_4,o_5,o_6\},D_3=\{o_7,o_8,o_9\},SInte{r}_{D_1,D_2}(P)=\frac{1\times 1+1\times 2+2\times 1}{{(2+4)}^2}=\frac{5}{36},SInte{r}_{D_1,D_3}$$(P)=\frac{0+0+1\times 2}{{(2+3)}^2}=\frac{2}{25},SInte{r}_{D_2,D_3}(P)=\frac{0+1\times 2+3\times 1}{{(4+3)}^2}=\frac{5}{49},ASInte{r}_{{\pi }_D}(P)=\frac{1}{3}{\displaystyle \sum _{i=1}^2{\displaystyle \sum _{j=i+1}^{3}SInte{r}_{D_i,D_j}(P)}}$$=\frac{14153}{132300}$.

4. Attribute Reduction of Knowledge Granularity for Incomplete Systems

Traditional attribute reduction methods are mostly aimed at datasets with no missing data. Various datasets in reality are often incomplete due to various subjective or objective factors. Therefore, we researched information systems with missing data and propose corresponding algorithms to improve the data quality of incomplete system reduction sets.

This paper discusses how to design a reduction method with the idea of clustering. For an ideal clustering effect, the objects of inter-cluster should be far away, and the objects of intra-cluster should be close together. Here, similarity is used to measure the distance between two different objects. The higher the similarity is, the closer the objects are. Conversely, the lower the similarity is, the farther the objects are. Based on the above analysis, we designed a formula to measure the importance of attributes as the following:

$$SI{M}_R=SAIntra+\lambda ·(1-ASInter)$$

(18)

where $\lambda$ is the weight, and $1-ASIntra$ is the dissimilarity of intra-cluster objects. We can set the importance of the intra-cluster similarity and inter-cluster similarity by using the size of the parameter $\lambda$. This method requires the adjustment of the parameters continuously, which consumes a lot of time. To this end, we first normalize $ASIntra$ and $ASInter$. Then, the two similarity calculations can be measured within a unified range, avoiding the parameter adjustment process.

4.1. Normalization of Inter-Cluster Similarity and Intra-Cluster Similarity

Given an incomplete decision system $IDS=(U,C\cup D,V,f),{\pi }_D=\{D_1,D_2,\cdots ,D_n\},P\subseteq$ $C,D_i,D_j\subset U/D$. Since the number of elements in each sub-division may be different, then the value range of its equivalent knowledge granularity may also be different. To calculate the average similarity of all subdivisions, they must be calculated in the same domain. For the sake of generality, we normalize the inter-cluster similarity and intra-cluster similarity. According to Definition 7 and Theorem 1, we can obtain $SIntr{a}_{D_i}(P)=EG{K}_{D_i}(P)$. When all data objects in the subdivision $D_i$ are indistinguishable with respect to the attribute set P, $EG{K}_{D_i}(P)=\frac{1}{2}(1-\frac{1}{\left|D_i\right|})$ takes the maximum value.

When all data objects in $D_i$ can be distinguished from each other, $EG{K}_{D_i}(P)=0$ takes the minimum value. So, the result of $EG{K}_{D_i}(P)\in \left[0,\frac{\left|D_i\right|-1}{2\left|D_i\right|}\right]$ is obtained. If the value of $EG{K}_{D_i}(P)$ is mapped to the range [0,1], the correction formula of $SIntr{a}_{D_i}(P)$ is defined as

$$SIntr{a}_{D_i}{(P)}^{\prime }=EG{K}_{D_i}(P)/\left(\frac{\left|D_i\right|-1}{2\left|D_i\right|}\right)=EG{K}_{D_i}(P)\frac{2\left|D_i\right|}{\left|D_i\right|-1}.$$

(19)

The average similarity of intra-cluster objects is corrected as follows:

$$ASIntr{a}_{{\pi }_D}(P)=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}SIntr{a}_{D_i}{(P)}^{\prime }}$$

(20)

$D_i$ and $D_j$ are two object sets of different clusters. Suppose $D_i/P=\{X_1,X_2,\cdots ,X_n,X_{\ast }\}$,$D_j/P=\{Y_1,Y_2,\cdots ,Y_m,Y_{\ast }\}$. If $f(X_l,P)=f(Y_l,P)(1\le l\le k)$, then $D_i\cup D_j/P=\{X_1\cup Y_1,X_2\cup Y_2,$ $\cdots ,X_k\cup Y_k,\cdots ,X_{k+1},\cdots ,X_n,Y_{k+1},\cdots ,Y_m,X_{\ast },Y_{\ast }\}$. According to Property 3, the similarity of inter-cluster respect to $D_i$ and $D_j$ is denoted as

$$SInte{r}_{D_i,D_j}(P)=\frac{1}{{(\left|D_i\right|+\left|D_j\right|)}^2}\left({\displaystyle \sum _{l=1}^k\left|X_l\right|\left|Y_l\right|}+\left|X_{\ast }\right|\left|\overline{Y_{\ast }}\right|+\left|\overline{X_{\ast }}\right|\left|Y_{\ast }\right|\right).$$

When ${\displaystyle {\cup }_{i=1}^k{X}_i\cup Y_i}=\varnothing ,$$X_{\ast }=\varnothing ,Y_{\ast }=\varnothing \},SInte{r}_{D_i,D_j}(P)=0$, takes the minimum value. When all data objects in $D_i$ and $D_j$ are indistinguishable, $SInte{r}_{D_i,D_j}(P)=\frac{\left|D_i\right|\left|D_j\right|}{{(\left|D_i\right|+\left|D_j\right|)}^2}$ takes the maximum value. Then, we can obtain $SInte{r}_{D_i,D_j}(P)\in \left[0,\frac{\left|D_i\right|\left|D_j\right|}{{(\left|D_i\right|+\left|D_j\right|)}^2}\right]$. The normalized formula of $SInte{r}_{D_i,D_j}(P)$ is as follows:

$$SInte{r}_{D_i,D_j}{(P)}^{\prime }=SInte{r}_{D_i,D_j}(P)\frac{{(\left|D_i\right|+\left|D_j\right|)}^2}{\left|D_i\right|\left|D_j\right|}.$$

(21)

The definition of the average similarity of inter-cluster objects is revised as follows:

$$ASInte{r}_{{\pi }_D}(P)=\frac{n(n-1)}{2}{\displaystyle \sum _{i=1}^{n-1}{\displaystyle \sum _{j=i+1}^{n}SInte{r}_{D_i,D_j}{(P)}^{\prime }}}.$$

(22)

After the similarities of inter-cluster and intra-cluster objects are normalized, $SI{M}_R=ASInter+\lambda ·(1-ASIntra)$ is revised as follows:

$$SI{M}_R=AIntr{a}_{{\pi }_D}(P)+1-AInte{r}_{{\pi }_D}(P).$$

(23)

Since $ASInte{r}_{{\pi }_D}(P)$ represents the similarity of intra-cluster objects, then the dissimilarity is 1−$ASInte{r}_{{\pi }_D}(P)$. If you use the formula of $SI{M}_R$ to measure the effect of clustering, the larger the value of $SI{M}_R$ is, the better the effect is.

4.2. The Knowledge Granularity Attribute Reduction Algorithm for Incomplete Systems (IKAR)

In Section 4.1, we discussed the similarities of inter-cluster and intra-cluster objects from the perspective of clustering, which provided a clear goal for the next step of attribute selection.

Definition 11 

(Equal knowledge granularity attribute reduction). Given an incomplete decision system $IDS=(U,$ $C\cup D,V,f)$, an attribute subset $R\subseteq C$ is an equal know-ledge granularity attribute reduction if and only if:

$$\begin{gathered} (1)R=\underset{P\subseteq C}{min}\left\{SI{M}_P\right\} \\ (2)\forall R^{\prime }\subset R,SI{M}_{R^{\prime }}>SI{M}_R \end{gathered}$$

(24)

In Definition 11, condition (1) is the jointly sufficient condition that guarantees that the equal knowledge granularity value induced from the reduction is minimal, and condition (2) is the individual necessary condition that guarantees the reduction is minimal.

According to Definition 11, we can find a smaller reduction set with an ideal classification effect. Property 2 proves that when the attribute decreases, the similarity of intra-cluster increases monotonically. Property 4 proves that the similarity of inter-cluster also increases when decreasing the attribute, so that the dissimilarity will decrease. Obviously, the formula $SI{M}_R$ cannot determine its monotonicity. To find the R when the $SI{M}_R$ is the largest in the conditional attribute, which is a combinatorial optimization problem, and trying the methods one by one is not the best way to solve the problem. So, we use a heuristic method to find the smallest reduced set. For any attribute $a\in C$, its inner significance is defined as follows:

$$Si{g}_C^a=SI{M}_C-SI{M}_{C-a}$$

(25)

The bigger the value of $Si{g}_C^a$ is, the more important the attribute is.

In order to obtain the optimal reduction set quickly, we adopt the deletion strategy. Firstly, the importance $Si{g}_C^a$ of attribute a is defined by the formula of $SIM$; then, sort the different $Si{g}_C^a$. Secondly, let R = C. The value of $SI{M}_C$ is used as the initial condition, which ensures that the clustering effect after reduction is better than the raw dataset. Then, remove the unimportant attribute a from the remaining attributes $C-R$ and calculate the value of $SI{M}_{R-a}$. If $SI{M}_{R-a}\ge SI{M}_R$, delete attribute a and continue; otherwise, the algorithm terminates. The details of the IKAR algorithm are shown in Figure 1.

Example 4. 

In Table 1, since $D_1=\{o_1,o_3\},D_2=\{o_2,o_4,$$o_5,o_6\},D_3=\{o_7,o_8,o_9\}$. Let $R=C$, according to the definition of $SI{M}_R$, we can obtain $SI{M}_R=\frac{72}{75}$, $SI{M}_{C-a}=\frac{71}{72}$, $SI{M}_{C-b}=\frac{100}{72}$, $SI{M}_{C-c}=\frac{60}{72}$ and $SI{M}_{C-e}=\frac{80}{72}$, then $Si{g}_C^a=\frac{4}{72}$, $Si{g}_C^b=-\frac{25}{72}$, $Si{g}_C^c=\frac{15}{72}$, $Si{g}_C^e=-\frac{5}{72}$. Since $SI{M}_{C-b}=\frac{100}{72}>SI{M}_R$, we delete the attribute b from R and let $R=R-b$, $SI{M}_R=\frac{100}{72}$. In the same way, we obtain that $SI{M}_{R-a}$$=\frac{94}{72}$, $SI{M}_{R-c}=\frac{92}{72}$, $SI{M}_{R-e}=\frac{108}{72}$, $Si{g}_R^a=\frac{6}{72}$, $Si{g}_R^c=\frac{8}{72}$ and $Si{g}_R^e=-\frac{8}{72}$. Since $SI{M}_{R-e}=\frac{108}{72}>SI{M}_R$, we delete the attribute e from R and let $R=R-e$. We calculate the value of $SI{M}_{R-a},SI{M}_{R-c}$, obtain the results of $SI{M}_{R-a}=\frac{82}{72}$ and $SI{M}_{R-c}=\frac{100}{72}$. Now, we have $Si{g}_R^a=\frac{18}{72}$ and $Si{g}_R^c=\frac{8}{72}$. If attribute c is deleted $SI{M}_{R-c}=\frac{100}{72}<SI{M}_R$, and the algorithm is terminated. We have $R=\{a,c\}$.

4.3. Time Complexity Analysis

The time complexity of Step 1 is $O(1)$. In Step 2, we calculate the value of $SI{M}_C$ which include $ASIntr{a}_{{\pi }_D}(C)$ and $ASInte{r}_{{\pi }_D}(C)$. The computational time complexity of intra-cluster $SIntr{a}_{D_i}(C)$ similarity is $O(\left|C\right|\left|D_i\right|)$, then the time complexity of calculating $ASIntr{a}_{{\pi }_D}(C)$ is $O(\left|U\right|\left|C\right|)$. Since the time complexity of calculating the similarity $SInte{r}_{D_i,D_j}(C)$ about inter-cluster $D_i$ and $D_j$ is $O((\left|D_i\right|+\left|D_j\right|)\left|C\right|)$, then time complexity of $ASInte{r}_{{\pi }_D}(C)$ is $O({\displaystyle \sum _{i=1}^{\left|U/D\right|-1}{\displaystyle \sum _{j=i+1}^{\left|U/D\right|}(\left|D_i\right|+\left|D_j\right|)\left|C\right|}})=O((\left|U/D\right|-1)\left|U\right|\left|C\right|)$. We can obtain the time complexity of Step 2 is $O(\left|U/D\right|\left|U\right|\left|C\right|)$. In Step 3, the consume time is $O(\left|U/D\right|\left|U\right|\left|C\right|)$. Since Step 4 utilizes the results of Step 3, the time complexity is $O(1)$. Step 5 is to sort the importance $Si{g}_R^a$ of each attribute, and the time complexity is $O(\frac{1}{2}{\left|C\right|}^2)$. Since the time complexity of calculating $Si{g}_R^a$ is $O(\left|U/D\right|\left|U\right|\left|R\right|)$, then the time complexity of Step 5 is $O(\left|U/D\right|\left|U\right|\left|R\right|)$+$O(\frac{1}{2}{\left|C\right|}^2)$. In Step 6, the time complexity of deleting a redundant attribute is $O(\left|U/D\right|\left|U\right|\left|R\right|)$, and Step 6 needs to be executed $\left|C\right|-\left|R\right|$ times, $R\subseteq C$, then the time complexity of Step 6 is $O(\left|U/D\right|\left|U\right|({\left|C\right|}^2-{\left|R\right|}^2)$. In summary, the time complexity of IKAR is $O(\left|U/D\right|\left|U\right|({\left|C\right|}^2-{\left|R\right|}^2)+\frac{1}{2}{\left|C\right|}^2)$.

5. Experiments Results Analysis

In order to evaluate the feasibility and effectiveness of the proposed algorithm in this paper, the complete dataset was preprocessed to obtain incomplete information systems, and many different attribute reduction algorithms were used for reduction. The reduction set obtained in the previous stage was classified and analyzed by multiple classifiers in the Weka Tool. It was compared with three other existing algorithms in terms of reduction set size, running time, accuracy, and algorithm stability. The specific experimental framework is shown in Figure 2.

All of the datasets are displayed in

Table 2. Description of twelve datasets.

ID	Datasets	Abbreviation	|U|	|C|	|D|
1	Promoters	Prom	106	57	2
2	Heart-statlog	Hear	270	13	2
3	Hepatitis	Hepa	155	19	2
4	HandWritten	Hand	5620	64	10
5	Chess kr-kp	Ches	3196	36	2
6	Splice	Spli	3190	61	3
7	Letters	Lett	20,000	17	26
8	Vote	Vote	435	16	2
9	Mushroom	Mush	8124	22	2
10	Qsar	Qsar	1055	42	2
11	Shuttle	Shut	43,500	9	7
12	Satimage	Sati	6435	36	6

. The twelve datasets selected were downloaded from UCI. In Table 2, $\left|U\right|$, $\left|C\right|$ and $\left|D\right|$ represent the number of objects, conditional attributes, and the categories of the decision attribute, respectively. In order to generate an incomplete information system, we deleted 12% attribute values from the raw datasets, and the missing values were randomly and uniformly distributed. The missing value of each attribute was removed with equal probability, which eliminated the impact of the later reduction set on the classification accuracy due to different attribute selection. During data preprocessing prior to the experiment, we kept the discrete data unchanged and discretized the continuous value data. The two classifiers, RF (random forest) and SMO of the Weka v3.8 software platform, were used to demonstrate the classification effect of the reduction set. In subsequent experiments, the reduced sets of each dataset were analyzed for accuracy using the cross method. All the objects in the reduced set were randomly divided into 10 equal parts, one of which was used as the test set, and the remaining 9 were used as the training set. During the classification and analysis process, the default settings of the Weka tool were used for all parameters. In this way, each reduction set was repeated 10 times for the cross experiment. Finally, the size of the reduction set, running time, and classification accuracy obtained by the 10 experiments were averaged. We executed all experiments on a PC with Windows 10, Intel(R) Core(TM) i7-10710U CPU @ 1.10 GHz, 1.61 GHz and 8 GB memory. Algorithms were coded in python and the software that was used is PyCharm Community Edition 2020.2.3 x64.

5.1. Reduction Set Size and Running Time Analysis

This section mainly verifies the feasibility of IKAR for dataset reduction from the perspective of reduction size and calculation time. Here, we have selected three other representative incomplete system attribute reduction approaches. For the convenience of the following description, these three methods are referred to as NGLE [44], IEAR [45], and PARA [46], respectively. NGLE is the neighborhood multi-granulation rough sets-based attribute reduction using Lebesque and entropy, and the method considers both algebraic view and information view. IEAR is the information entropy attribute reduction for incomplete set-valued data using the similarity degree function and proposed $\lambda$-information entropy. PARA is the positive region attribute reduction using indiscernibility and discernibility relation.

The attribute reduction size of four algorithms is shown in

Table 3. The attribute reduction size with the four methods on the twelve UCI datasets.

Datasets	IKAR	NGLE	IEAR	PARA
Prom	5	4	5	5
Hear	10	10	9	10
Hepa	9	10	7	10
Hand	12	11	10	11
Ches	29	30	29	30
Spli	9	10	9	9
Lett	9	9	8	9
Vote	10	9	8	10
Mush	4	5	5	5
Qsar	31	30	29	31
Shut	4	5	4	5
Sati	10	10	9	11
Ave	11.833	11.917	11.00	12.083
Best	4	1	11	1

.

Table 4. The run time with the four methods on the twelve UCI datasets.

Datasets	IKAR	NGLE	IEAR	PARA
Prom	0.98	2.042	3.231	1.553
Hear	0.03	0.16	2.09	0.33
Hepa	0.06	0.11	2.13	0.25
Hand	92.93	81.61	3222.89	91.28
Ches	6.35	140.98	3075.25	590.79
Spli	35.43	205.86	6557.61	309.79
Lett	3.98	6.93	117.87	8.47
Vote	0.06	0.38	6.09	1.01
Mush	4.45	411.09	4544.04	827.16
Qsar	3.84	4.97	98.76	6.61
Shut	8.98	886.08	3974.70	1133.25
Sati	25.30	125.61	2322.02	154.04
Ave	15.20	155.49	1993.89	260.38
Best	11	1	0	0

shows the time consumed by the reduction process of the four algorithms in seconds. Ave represents the average value, Best in Table 3 represents the number of minimum reduction sets obtained, and Best in Table 4 stands for the number of times which the running time was the shortest. From Table 3, it can be seen that the average reduction set size of the IKAR algorithm is 11.833, and the average reduction set size obtained by the NGLE, IEAR, and PARA algorithms are 11.917, 11.00, and 12.083, respectively. IKAR obtained the minimum reduction set on the Ches, Spli, Mush, and Shut datasets, and the reduction effect was slightly better than the NGLE and PARA, but not as ideal as the IEAR algorithm. In the 12 datasets in Table 2, the IEAR algorithm obtains the minimum reduction set in 11 datasets.

From the consumption time of Table 4, we can find that the reduction advantage obtained by the IKAR algorithm is not obvious, but that the IKAR algorithm is obviously better than the other three algorithms. When calculating the reduction set of 12 datasets, the average time required by IKAR is 15.20 s, whilst the average times consumed by the other three algorithms are (155.40, 2003.56, and 260.38) seconds, respectively. From the experimental results in Table 4, it can also be found that the IKAR algorithm only needs 25.30 s to reduce the Shut dataset, and the running time of the other three algorithms of NGLE, IEAR, and PARA is (886.08, 3974.02, and 133.25) seconds. When reducing the Mush dataset, IKAR takes 4.45 s, and the other three algorithms take (411.09, 4544.04, and 827.16) seconds. Of course, the IKAR algorithm also has shortcomings, and when the number of sample categories is higher, the algorithm is more time-consuming. For example, if there are 10 different categories of samples in the Hand dataset, the calculation time of the IKAR algorithm takes 92.93 s. At this time, the time consumption of the NGLE and PARA algorithms is (81.61 and 91.28) seconds, respectively. It takes less time than that of the IKAR algorithm.

From the above analysis, we can obtain that the IKAR algorithm can effectively reduce the dataset, it is obviously better than similar algorithms in terms of reduction speed.

5.2. Changes of the Classification Accuracy When Missing Value

It is not enough to evaluate the overall performance of a reduction algorithm only from the size and running time of the reduction set. Here, we further analyze the performance of the above four algorithms from the perspective of the classification accuracy of the reduction set. In order to find out the influence of missing values on the IKAR algorithm, we selected four datasets in Table 2, including Heas, Prom, Hepa, and Stai. During the experiment, we divided the missing values into 5 different levels, 2%, 4%, 6%, 8%, and 10%. First, 2% of the data objects in the original dataset were randomly selected, and random deletion is conducted in these data objects of attribute values for different attributes to generate an incomplete dataset. In order to reduce the bias of attribute selection that may be caused by random deletion, we used the 2% data just obtained as a basis, then selected attributes with the same probability, and then deleted the 2% data, which generated an incomplete dataset with a missing value of 4% and so on, generating a dataset with a missing value of 6%, 8%, and 10%, respectively. After the incomplete dataset was ready, we used IKAR, NGLE, IEAR, and PARA algorithms for reduction and used SMO and RF classifiers to analyze the classification accuracy. The specific results are shown in Figure 3 and Figure 4. The horizontal axis in Figure 3 and Figure 4 represent the proportion of deletion data objects in the dataset, and the vertical axis represents the classification accuracy.

Figure 3 and Figure 4 show the change trend diagrams obtained by using the SMO and RF classifiers to analyze the accuracy of the reduced dataset. From the results of Figure 3 and Figure 4, it can be seen that when increasing the missing data, the classification accuracy of the above four algorithms has a downward trend. For example, under the SMO classifier, when the proportion of missing values in the dataset Heas changes from 2% to 4%, the accuracy changes in IKAR and other NGLE, IEAR and PARA algorithms are (84.01→83.69), (80.32→79.69), (77.01→76.86), and (79.24→78.92). Under the RF classifier, when the proportion of missing values in the dataset Heas changes from 2% to 4%, in the IKAR and other NGLE, IEAR, and PARA algorithms, the accuracy changes are (81.43→80.94), (77.85→76.28), (74.79→73.33), and (76.24→74.86), respectively. The main reason for this phenomenon is that as the proportion of missing data increases, more and more data objects cannot be distinguished, resulting in a decrease in classification accuracy. From the trend diagrams in Figure 3 and Figure 4, it can be seen that the IKAR algorithm changes smoothly. The other three algorithms have a greater impact on the classification accuracy of the reduced set as the proportion of missing data increases. For example, under the RF classifier, when the missing proportion of the Hepa dataset changes from 2% to 10%, the accuracy of IKAR changes to 2.09, while the accuracy changes of the NGLE, IEAR, and PARA algorithms are 3.2, 3.19, and 4.49, respectively. Under the SMO classifier, on the classification accuracy of the Heas dataset, the IKAR algorithm changes value is 1.12, and for the other three algorithms is 3.37, 2.28, and 2.93, respectively.

5.3. Classification Accuracy Analysis

The previous experiments have compared the changes in the classification accuracy when missing value exist. With the change in the proportion of missing values, the accuracy of the IKAR algorithm can not only change smoothly but can also obtain a better classification effect on multiple datasets. We used the SMO and RF classifiers to analyze the accuracy of the previous reduction set, and the detailed content is shown in Figure 5 and Figure 6, respectively. In Figure 5 and Figure 6, missing values represent the incomplete raw dataset and Ave denotes the average accuracy for the different datasets. Under the classifier SMO, the IKAR algorithm obtained an average accuracy of 85.31%, and the average accuracy obtained by the other three algorithms NGLE, IEAR, PARA, and raw datasets were only 78.72, 78.78, 78.81, 84.49(%), respectively. Among the 12 datasets in Table 2, the IKAR algorithm was the highest 11 times. The classification accuracy of IKAR is higher than that of the raw set.

For example, the classification accuracy of IKAR is (94.76, 99.68, 93.86)% in the datasets of Hand, Shut, and Spli. The classification accuracy of IEAR is (81.59, 85.93, 81.53)% and the average accuracy is 10 percentage points lower than IKAR. Similarly, on the RF classifier, the average classification accuracy of the IKAR (86.24%) is higher than the raw dataset (85.09%) and those obtained by the other three algorithms obtained (80.79, 79.77, 81.14)%, respectively.

From the results shown in Figure 5 and Figure 6, we can obtain three conclusions as follows: (1) The classification accuracies obtained by the above four attribute algorithms are closer to those of the raw dataset, which indicates that all four algorithms are able to effectively reduce the incomplete dataset. (2) The algorithm has better overall performance. The main reason is that the IKAR algorithm considers both the correlation between attributes and the influence of attributes on the classification two factors, so it obtains a more ideal classification accuracy. (3) The classification accuracy of the IKAR algorithm is higher than the original dataset, and the main reason for this is that the IKAR algorithm effectively eliminates redundant attributes and reduces the influence of noisy data on classification.

5.4. Lgorithm Stability Analysis

To indicate the statistical significance of classification results, the non-parametric Friedman test and Bonferroni–Dunn test methods were used to the analyze classification accuracy of each classifier with different methods in Section 5.2, where the Friedman test is a statistical test that uses the ranking of each method on each dataset. The Friedman statistic is described as follows:

$${\chi }_F^2=\frac{12N}{t(t+1)}{\displaystyle \sum _{i=1}^s{R}_i^2}-3t(t+1)$$

(26)

$$F_F=\frac{(N-1){\chi }_F^2}{N(t-1)-{\chi }_F^2}$$

(27)

where N is the number of datasets, and t is the categories of algorithms. $R_i$ represents the average rank of the classification effect ranking of the i-th algorithm on all datasets, and the statistics $F_F$ obey the Fisher distribution with t − 1 and (t − 1)(N − 1) degrees of freedom. If the value of $F_F$ is bigger than $F_{\alpha }(t-1,N-1)$, then the original hypothesis does not hold.

The $F_F$ test value can judge where these algorithms are different, but it cannot indicate the superiority of the algorithm. In order to explore which algorithm is better, we used the Bonferroni–Dunn test to calculate the critical value range of the average sequence value difference, which is defined as follows:

$$C{D}_{\alpha }=q_{\alpha }\sqrt{\frac{t(t+1)}{6N}}.$$

(28)

If the difference between the average ranking values of the two algorithms exceeds the critical region $C{D}_{\alpha }$, the hypothesis that ‘the performance of the two algorithms is the same’ will be rejected with the corresponding confidence. Otherwise, the two algorithms perform differently, and the algorithm with the higher average rank is statistically better than the algorithm with the lower average rank. Generally, we set $\alpha$ = 0.05.

In order to compare the stability of the IKAR algorithm in this paper, we chose three other similar algorithms, NGLE, LEAR, and PARA, to reduce the datasets in Table 2. Then, the previous reduction results were analyzed by the classifiers SMO and RF using the Weka tool. The classification accuracy was detected by the Friedman test and Bonferroni–Dunn test. When t = 4 and N = 12, then ${\chi }_F^2$ = 23.4 and $F_F$ = 20.429. Under the classifier SMO, the classification accuracy of the four algorithm reduction sets is sorted, and the number 1 represents the most ideal. The details of the sorting results are shown in

Table 5. Ranking and standard of classification accuracy under SMO classifier.

Datasets	IKAR		NGLE		IEAR		PARA
	STD	Rank	STD	Rank	STD	Rank	STD	Rank
Prom	±0.031	1	±1.362	2	±2.019	4	±1.294	3
Heas	±0.015	1	±1.455	3	±2.318	4	±1.521	2
Hepa	±0.038	1	±2.483	4	±1.985	4	±1.984	3
Hand	±0.125	1	±2.784	2	±2.537	4	±2.183	3
Krkp	±0.247	2	±1.561	4	±1.783	1	±3.420	3
Spli	±0.035	1	±2.489	2	±3.015	4	±3.019	3
Lett	±0.512	1	±3.016	3	±1.985	4	±2.184	2
Vote	±0.187	1	±1.392	2	±2.417	4	±2.478	3
Mush	±0.261	1	±2.765	3	±2.581	4	±2.104	2
Qsar	±0.382	1	±2.054	3	±2.932	4	±3.013	2
Shut	±0.049	1	±1.937	3	±1.995	4	±1.329	2
Sati	±0.536	1	±2.349	3	±3.246	4	±2.586	3
AveRank		1.08		2.75		3.58		2.58

. The average ranking of IKAR, NGLE, IEAR, and PARA algorithms are 1.08, 2.75, 3.58, and 2.58 in turn.

Since the critical value of $F_{0.05}(3,33)$ is 2.892 and $F_F\succ 2.892$, we can reject the original hypothesis at $\alpha =0.05$ under the Friedman test. So, there are statistical differences in classification accuracy among the above four algorithms. Next, $q_{0.05}=$ 2.569 and $C{D}_{0.05}=$ 1.354, and the results of Bonferroni–Dunn test for these four algorithms at $\alpha =0.05$ is shown in Figure 7. The accuracy of the algorithm on the left side of the coordinate axis is relatively high in Figure 7. From Figure 7, we know that the average accuracy ranking of IKAR is s1 = 1.08. Among the other three algorithms, the better-ranking algorithm, PARA, has an average ranking value of s2 = 2.58. Since $\left|s_1-s_2\right|$ = 1.5 and $\left|s_1-s_2\right|\succ C{D}_{0.05}$, we can find that the IKAR algorithm is significantly superior to the PARA. In the same way, the IKAR is better on accuracy than NGLE and LEAR. However, there is no obvious difference in the ranking of the NGLE, LEAR, and PARA algorithms.

For the same reason, under the RF classifier, the average ranking values of the classification accuracy of the above four algorithms are 1.00, 2.83, 3.67, and 2.50 as shown in

Table 6. Ranking and standard of classification accuracy under RF classifier.

Datasets	IKAR		NGLE		IEAR		PARA
	STD	Rank	STD	Rank	STD	Rank	STD	Rank
Prom	±0.181	1	±1.673	2	±2.829	4	±2.639	3
Heas	±0.273	1	±2.442	3	±2.719	4	±2.521	2
Hepa	±0.357	1	±2.873	4	±2.683	2	±3.038	3
Hand	±0.362	1	±2.361	2	±3.731	4	±2.359	3
Krkp	±0.652	1	±2.048	3	±1.984	3	±2.763	2
Spli	±0.248	1	±1.994	2	±3.565	4	±3.558	3
Lett	±0.583	1	±3.246	3	±2.783	4	±3.160	2
Vote	±0.652	1	±1.817	2	±3.015	4	±2.629	3
Mush	±0.393	1	±3.092	3	±2.928	4	±1.972	2
Qsar	±0.488	1	±2.636	2	±1.997	4	±3.215	3
Shut	±0.391	1	±2.743	4	±2.963	3	±2.388	2
Sati	±0.475	1	±1.982	4	±2.837	3	±3.086	3
AveRank		1		2.83		3.67		2.5

. Since ${\chi }_F^2$ = 26.8 and $F_F$ = 32.043, then $F_F\succ F_{0.05}$. We can see that the classification accuracy of these four algorithms is significantly different in the statistical sense under the RF classifier. From Figure 8, we can ascertain that the IKAR algorithm is significantly different from the NGLE classification, while the ranking of the NGLE, LEAR, and PARA algorithms have no obvious differences among each other.

In Table 5 and Table 6, the STD represents the standard deviation of the classification accuracy of different algorithms’ reduction datasets. From Table 5 and Table 6, we know that the STD of the IKAR algorithm is smallest among the four algorithms. The STD of the classification accuracy of algorithm IKAR is less than 1 for both SMO and RF classifiers, while the STD values of the other algorithms are bigger than 1, and some are even greater than 3. For example, the STD of IKAR in the Heas dataset is ±0.015 under the SMO, but the STD values of NGLE, LEAR, and PARA are ±1.455, ±2.318, and ±1.521, respectively. On RF classifiers, the IKAR algorithm has the same stability in classification accuracy. In the dataset Spli, the STD of IKAR’s classification accuracy is ±0.248; meanwhile, the STD values of the other three algorithms are ±1.994, ±3.565 and ±3.558, respectively.

Therefore, all the test results demonstrate that there is no consistent evidence to denote statistical differences between any two of the four approaches under the SMO and RF classifier. In general, the IKAR model is better than the other models in stability.

6. Conclusions

In the face of incomplete systems, most of the traditional attribute reduction models cannot obtain effective reduction rules and affect the classification accuracy of the reduced set. Therefore, we propose a new attribute reduction method, IKAR, based on the clustering background for incomplete system. IKAR uses the tolerance principle to calculate the information of knowledge granularity and to measure the similarity of data objects. When selecting attributes, the similarity of intra-cluster objects should be as large as possible, and the similarity of inter-cluster objects should be as small as possible. The innovations of this paper are manifested in the following four aspects: (1) Use of the tolerance principle to quickly calculate knowledge granularity; (2) Use of knowledge granularity to calculate the similarity of inter-cluster and intra-cluster objects; (3) Use of the idea of clustering to calculate the importance of attributes; (4) In addition to conventional time, space, and precision analysis, it also analyzes the stability of datasets with different percent missing value. All the experiments show that the IKAR algorithm is not only superior in reducing time compared to the other three algorithms, but it also has an excellent performance in terms of accuracy and stability. Of course, the IKAR algorithm also has some shortcomings. For example, it is unsuitable for datasets with multiple decision values, and complex data types and the dynamical changing of the datasets are not considered.

In our future research endeavors, we intend to work in the following four aspects. (1) We focus on attribute reduction of incomplete systems with mixed data types, especially on how to deal with missing data. (2) In addition, we will integrate the incremental learning method into the knowledge granularity reduction model in the background of clustering. (3) To address even big datasets more efficiently, applying GPU and MapReduce technologies to design parallel attribute reduction models or acceleration strategy is a very popular research topic.