Research on a Decision Tree Classification Algorithm Based on Granular Matrices

Abstract

The decision tree is one of the most important and representative classification algorithms in the field of machine learning, and it is an important technique for solving data mining classification tasks. In this paper, a decision tree classification algorithm based on granular matrices is proposed on the basis of granular computing theory. Firstly, the bit-multiplication and bit-sum operations of granular matrices are defined. The logical operations between granules are replaced by simple multiplication and addition operations, which reduces the operation time. Secondly, the similarity between granules is defined, the similarity metric matrix of the granular space is constructed, the classification actions are extracted from the similarity metric matrix, and the classification accuracy is defined by weighting the classification actions with the probability distribution of the granular space. Finally, the classification accuracy of the conditional attribute is used to select the splitting attributes of the decision tree as the nodes to form forks in the tree, and the similarity between granules is used to judge whether the data types in the sub-datasets are consistent to form the leaf nodes. The feasibility of the algorithm is demonstrated by means of case studies. The results of tests conducted on six UCI public datasets show that the algorithm has higher classification accuracy and better classification performance than the ID3 and C4.5.

1. Introduction

Classification is not only a common problem in life but also a hot problem for research in fields such as machine learning and data mining. The main task of classification is to build a model that reflects the mapping relationship between samples and labels by generalizing and learning a series of sample data with labels [1]. The decision tree is one of the most important and representative classification algorithms, and it is an important technique to solve classification tasks in data mining. With the advantages of fast classification speed, a simple rule generation process, and strong interpretive power, the decision tree has received broad interest.

The core idea of the decision tree algorithm is to find the appropriate splitting attributes as the nodes of the tree, divide the dataset to form a fork in the tree, and recursively call the forking process for each subset of data. The final decision tree is generated after all subsets contain the same type of data. The constructed decision tree can then classify new data samples. The key to the decision tree classification algorithm is selecting the splitting attributes. The information gain, Taylor’s formula, the Gini coefficient, and granular computing are the most commonly used attribute selection operators. The splitting attributes were selected through information gain and its improvements in classical ID3, C4.5, E-ID3, R-C4.5, etc. To simplify the logarithmic process, many mathematical methods have been used in the literature to select the splitting attributes, such as Taylor’s formula, McLaughlin’s formula, and equivalence substitution instead of the entropy function [2,3]. The splitting attributes were selected through the Gini coefficient and its improvements in CART [4], SLIQ [5], SPRINT [6], FS-DT [7], model decision trees [8], etc. The splitting attributes were selected through granular computing in FDT [9], IFDT [10], decision trees based on fuzzy sets [11], ordered decision trees [12,13], and decision tree algorithms based on approximation accuracy [14,15], the purity of attributes [16], and the importance of attributes [17], among others. In order to further optimize the classification performance of decision trees, many optimized decision tree classification models have been proposed through the integration of commonly used attribute selection operators with other theories. Examples include a decision tree integrating information gain and the Gini index [18], a decision tree integrating information entropy and the correlation coefficient [19] or the covariance [20], a decision tree based on ant colony optimization [21], and a decision tree with parallelization [22]. These optimized decision tree models extended the decision tree classification method and improved decision tree theory.

Granular computing is a new computing paradigm in the current field of intelligent information processing [23] and is a new method of solving complex problems with multi-granularity, multi-perspective, and multi-level aspects in the form of knowledge granules. Granular computing has attracted much attention because it is in line with the basic idea of human processing and solving complex problems [24]. Granular computing plays an important role in decision tree classification algorithms such as fuzzy sets, rough sets, word computation, quotient spaces, cloud models, etc. [25,26,27,28]. The granular matrix [29] is a new mathematical model proposed on the theoretical basis of granular computing. A granular matrix is a binary representation of the granular structure that presents the classification ability of knowledge and can well solve problems with an incomprehensible nature and poorly intuitive operations.

Based on the above analysis, in this paper, we aim to integrate the algebraic operations of granular matrices into the decision tree algorithm, propose a decision tree classification algorithm based on granular matrices, and give a new decision tree division method based on the similarity measure matrix of the granular structure. The test results show that the algorithm has higher classification accuracy and better classification performance than the ID3 and C4.5.

The main innovative works of this paper are as follows:

(1)

A division of attributes is considered as a granular structure. The “bit-multiplication” and “bit-sum” operations of the granular matrix are defined. The similarity measure matrix of the granular structure is given.

(2)

The method of extracting the classification decision matrix and calculating the classification accuracy of conditional attributes is given and used as the principle of decision tree splitting. This is a new decision tree construction method.

(3)

The process is simplified by the operation mode of binary granules that applies from the mathematical operation of theoretical description to the practical operation of the computer model.

The article is structured as follows.

In Section 2, the granular structure of the decision information system is introduced in detail, these operation rules of the “bit-multiplication” and “bit-sum” are given, and the similarity measure matrix of the granular structure is given. In Section 3, the decision tree classification algorithm based on granular matrices is introduced in detail, and the efficiency of the algorithm is analyzed. In Section 4, a comparative test is given between the presented algorithm and the classical ID3 and C4.5; the test results are analyzed in detail. Finally, a summary is given for the decision tree classification algorithm based on granular matrices, and subsequent research contents are proposed.

2. Basic Concepts

Granular computing is recognized as a new conceptual and computational paradigm for information processing [30]. Granulation, granules, and granularity are the most fundamental concepts in granular computing. Granulation is the division of a theoretical domain by indistinguishable relations, a granule is a block formed by this division, and granularity is a homogenized measure of the coarseness of information.

2.1. Information System

Definition 1 

[31]. An information system is of the form $K=\left(U,AT,V,f\right)$, abbreviated as $K=\left(U,AT\right)$, where $U$ is a dataset, referred to as the domain; $AT$ is the attribute set; and $V={\cup }_{a\in AT}{V}_a$ is the domain of values, i.e.,$\forall x\in U,a\in AT$, we have $f_a\left(x\right)\in V_a$. $f:U\to V$ is called the information function.

In particular, if $AT$ contains the conditional attributes set $C$ and the decision attributes set $D$, i.e., $AT=C\cup D,C\cap D=\varphi$, then $K=\left(U,AT\right)$ is a decision information system or decision table, abbreviated as $K=\left(U,C\cup D\right)$.

For example,

Table 1. A decision information system, K = (U,C$\cup$D).

U	C			D
	Color	Price	Size	Buy
1	White	High	Full	No
2	White	Low	Compact	Yes
3	Black	Low	Compact	No
4	Black	Low	Full	Yes
5	Grey	High	Full	Yes
6	Grey	High	Compact	No

presents a decision information system. Here, $U=\left\{1,2,3,4,5,6\right\}$, conditional attributes set $C=\left\{color,price,size\right\}$, and decision attributes set $D=\left\{Buy\right\}$.

Definition 2 

[31]. In $K=\left(U,AT\right)$, $\forall R\subseteq AT$, an indistinguishable relation $IND\left(R\right)$ is defined: $IND\left(R\right)=\left\{\left(x,y\right)\in U\times U|\forall a\in R,f_a\left(x\right)=f_a\left(y\right)\right\}$. The domain $U$ is divided into $U/IND\left(R\right)=\left\{X_1^R,X_2^R,\cdots ,X_m^R\right\},1\le m\le \left|U\right|$, under the effect of $IND\left(R\right)$. Here, $X_i^R$ is the information granule determined by $R$, abbreviated as a granule; $U/IND\left(R\right)$ is the granular structure; $P_i^R=\left|X_i^R\right|/\left|U\right|$ is the probability distribution of $U/IND\left(R\right)$; and $\left|*\right|$ denotes the number of elements contained in the granule.

For example, if $R=\left\{Colour\right\}$, then we have the granular structure $U/IND\left(R\right)=\left\{\left\{1,2\right\},\left\{3,4\right\},\left\{5,6\right\}\right\}$; $X_R^1=\left\{1,2\right\},X_R^2=\left\{3,4\right\}$, and $X_R^3=\left\{5,6\right\}$ are the information granules determined by $R$; and $P_i^R=\left|X_i^R\right|/\left|U\right|=1/2,i=1,2,3$, from Table 1.

In particular, in a decision information system $K=\left(U,C\cup D\right)$, $X_i^P$ is the conditional granule determined by $P\subseteq C$, and $X_i^Q$ is the decision granule determined by $Q\subseteq D$.

Definition 3 

[32]. In $K=\left(U,AT\right)$, $U/IND\left(R\right)=\left\{X_1^R,X_2^R,\cdots ,X_m^R\right\}$ is the granular structure determined by $R\subseteq AT$, and $GD(R)={\displaystyle \sum _{i=1}^m{\left|X_i^R\right|}^2/{\left|U\right|}^2}$ is called the granularity of $R$.

For example, if $R=\left\{Colour\right\}$, then $GD(R)={\displaystyle \sum _{i=1}^m{\left|X_i^R\right|}^2/{\left|U\right|}^2}=\left(2^2+2^2+2^2\right)/6^2=1/3$, from Table 1.

In granular computing, information systems are also called knowledge bases, and attributes are called knowledge considered as a classification capability. The stronger the classification capability of the knowledge (attributes $R$), the finer the division of the domain, the smaller the granularity $GD(R)$, and the more important the attributes $R$. Particularly, if $U/IND\left(R\right)=\left\{u_1,u_2,\cdots ,u_{\left|U\right|}\right\}$, then $U/IND\left(R\right)$ is the finest division of the domain $U$, and $GD(R)=1/\left|U\right|$. If $U/IND\left(R\right)=U$, then $U/IND\left(R\right)$ is the coarsest division of the domain $U$, and $GD(R)=1$.

2.2. The Granular Matrix and Its Operations

The granular structure and the classification ability of knowledge are intuitively presented in the form of a Boolean matrix in granular computing, which can well solve these problems of knowledge with incomprehensible nature and poorly intuitive operations. The “bit-multiplication” and “bit-sum” operations of the granular matrix are defined, instead of intersection and concatenation operations on the information granules, in this section.

Definition 4.

In $K=\left(U,AT\right)$, $U/IND\left(R\right)=\left\{X_1^R,X_2^R,\cdots ,X_m^R\right\},1\le m\le \left|U\right|$ is the granular structure determined by $R\subseteq AT$. If the mapping $x_{ij}^R=\left\{\begin{array}{cc}1,& u_j\in X_i^R\\ 0,& u_j\notin X_i^R\end{array}\right.,j=1,2,\cdots ,\left|U\right|$ is defined, then $X_i^R$ can be represented by a Boolean row $\left(x_{i1}^R,x_{i2}^R,\cdots ,x_{i\left|U\right|}^R\right)$, i.e., $\overrightarrow{X_i^R}=\left(x_{i1}^R,x_{i2}^R,\cdots ,x_{i\left|U\right|}^R\right)$. $M^R=\left(\overrightarrow{X_1^R};\overrightarrow{X_2^R};\cdots ;\overrightarrow{X_m^R}\right)$is then the granular matrix induced by $R$.

For example, if $R=\left\{Colour\right\}$, then $\overrightarrow{X_1^R}=\left(1,1,0,0,0,0\right)$, $\overrightarrow{X_2^R}=\left(0,0,1,1,0,0\right)$, $\overrightarrow{X_3^R}=\left(0,0,0,0,1,1\right)$, and $M^R=\left(\begin{array}{cccccc}1& 1& 0& 0& 0& 0\\ 0& 0& 1& 1& 0& 0\\ 0& 0& 0& 0& 1& 1\end{array}\right)$, from Table 1.

In particular, if $GD(R)=1$, then $M^R=\left(1,1,\cdots ,1\right)$ is a row vector; if $GD(R)=1/\left|U\right|$, $M^R=I$ is a unit matrix.

In $K=\left(U,AT\right)$, suppose that $M^P=\left(\overrightarrow{X_1^P};\overrightarrow{X_2^P};\cdots ;\overrightarrow{X_m^P}\right)$ and $M^Q=\left(\overrightarrow{X_1^Q};\overrightarrow{X_2^Q};\cdots ;\overrightarrow{X_n^Q}\right)$ are granular matrices induced by $P,Q\left(P,Q\subseteq AT\right)$, respectively, where $\overrightarrow{X_i^P}=\left(x_{i1}^P,x_{i1}^P,\cdots ,x_{i\left|U\right|}^P\right)$ and $\overrightarrow{X_j^Q}=\left(x_{j1}^Q,x_{j1}^Q,\cdots ,x_{j\left|U\right|}^Q\right)$, $1\le i\le m\le \left|U\right|$, $1\le j\le n\le \left|U\right|$.

The “bit-multiplication” and “bit-sum” operations of these granular matrices are defined as follows:

Definition 5.

$M^P\otimes M^Q={\left(\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}\right)}_{mn\times \left|U\right|}$ is called “bit-multiplication” between the granular matrices $M^P$ and $M^Q$, where $\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}=\left(x_{i1}^P\cdot x_{j1}^Q,x_{i2}^P\cdot x_{j2}^Q,\cdots ,x_{i\left|U\right|}^P\cdot x_{j\left|U\right|}^Q\right)$.

In the “bit-multiplication, $\otimes$“operation between $M^P$ and $M^Q$, all elements of each row in $M^P$ are multiplied by the corresponding elements of all rows in $M^Q$. Each row vector $\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}$ in $M^P\otimes M^Q$ is identical to the Boolean row vector of the information granules $X_i^P\cap X_j^Q$. The matrix $M^P\otimes M^Q$ obtained through the “bit-multiplication” operation between $M^P$ and $M^Q$ is identical to the granular matrix $M^{P\cup Q}$ induced by $P\cup Q$, i.e., $M^{P\cup Q}=M^P\otimes M^Q$.

For example, if $P=\left\{Price\right\}$ and $Q=\left\{Size\right\}$, then $\overrightarrow{X_1^P}=\left(1,0,0,0,1,1\right)$, $\overrightarrow{X_2^P}=\left(0,1,1,1,0,0\right)$, $\overrightarrow{X_1^Q}=\left(1,0,0,1,1,0\right)$, $\overrightarrow{X_2^Q}=\left(0,1,1,0,0,1\right)$, $M^P=\left(\begin{array}{cccccc}1& 0& 0& 0& 1& 1\\ 0& 1& 1& 1& 0& 0\end{array}\right)$, and $M^Q=\left(\begin{array}{cccccc}1& 0& 0& 1& 1& 0\\ 0& 1& 1& 0& 0& 1\end{array}\right)$, from Table 1.

Based on Definition 5, $M^P\otimes M^Q=\left(\begin{array}{cccccc}1& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 1\\ 0& 0& 0& 1& 0& 0\\ 0& 1& 1& 0& 0& 0\end{array}\right)$.

Definition 6.

$M^P\oplus M^Q={\left(\overrightarrow{X_i^P}+\overrightarrow{X_j^Q}\right)}_{mn\times \left|U\right|}$ is called the “bit-sum” between the granular matrices $M^P$ and $M^Q$, where $\overrightarrow{X_i^P}+\overrightarrow{X_j^Q}=\left(x_{i1}^P+x_{j1}^Q,x_{i2}^P+x_{j2}^Q,\cdots ,x_{i\left|U\right|}^P+x_{j\left|U\right|}^Q\right)$.

In the “bit-multiplication, $\oplus$“operation between $M^P$ and $M^Q$, all elements of each row in $M^P$ are added to the corresponding elements of all rows in $M^Q$. The row vector $\overrightarrow{X_i^P}+\overrightarrow{X_j^Q}$ is the element of $M^P\oplus M^Q$ in row $\left(i-1\right)n+j$. $\overrightarrow{X_i^P}+\overrightarrow{X_j^Q}-\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}$ is the element of $M^P\oplus M^Q-M^P\otimes M^Q$ in row $\left(i-1\right)n+j$ and is consistent with the row vector of the information granules $X_i^P\cup X_j^Q$.

For example, if $P=\left\{Price\right\}$ and $Q=\left\{Size\right\}$, then based on Definition 6, $M^P\oplus M^Q=\left(\begin{array}{cccccc}2& 0& 0& 1& 2& 1\\ 1& 1& 1& 0& 1& 2\\ 1& 1& 1& 2& 1& 0\\ 0& 2& 2& 1& 0& 1\end{array}\right)$ and $M^P\oplus M^Q-M^P\otimes M^Q=\left(\begin{array}{cccccc}1& 0& 0& 1& 1& 1\\ 1& 1& 1& 0& 1& 1\\ 1& 1& 1& 1& 1& 0\\ 0& 1& 1& 1& 0& 1\end{array}\right)$, from Table 1.

2.3. The Similarity Measure Matrix

In information systems, the division $U/IND\left(R\right)$ determined by $R$ is also called a granular structure. From the perspective of granular computing, neither information entropy nor knowledge granularity is able to measure the differences between granular structures [33]. Equivalence is not implied for one granular structure and another with the same granularity, i.e., granularity cannot portray the differences [34] or similarities between granular structures.

For example, if $P=\left\{Price\right\}$ and $Q=\left\{Size\right\}$, then $GD(P)=GD(Q)=1/2$, but $IND(P)\ne IND(Q)$.

The distance of knowledge was proposed in the literature to measure the difference between granular structures [34,35]. The similarity measure is defined between information granules based on set similarity [35], and the similarity measure matrix for granular structures is constructed in this section.

In $K=\left(U,AT\right)$, suppose that $M^P=\left(\overrightarrow{X_1^P};\overrightarrow{X_2^P};\cdots ;\overrightarrow{X_m^P}\right)$ and $M^Q=\left(\overrightarrow{X_1^Q};\overrightarrow{X_2^Q};\cdots ;\overrightarrow{X_n^Q}\right)$ are granular matrices induced by $P,Q\left(P,Q\subseteq AT\right)$, respectively, where $\overrightarrow{X_i^P}=\left(x_{i1}^P,x_{i1}^P,\cdots ,x_{i\left|U\right|}^P\right)$ and $\overrightarrow{X_j^Q}=\left(x_{j1}^Q,x_{j1}^Q,\cdots ,x_{j\left|U\right|}^Q\right)$, $1\le i\le m\le \left|U\right|$, $1\le j\le n\le \left|U\right|$.

The similarity measure between information granules and the similarity measure matrix for granular structures are defined as follows.

Definition 7.

$s\left(X_i^P,X_j^Q\right)=\left|X_i^P\cap X_j^Q\right|/\left|X_i^P\cup X_j^Q\right|$ is called the similarity between $X_i^P$ and $X_j^Q$. If the row vectors of the information granules $X_i^P$ and $X_j^Q$ are $\overrightarrow{X_i^P}=\left(x_{i1}^P,\cdots ,x_{i\left|U\right|}^P\right),\overrightarrow{X_i^Q}=\left(x_{i1}^Q,\cdots ,x_{i\left|U\right|}^Q\right)$, respectively, then the similarity is 

$$s\left(X_i^P,X_j^Q\right)=\frac{\left|\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}\right|}{\left|\overrightarrow{X_i^P}+\overrightarrow{X_j^Q}-\overrightarrow{X_i^P}\otimes \overrightarrow{X_j^Q}\right|}=\frac{{\displaystyle {\sum }_{k=1}^{\left|U\right|}{x}_{ik}^P\cdot x_{jk}^Q}}{{\displaystyle {\sum }_{k=1}^{\left|U\right|}\left(x_{ik}^P+x_{jk}^Q-x_{ik}^P\cdot x_{jk}^Q\right)}}.$$

If $\overrightarrow{X_i^P}=\overrightarrow{X_j^Q}$, then $s\left(X_i^P,X_j^Q\right)=1$. If $\overrightarrow{X_i^P}\cdot \overrightarrow{X_j^Q}=0$, then $s\left(X_i^P,X_j^Q\right)=0$. In particular, if $\forall X_i^P,X_j^P\subseteq U/IND\left(P\right)$, then $s\left(X_i^P,X_j^P\right)=0,i\ne j$.

Definition 8.

$M^{s\left(P\cdot Q\right)}=\left(\overrightarrow{s_1\left(P\cdot Q\right)};\cdots \overrightarrow{;s_m\left(P\cdot Q\right)}\right)$ is called the similarity measure matrix between $U/IND\left(P\right)$ and $U/IND\left(Q\right)$, where $\overrightarrow{s_i\left(P,Q\right)}=\left(s\left(X_i^P,X_1^Q\right),\cdots ,s\left(X_i^P,X_n^Q\right)\right)$.

For example, if $P=\left\{Price\right\}$ and $Q=\left\{Size\right\}$, then $\overrightarrow{X_1^P}=\left(1,0,0,0,1,1\right)$, $\overrightarrow{X_2^P}=\left(0,1,1,1,0,0\right)$, $\overrightarrow{X_1^Q}=\left(1,0,0,1,1,0\right)$, and $\overrightarrow{X_2^Q}=\left(0,1,1,0,0,1\right)$. Based on Definition 7, $s\left(X_1^P,X_1^Q\right)=1/2$, $s\left(X_1^P,X_2^Q\right)=1/5$, $s\left(X_2^P,X_1^Q\right)=1/5$, and $s\left(X_2^P,X_2^Q\right)=1/2$. Based on Definition 8, $M^{s\left(P\cdot Q\right)}=\left(\begin{array}{cc}1/2& 1/5\\ 1/5& 1/2\end{array}\right)$, from Table 1.

In particular, if $U/IND\left(P\right)=U/IND\left(Q\right)$, then $M^P=M^Q$, and the similarity measure matrix $M^{s\left(P\cdot Q\right)}=I$ is the unit matrix.

The similarity measure matrix portrays the correlation between attribute divisions. The element at each position in the matrix presents the degree of similarity between information granules.

3. A Decision Tree Classification Algorithm Based on Granular Matrices

The decision tree classification algorithm based on granular matrices (GMDT) is a new decision tree construction method. In this algorithm, the classification accuracy of conditional attributes is used to select the classified attributes as the nodes of the tree, and it is calculated by weighting classification actions using the probability distribution determined by conditional attributes.

3.1. Selection Criteria of Classification Attributes

The key technique of a decision tree classification algorithm is how to select the appropriate splitting attribute to determine the nodes of the tree. The classification accuracy of conditional attributes is defined and used to select the splitting attribute.

In $K=\left(U,C\cup D\right)$, suppose that $M^{s\left(P\cdot D\right)}=\left(\overrightarrow{s_1\left(P\cdot D\right)};\cdots \overrightarrow{;s_m\left(P\cdot D\right)}\right)$ is the similarity measure matrix between $U/IND\left(P\right),P\subseteq C$ and $U/IND\left(D\right)$, where $\overrightarrow{s_i\left(P,D\right)}=\left(s\left(X_i^P\cdot X_1^D\right),\cdots ,s\left(X_i^P\cdot X_n^D\right)\right)$.

The classification accuracy of conditional attributes is then defined as follows.

Definition 9.

In $K=\left(U,C\cup D\right)$, if the mapping $s_i\left(P\cdot D\right)=\max \left\{s\left(X_i^P\cdot X_1^D\right),\cdots ,s\left(X_i^P\cdot X_n^D\right)\right\}$ is defined in $\overrightarrow{s_i\left(P\cdot D\right)}$, then $a\left(P\cdot D\right)=\left(s_1\left(P\cdot D\right),s_1\left(P\cdot D\right),\dots ,s_m\left(P\cdot D\right)\right)$ is called the classification action of $P$, and ${\alpha }^{\left(P,D\right)}={\displaystyle \sum _{i=1}^m{P}_i^P\cdot a_i\left(P\cdot D\right)}$ is called the classification accuracy of $P$ relative to the decision attributes $D$.

For example, if $P=\left\{Price\right\}$ and $D=\left\{Buy\right\}$, then $\overrightarrow{X_1^P}=\left(1,0,0,0,1,1\right)$, $\overrightarrow{X_2^P}=\left(0,1,1,1,0,0\right)$, $\overrightarrow{X_1^D}=\left(1,0,1,0,0,1\right)$, and $\overrightarrow{X_2^D}=\left(0,1,0,1,1,0\right)$. Based on Definition 7, $s\left(X_1^P,X_1^D\right)=1/2$, $s\left(X_1^P,X_2^D\right)=1/5$, $s\left(X_2^P,X_1^D\right)=1/5$, $s\left(X_2^P,X_2^D\right)=1/2$, and $P_i^P=\left|X_i^P\right|/\left|U\right|=1/2,i=1,2$. Based on Definition 8, $M^{s\left(P\cdot D\right)}=\left(\begin{array}{cc}1/2& 1/5\\ 1/5& 1/2\end{array}\right)$. Based on Definition 9, the classification action of $P$ is $a\left(P\cdot D\right)=\left(1/2,1/2\right)$, and the classification accuracy of $P$ relative to the decision attributes $D$ is ${\alpha }^{\left(P,D\right)}=1/2$, from Table 1.

The classification accuracy ${\alpha }^{\left(P,D\right)}$ of $P$ relative to $D$ portrays the cognitive ability of $U/IND\left(D\right)$ based on $U/IND\left(P\right)$. In particular, ${\alpha }^{\left(P,D\right)}=1$ shows that the cognition (division) of $U/IND\left(P\right)$ to $U/IND\left(D\right)$ is identical. When the splitting attribute of the decision tree is selected according to ${\alpha }^{\left(P,D\right)}$, a splitting attribute highly consistent with the classification result of the decision attribute can be extracted from the many classification attributes as the node of the tree, and the classification accuracy of the decision tree can thus be improved.

In addition, $GD(P)=1$ indicates that the attribute has no classification ability and cannot be used as a splitting attribute of the decision tree.

3.2. Selection Method of Leaf Nodes

In the decision tree branching process, the dataset is divided into multiple subsets by the splitting attribute, and the branching process is recursively invoked for each subset until each subset contains only the same type of data, at which point the classification ends. In granular computing, the elements contained in each information granule are indistinguishable; that is to say, the data types are consistent. In GMDT, if $s\left(X_i^P,X_j^D\right)=\left|X_i^P\cap X_j^D\right|/\left|X_i^P\cup X_j^D\right|=\left|X_i^P\right|/\left|X_j^D\right|$ between $X_i^P$ and $X_j^D$, then $X_i^P\subseteq X_j^D$. This indicates that the conditional attribute cannot further subdivide the decision granule $X_j^D$, and $X_i^P$ will be made a leaf node of the decision tree.

3.3. Algorithm Description and Analysis

The core idea of GMDT is to select the conditional attribute with the highest classification accuracy as the node of the tree through the operation of the granular matrix; the branching process is recursively invoked for each subset until each subset contains only the same type of data, at which point the classification ends. The specific algorithm flow is shown in Algorithm 1.

Algorithm 1: Decision Tree Algorithm Based on Granular Matrices

Input: K = (Data: the set of training samples, Attributes: the set of conditional and decision attributes) Output: Tree

1: Classification accuracy(k,data,attributes) 2: if k >= len(attributes)    return 3: if Granular Degree(data, conditional attributes, k) == 1    Classification accuracy[k] = 0   else    Granular Matrix (Data, Attributes)    Similar Matrix = Granular Matrix (conditional) ⭙ Granular Matrix(decision)    Classification Action = max (Similar Matrix, axis = 1)    Classification accuracy[k] = Classification Action*Granular Degree    Classification accuracy (k + 1,data,attributes) 4: select Node = max (Classification accuracy) 5: Tree = Tree + Node 6: update (data, attributes), Classification accuracy = {} 7: if data! = null 8:  go to 1 9: else 10:  return tree

3.4. Time and Space Complexity

Time complexity: Assuming that the data-set has n samples and m attributes, in phase 1, the computational effort of the algorithm is mainly to calculate the granular matrix induced by each attribute, which produces a time complexity of O(m × n²). Next, according to the granular matrices induced by the attributes, bit-multiplication and bit-sum operations are performed to calculate the similarity matrix between each information grain and the decision grain in the categorical attributes. Assuming that there are k classification attributes, the time complexity is O(k × n²). Finally, in the classification accuracy function, for each attribute, its classification accuracy must be calculated. Assuming that there are n data samples and m attributes, the time complexity of computing the classification accuracy for each attribute is O(n × m). In the second stage, the amount of calculation is mainly due to recursive calls. If the data-set is not empty, the classification accuracy function and the whole algorithm are called recursively. Assuming that the size of the data-set is p and the recursive depth of the algorithm is d, the time complexity of the recursive calls is O(p × d). Thus, the time complexity of this algorithm is O(m × n²).

Space complexity: The process of recursively calling the classification accuracy function and the whole process of the algorithm involve the use of stack space. Assuming that the recursion depth is d, each recursive call needs to save the local variables and parameters of the function, so the space complexity of this algorithm is O(d).

3.5. Calculation Example

The decision information system $K=\left(U,C\cup D\right)$ is shown in

Table 2. A decision information system, K = (U,C$\cup$D).

U	C						D
	Color	Root	Stroke	Texture	Umbilical	Touch	Good
1	D-green	Curl up	Turbid	Clear	Depressed	Smooth	Yes
2	Jet-black	Curl up	Dull	Clear	Depressed	Smooth	Yes
3	Jet-black	Curl up	Turbid	Clear	Depressed	Smooth	Yes
4	D-green	Curl up	Dull	Clear	Depressed	Smooth	Yes
5	Plain	Curl up	Turbid	Clear	Depressed	Smooth	Yes
6	D-green	Slightly curl	Turbid	Clear	Concave	Soft	Yes
7	Jet-black	Slightly curl	Turbid	S-vague	Concave	Soft	Yes
8	Jet-black	Slightly curl	Turbid	Clear	Concave	Smooth	Yes
9	Jet-black	Slightly curl	Dull	S-vague	Concave	Smooth	No
10	D-green	Stiff	Melodious	Clear	Flat	Soft	No
11	Plain	Stiff	Melodious	Vague	Flat	Smooth	No
12	Plain	Curl up	Turbid	Vague	Flat	Soft	No
13	D-green	Slightly curl	Turbid	S-vague	Depressed	Smooth	No
14	Plain	Slightly curl	Dull	S-vague	Depressed	Smooth	No
15	Jet-black	Slightly curl	Turbid	Clear	Concave	Soft	No
16	Plain	Curl up	Turbid	Vague	Flat	Smooth	No
17	D-green	Curl up	Dull	S-vague	Concave	Smooth	No

.

Based on Definition 7, the similarity measure matrices of $M^c,M^r,M^s,M^{te},M^u,$ and $M^D$ are

$$\begin{gathered} M^{s\left(c,D\right)}=\left(\begin{array}{cc}3/11& 3/12\\ 4/10& 2/13\\ 1/12& 4/10\end{array}\right), M^{s\left(r,D\right)}=\left(\begin{array}{cc}5/11& 3/14\\ 3/12& 4/12\\ 0& 2/9\end{array}\right), M^{s\left(s,D\right)}=\left(\begin{array}{cc}6/12& 4/15\\ 2/11& 3/11\\ 0& 2/9\end{array}\right) \\ {M}^{s\left(te,D\right)}=\left(\begin{array}{cc}7/10& 2/16\\ 1/12& 4/10\\ 0& 3/9\end{array}\right), M^{s\left(u,D\right)}=\left(\begin{array}{cc}5/10& 2/14\\ 3/11& 3/12\\ 0& 4/9\end{array}\right), M^{s\left(to,D\right)}=\left(\begin{array}{cc}6/14& 6/15\\ 2/11& 3/11\end{array}\right) \end{gathered}$$

The classification decision actions are as follows:

$$\begin{gathered} a\left(c,D\right)=\left(\frac{3}{11},\frac{4}{10},\frac{4}{10}\right), a\left(r,D\right)=\left(\frac{5}{11},\frac{4}{12},\frac{2}{9}\right), a\left(s,D\right)=\left(\frac{6}{12},\frac{3}{11},\frac{2}{9}\right) \\ a\left(te,D\right)=\left(\frac{7}{10},\frac{4}{10},\frac{3}{9}\right), a\left(u,D\right)=\left(\frac{5}{10},\frac{3}{11},\frac{4}{9}\right), a\left(to,D\right)=\left(\frac{6}{14},\frac{3}{11}\right) \end{gathered}$$

The classification accuracies of Color, Root, Stroke, Texture, Umbilical, and Touch relative to $D$ are ${\alpha }^{\left(c,D\right)}=0.3551$, ${\alpha }^{\left(r,D\right)}=0.3773$, ${\alpha }^{\left(s,D\right)}=0.4005$, ${\alpha }^{\left(te,D\right)}=0.5471$, ${\alpha }^{\left(u,D\right)}=0.4067$, and ${\alpha }^{\left(to,D\right)}=0.3827$.

Obviously, ${\alpha }^{\left(te,D\right)}=0.5471$ is the highest value, so Texture is selected as the splitting attribute to form a node in the tree to divide the data-set.

Similarly, the classification accuracies of the conditional attributes in each sub-dataset are calculated, and the attribute with the highest accuracy is selected as the division attribute for recursion; the constructed decision tree is shown in Figure 1.

4. Experiments and Result Analysis

4.1. Experimental Environment

The computer hardware and environment configuration were an AMD Ryzen 75800H 3.20 GHz processor with Radeon Graphics, 16.0 GB of RAM, a Windows 10 operating system, and Python version 3.10.

Many existing decision tree models were developed and optimized by combining other theories based on commonly used operators such as the information gain, Taylor’s formula, the Gini coefficient, granular computing, etc. The basis of these decision tree models is these commonly used operators. The decision tree classification algorithm based on granular matrices proposes a new splitting attribute selection operator, i.e., the classification accuracy of the splitting attribute relative to the decision attribute, changing the core of the classification algorithm. Therefore, the classical ID3 and C4.5 were selected for comparison with GMDT in this paper.

In our experiments, more than 20 sets of classification data were selected from UCI through a “classification” keyword search to test the effectiveness of the classification algorithm, including Iris, Heart Disease, Breast Cancer, Adult, Win, CRX, Wine Quality, Car Evaluation, Acute Inflammations, Krkogt, dermatology, etc. In the actual situation where these experimental data were not preprocessed, the classification accuracy of the GMDT, ID3, and C4.5 algorithms on all but the Acute Inflammations, Iris, Breast Cancer, CRX, Mushrooms, and Adult datasets was less than 60%. Therefore, only the experimental results on datasets with classification accuracy higher than 60%, as shown in

Table 3. Description of UCI datasets.

No.	Dataset	Samples	Attributes	Class
1	Acute Inflammations	121	6	2
2	Iris	151	4	3
3	Breast Cancer	286	9	2
4	CRX	654	15	2
5	Mushrooms	8125	21	2
6	Adult	32,561	14	2

, are presented.

4.2. Algorithm Comparison Experiment

(1)

Comparison of classification accuracy

The six experimental datasets in Table 2 were selected and partitioned into 70% for the training set and 30% for the test set. ID3, C4.5, and GMDT were run. The classification accuracies calculated in the experiment are shown in

Table 4. Classification accuracy.

No.	Data	Classification Accuracy
		ID3	C4.5	GMDT
1	Acute Inflammations	0.64	0.64	0.64
2	Iris	0.80	0.87	0.90
3	Breast Cancer	0.46	0.53	0.61
4	CRX	0.31	0.34	0.63
5	Mushrooms	0.80	0.87	0.90
6	Adult	0.27	0.45	0.75

.

In order to intuitively present the experimental results, “blue” was chosen to represent the experimental calculation results of ID3, “red” was chosen to represent the experimental calculation results of C4.5, and “orange” was chosen to represent the experimental calculation results of GMDT. A histogram of the experimental results of the three classification algorithms was drawn, as shown in Figure 2.

The following can be clearly observed: ① The accuracy of the GMDT classification algorithm was higher than that of the classical ID3 and C4.5, especially on the CRX and Adult datasets. ② The classification accuracies of all three algorithms were affected by the data type, the classification accuracies for different data types were different, and the classification accuracies of the three algorithms were basically at the same level in the same data. The maximum variance of the classification accuracy was only 0.0392, as shown in Figure 3, which indicates that the robustness of the GMDT decision tree classification algorithm and that of the classical ID3 and C4.5 classification algorithms are basically consistent.

(2)

Error analysis

To further compare and analyze the classification performance of ID3, C4.5, and GMDT, the numbers of correctly classified, unclassified, and misclassified samples in the test set were extracted from the experimental calculation results and expressed as a ternary ordered array (correct classification, no classification, and classification error), as shown in

Table 5. Error analysis.

No.	Data	Error Analysis
		ID3	C4.5	GMDT
1	Acute Inflammations	(23, 0, 13)	(23, 0, 13)	(23, 0, 13)
2	Iris	(36, 7, 2)	(39, 3, 3)	(41, 3, 1)
3	Breast Cancer	(35, 10, 31)	(40, 6, 30)	(46, 0, 30)
4	CRX	(64, 110, 34)	(70, 101, 37)	(131, 20, 57)
5	Mushrooms	(1950, 340, 148)	(2126, 168, 144)	(2197, 0, 241)
6	Adult	(2679, 5937, 1153)	(4396, 4662, 738)	(7352, 33, 2384)

.

In order to intuitively present the experimental results, “blue” was chosen to represent the experimental calculation results of ID3, “red” was chosen to represent the experimental calculation results of C4.5, and “orange” was chosen to represent the experimental calculation results of GMDT. Histograms of the experimental results from the three classification algorithms in terms of the rate of unclassified data, the rate of classification errors, and the classification potential are plotted in Figure 4, Figure 5, and Figure 6, respectively.

Since the model training of the GMDT classification algorithm requires the classification results of split attributes to be highly consistent with the decision classification, the completed training decision tree classification model showed slight overfitting in the test set. The specific performance was as follows: ① The unclassified proportion of the GMDT classification algorithm was much smaller than those of ID3 and C4.5. As shown in Figure 4, the unclassified ratio of GMDT was 0.096 in the CRX dataset, while the unclassified ratios of ID3 and C4.5 were as high as 0.53 and 0.49, respectively. In the Adult dataset, the unclassified ratio of GMDT was only 0.0034, while the unclassified ratios of ID3 and C4.5 were as high as 0.61 and 0.48, respectively. The difference became more prominent as the sample size of the dataset increased. ② The classification error rate of the GMDT algorithm increased with increasing data volume and data diversity. As shown in Figure 5, the classification error rate of the GMDT classification algorithm was greater than those of ID3 and C4.5 in both the CRX and Adult datasets.

(3)

Comparison of classification potential

The set-pair potential [36] is used to portray the developmental trend of a system by comparing the same degree with the opposition degree. In this paper, the classification accuracy was taken as the same degree, while the error rate was regarded as the opposition degree; the classification potential of ID3, C4.5, and GMDT was thus compared. As shown in Figure 6: the classification potential was basically consistent, even though the classification error rate of GMDT was higher than those of ID3 and C4.5 in the two large-sample datasets, CRX and Adult, which indicates that the slight overfitting did not affect GMDT’s classification performance. GMDT’s classification potential in the Adult dataset was much higher than that of ID3 and C4.5, which indicates that GMDT has better classification performance than ID3 and C4.5.

5. Conclusions

In this paper, a decision tree classification algorithm based on granular matrices was proposed and introduced in detail. Experiments were performed to compare GMDT and the classical ID3 and C4.5; the test results showed that the proposed algorithm has higher classification accuracy and better classification performance than ID3 and C4.5. These studies further optimized the classification performance and extended the classification method. Further research is needed on the operational laws of granular matrices; research following on from this paper will focus on constructing multivariate decision trees and random forests based on similarity granular matrices.