Multispectral Image Segmentation Based on a Fuzzy Clustering Algorithm Combined with Tsallis Entropy and a Gaussian Mixture Model

Abstract

Accurate multispectral image segmentation is essential in remote sensing research. Traditional fuzzy clustering algorithms used to segment multispectral images have several disadvantages, including: (1) they usually only consider the pixels’ grayscale information and ignore the interaction between pixels; and, (2) they are sensitive to noise and outliers. To overcome these constraints, this study proposes a multispectral image segmentation algorithm based on fuzzy clustering combined with the Tsallis entropy and Gaussian mixture model. The algorithm uses the fuzzy Tsallis entropy as regularization item for fuzzy C-means (FCM) and improves dissimilarity measure using the negative logarithm of the Gaussian Mixture Model (GMM). The Hidden Markov Random Field (HMRF) is introduced to define prior probability of neighborhood relationship, which is used as weights of the Gaussian components. The Lagrange multiplier method is used to solve the segmentation model. To evaluate the proposed segmentation algorithm, simulated and real multispectral images were segmented using the proposed algorithm and two other algorithms for comparison (i.e., Tsallis Fuzzy C-means (TFCM), Kullback–Leibler Gaussian Fuzzy C-means (KLG-FCM)). The study found that the modified algorithm can accelerate the convergence speed, reduce the effect of noise and outliers, and accurately segment simulated images with small gray level differences with an overall accuracy of more than 98.2%. Therefore, the algorithm can be used as a feasible and effective alternative in multispectral image segmentation, particularly for those with small color differences.

1. Introduction

Image segmentation, which is a remote sensing technique dividing complex images into several continuous homogeneous regions, is fundamental in object-based image analysis [1,2,3,4]. There are three main approaches to implement segmentation, namely: threshold-based [5,6], clustering-based [7], and statistics-based [8,9] algorithms. Among them, the clustering segmentation method [10] is widely used in remote sensing applications. However, along with the improvements in remote sensing sensors, multispectral image segmentation requires more complex segmentation algorithms due to the large number of spectral bands and detail feature information [11,12,13]. Thus, research exploring new techniques in image segmentation is of great significance.

The clustering method based on the fuzzy set theory is an effective tool to deal with the uncertainty of image information, which is easy to operate and can guarantee segmentation accuracy [14]. Previous studies have proposed different algorithms for implementing the clustering-based approach. The fuzzy set has been conceived by Zadeh [15], which can exhibit the uncertainty that an individual point is similarity to all the clusters. Bezdek [16] has proposed the fuzzy clustering method based on least-squared error criterion, called the Fuzzy C-means (FCM) algorithm. In this method, the weighting exponent m has been introduced to control the extent of membership sharing between fuzzy clusters [17]. Dunn [18] has improved the FCM algorithm using the Euclidean distance to quantify the Dissimilarity Measure (DSM). The algorithm has used the fuzzy membership of pixel to clusters as weight to construct the objective function in image segmentation. The optimal segmentation of images can be obtained through minimization of the objective function and derivation of the iterative algorithms for computing the membership functions of clusters in question. Although the FCM proposed by Bezdek has become ubiquitous in image segmentation, the approach continues to have major significant disadvantages, including: having no theoretical basis on weighting exponent m selection, neglecting image pixel grayscale correlation, sensitivity to noise and outliers, and inability to fit multi-peak characteristics of a remote sensing image. In order to address these constraints, the FCM algorithm is improved mainly through two main aspects: objective function and dissimilarity measure.

With regards to the objective function, in order to overcome the lack of theoretical basis of the weighting exponent m, several improved fuzzy clustering algorithms based on the entropy theory have been proposed. For example, the Shannon entropy has been introduced into the FCM algorithm by Li et al. [19], leading to a new fuzzy clustering algorithm, wherein the minimization of the objective function is achieved using the Maximum Entropy Inference (MEI) within the context of FCM. Compared with the other FCM algorithms, this method has clear mathematical features and physical significance, which has piqued the interest of a large number of researchers. Miyamoto et al. [20] and Tran et al. [21] has introduced Shannon Entropy into the objective function of FCM using an arithmetic weighted system and proposed the Entropy FCM (EFCM) algorithm. Li K. et al. [22] has proposed the Generalized Entropy FCM (GEFCM) algorithm based on the Generalized Entropy, utilizing the adjustment of two parameters that would achieve better segmentation results, which has resulted in challenges in finding the optimal solution. Liao S. et al. [23] has introduced the fuzzy entropy that redefined the objective function. In this algorithm, the membership function is affected not only by distance measure but also by the fuzzy entropy, which reduces the adverse effects of noise and outliers in the cluster centers and drives the clustering results to become more objective.

All the aforementioned algorithms make the Shannon entropy as the entropy regularization term in the objective function to achieve the optimal effect of the FCM. However, the usage of the Shannon entropy is significantly limited, given its inability to consider the influence of cluster size and cluster correlation on segmentation results [24]. When the cluster scale is largely different (clusters with large pixel values coexisting with other clusters with small number of pixels), the algorithm can easily segment the edge pixels of the larger-scale cluster into the smaller-scale cluster. Afterward, the Tsallis entropy has been proposed to measure the nonextensivity of the physical system, which uses an exponential weighted form of entropy to describe the interrelation between subsystems [25,26,27]. Over the years, the Tsallis entropy has become widely used in image segmentation due to its universality. As the generalized form of the Shannon entropy, the Tsallis entropy could describe the correlation described by the nonextensive parameter q between the grayscale value of pixels in images. Image segmentation methods based on the Tsallis entropy considered the interaction between clusters, which is favorable in segmenting images with nonextensive features [28,29,30,31,32]. Among them, a new multilevel thresholding technique based on the Tsallis entropy for colored satellite image has been proposed, achieving segmentation of complex images with small color differences and having the ability to describe the interrelationship between clusters [28]. Kaur et al. have proposed a hybrid multilevel thresholding technique that combines intuitionistic fuzzy sets and Tsallis entropy for the automatic delineation of tumors in magnetic resonance images with vague boundaries and poor contrast [29]. However, both techniques that applied the fuzzy Tsallis entropy to threshold segmentation still suffer a major limitation, as they can only segment one-dimensional images. In order to describe the long-range interaction between grayscale values of color image, Yasuda [33] has maximized the Tsallis entropy within the FCM framework and made a quantitative description of the relationship between the nonextensive parameter (q) and temperature through the Deterministic Annealing (DA) method. Although this method explains the tangible implications of maximizing Tsallis entropy from a mathematical perspective in detail, it undermines the role of FCM and makes the solution process more complex.

Aside from refining the objective function, a number of researchers have examined the dissimilarity measure based on the distance function and probability distribution model to improve further the accuracy of the fuzzy clustering algorithm. Most traditional fuzzy clustering algorithms used the cluster center to represent the cluster, and the dissimilarity measure has usually been defined by the Euclidean distance between the pixel and the cluster center [16,17]. However, the dissimilarity measure based on the Euclidean distance fails to accurately distinguish the clusters with non-spherical distribution and cannot effectively segment detail information. The traditional fuzzy clustering segmentation algorithm based on the Euclidean measure is sensitive to noise and outliers, and the segmentation results are easily affected by the shape, density, and size of clusters (i.e., pixel distribution in feature space and the number of pixels in homogeneous regions). In addition, the algorithm has an extreme dependence on the initial cluster center, and thus cannot effectively be used for multispectral remote sensing image segmentation. Other scholars have used the Mahalanobis distance to consider the influence of clustering variance on dissimilarity measure, which can more effectively distinguish clusters with varying shapes and scales [34,35]. However, it is still expressed using the cluster center, which can reduce the resulting accuracy.

To better describe the cluster, some researchers have used probability distribution to define the dissimilarity measure of the fuzzy clustering algorithm [36,37,38], which can compensate, to some extent, for the shortcomings of distance dissimilarity measure. The factor probability distribution clustering (FPDC) method uses the Tucker3 decomposition to perform linear transformation and assigns pixels to clusters based on the probability that they belong to a given cluster [36]. This probability distribution clustering performs well when the clusters are elliptically shaped, but it becomes unstable when the number of variables is large, and the variables are correlated. Chen et al. have proposed a probabilistic distance function based on kernel density estimation for categorical attributes [37].

With gradual improvements in their spatial resolution, recent remote sensing images generally have multipeak characteristics, and their distribution characteristics become more complex. In order to describe the multi-peak distribution characteristics more accurately, the Mixture Model (MM) has been developed to create an image segmentation model [39,40]. For example, agglomerative clustering, based on combinations of non-Gaussian distributions, has been developed for statistical-based image segmentation [41]. This method adopts higher-order statistics in pixel distributions instead of second-order statistics as correlation. It appends non-Gaussian mixture models estimated for every clustering building with a hierarchical structure by using Kullback–Leibler divergence. Although agglomerative clustering performs well in segmenting regions of images in similar areas, varying levels of dependence of the variables increase the complexity of the algorithm. Because of its simple structure and the fact that most remote sensing images have Gaussian characteristics, the Gaussian Mixture Model (GMM), which is a mathematical technique fitting complex distribution through a set of simple density distribution weighted combination, have been proposed to fit the multipeak distribution feature data of remote sensing images [42,43].

Traditional FCM is sensitive to noise and outliers, and do not consider the gray information of image pixels. Also, the results are easily influenced by shape, density, and size of the clusters. To address these limitations, we developed the fuzzy clustering algorithm combined with the Tsallis entropy and the Gaussian mixture model (TGMM-FCM) as an alternative approach to image segmentation. The fuzzy Tsallis entropy is introduced into FCM as a regular term to define an improved objective function. Compared with the traditional objective function, this objective function not only considers the image pixel gray information but also considers the pixel distribution characteristics (i.e., the correlation degree between gray levels). The GMM is then used to represent clusters that can accurately fit the multi-peak features of remote sensing images. The labeling field is established with the Hidden Markov Random Field (HMRF) [44,45], and is used to characterize the prior probability of the neighborhood spatial relationship to effectively eliminate noise effects. The spatial function between neighborhood pixels based on the labeling field is designated as the prior probability of image pixels, and the prior probability is taken as the weight of GMM components to be used in constructing GMM dissimilarity measure.

The remainder of this paper is organized as follows: in Section 2, the proposed methodology is succinctly discussed. This section also describes image segmentation evaluation methods and optimization techniques employed in this study and quantitatively measures the effective value range of the nonextensive parameter q. In Section 3, the results from the segmentation experiments of simulation and multispectral images are presented. The qualitative and quantitative evaluation of accuracy is discussed, which is then related to the effectivity analysis of the TGMM-FCM algorithm. Finally, Section 4 presents the study’s conclusions.

2. Methodology

2.1. Overview

This paper proposes the Fuzzy Tsallis Entropy-Clustering Algorithm Combined with Gaussian Mixture Model (TGMM-FCM). The framework of the proposed algorithm is presented in Figure 1. The key concepts include the adoption of the fuzzy Tsallis entropy to describe the correlation between pixel grayscale and the use of the Gaussian Mixture Model (GMM) for fitting the multi-peak features. This method is employed to develop an efficient procedure for multispectral remote sensing image segmentation, which would be able to overcome the shortcomings of traditional FCM and improve the segmentation accuracy of multispectral images.

2.2. Fuzzy Tsallis Entropy-Clustering Segmentation Model

Let X = {x₁, x₂, …, x_i, …, x_N} be the remote sensing images of p dimensions, where x_i = (x_i1, x_i2, …, x_ip)^T is the spectral measure vector of i^th pixel, and N is the number of image pixels. The images can be divided into c homogeneous regions (i.e., clusters) by a fuzzy clustering algorithm. Let {X_j: 1 ≤ j ≤ c} ⊂ F(X), such that F(X) represents the division of image X. Let U = [u_ij]_N×c be the fuzzy membership matrix to characterize the fuzzy segmentation of image, where the fuzzy membership function u_ij is used to describe the membership of the pixel i to the clustering j. In order to determine the optimal U, we need to perform the following: (1) define the dissimilarity measure of pixels and clustering; (2) structure the objective function by this measure; and, (3) obtain the optimal fuzzy segmentation result by minimizing the objective function.

The Shannon entropy satisfies the additivity criterion, assuming the systems or pixels are independent of each other. However, the idealized model does not perform adequately with complex images since it cannot describe the interaction between pixels of an image. Tsallis [25,26,27] has introduced the nonextensive parameter q based on the Shannon entropy and proposed a type of nonextensive entropy, which works well in describing the thermodynamic properties of nonextensive physical systems:

$$S_q=-\frac{{\displaystyle \sum _{j=1}^c{\left(P_j\right)}^q}-1}{q-1}$$

(1)

where, q {q > 0 | q ≠ 1} is an index used to describe the nonextensive extent. When q approaches 1, the Tsallis entropy and Shannon entropy become equivalent. When P_j approaches 1/c, the Tsallis entropy and Reny entropy become equal. When 0 < q < 1 and P_j^q > P_j, the small probability events are increased, while the big probability events are suppressed. When q > 1 and P_j^q < P_j, the small probability events are suppressed, while the big probability events are intensified. The best value range of q is [0.8, 1.5] [46].

As article [47] said, the Tsallis entropy, a kind of generalized entropy, can both sufficiently characterize the cluster-to-cluster interactions and be used for the quantification of nonextensive images. According to the definition of the fuzzy entropy [31,48], the Tsallis entropy can accurately be defined as the expected value of fuzzy membership [33], which is called the Fuzzy Tsallis Entropy Function E(U), to present the average fuzzy extent of images:

$$E(\mathit{U})=-\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q}}-1}{q-1}$$

(2)

The fuzzy Tsallis entropy function has been proven to satisfy the following conditions: (1) When u_ij = 1/c, the E(U) will reach the maximum value, and decreases as u_ij moves closer to 0 or 1. (2) As for the independent fuzzy sets, E(U) satisfies the pseudo-additive criterion. The fuzzy Tsallis entropy can describe the average uncertainty (ambiguity) of fuzzy sets, which is often used as an indicator of cluster validity. The smaller the value of E(U), the smaller the average uncertainty of the fuzzy set is, which produces clearer segmentation results.

The TGMM-FCM is an improved algorithm based on the Fuzzy Entropy-Clustering (FEC). The objective function of FEC is defined as the difference between the objective function of the Hard C-means (HCM) and the fuzzy Shannon entropy multiplied by a fuzzy factor [21]. The minimization process of the objective function can simultaneously satisfy the requirements of the minimum distance and the maximum fuzzy entropy function, in order to obtain the optimal division. In this study, we define the objective function of the new model as the difference between the FCM’s objective function and the fuzzy Tsallis entropy:

$$\begin{array}{ll}{J}_{mq}& ={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^m{d}_{ij}^{}}}-E\left(\mathit{U}\right)\\ & ={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^m{d}_{ij}^{}}}+\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q}}-1}{q-1}\end{array}$$

(3)

where i is the pixel index, j is the clustering index, N is the total number of image pixels, c is the number of clusters, d_ij is the dissimilarity measure from i^th pixel to j^th center of clustering, and u_ij is the membership of the pixel i to the clustering j. The E(U), as the fuzzy Tsallis entropy regularization term, can improve the smoothing of the improved fuzzy algorithm [33]. It can also describe the interaction of the pixels’ gray values to better segment images with only small grayscale differences.

2.3. GMM Dissimilarity Measure

Based on their statistical attributes, most remote sensing data can be characterized by a Gaussian distribution. Thus, in this study, the dissimilarity measure of pixels to clusters is established with the use of the GMM. The mean vector and covariance matrix can be calculated using the GMM dissimilarity measure, and the statistical distribution can be fitted with high precision. Suppose that pixels belonging to the same homogeneous region are subject to the same Gaussian probability distribution, i.e., P(x_i | v_j, Σ_j), determined by the mean v_j and covariance Σ_j of clusters. In the image segmentation, every pixel can belong to each cluster due to the fuzziness. In this paper, the prior probability from the HMRF is used as weight of the mixture model, and the Gaussian probability distribution is used as the component function of GMM. The formula of the Gaussian Mixture Model P(x_i | w_i, v, Σ) is:

$$P\left(x_i|w_i,v,\Sigma \right)={\displaystyle \sum _{j=1}^c{w}_{ij}p\left(x_i|v_j,{\Sigma }_j\right)}$$

(4)

where,

$$p\left(x_i|v_j,{\Sigma }_j\right)=\frac{1}{{\left(2\pi \right)}^{\frac{p}{2}}{\left|{\Sigma }_j\right|}^{\frac{1}{2}}}\exp \left(-\frac{1}{2}{\left(x_i-v_j\right)}^{\mathrm{T}}{\Sigma }_j{}^{-1}\left(x_i-v_j\right)\right)$$

(5)

For computational convenience, the dissimilarity measure d_ij can be defined as the negative logarithm of the j^th, component in the GMM:

$$d_{ij}\triangleq -\ln w_{ij}p\left(x_i|v_j,{\Sigma }_j\right)$$

(6)

Combining Equation (5) and Equation (6), the dissimilarity measure d_ij of the i^th pixel to j^th cluster can be written as:

$$d_{ij}=\frac{p}{2}\ln \left(2\pi \right)+\frac{1}{2}\ln \left[\det \left({\mathit{\Sigma }}_j\right)\right]+\frac{1}{2}\left[{\left({\mathit{x}}_i-{\mathit{v}}_j\right)}^T{\mathit{\Sigma }}_j^{-1}\left({\mathit{x}}_i-{\mathit{v}}_j\right)\right]-\ln w_{ij}$$

(7)

where, w_i = {w_ij; j = 1, 2, …, c} is the prior probability vector, such that each element w_ij is the prior probability of i^th pixel to j^th cluster; v = {v_j; j = 1, 2, …, c} is the mean cluster vector set, such that each element v_j is the mean vector of j^th cluster; Σ = {Σ_j; j = 1, 2, …, c} is a set of covariance matrix, such that each element Σ_j is the covariance matrix of cluster j; and, p is the number of bands of remote sensing images. The dissimilarity measure expressed in Equation (7) can reconcile the contributions of the different bands to the segmentation results, and can effectively use the spectrum information of the different bands to improve the segmentation accuracy of remote sensing images.

2.4. Label Field Model

In order to introduce the special relationship of pixels into the dissimilarity measure, w = {w_ij; i = 1, 2, …, c, j = 1, 2, …, n} is defined by the Hidden Markov Random Field (HMRF) function that can characterize the neighborhood relation. The prior probability w_ij presents the role of the neighborhood pixels with the same label as the central pixel. In general, compared with the non-neighborhood pixels, the neighborhood pixels are more likely to have the same label. Thus, the introduction of the prior probability based on neighborhood relationship can effectively avoid segmentation errors from noise.

In this study, the label field is modeled by the HMRF. Assume that L = (l₁, l₂, …, l_N) is the label field of a given image X, where l_i∈{1, 2, …, c} is the label of the i^th pixel. The maximum membership criterion is adopted to realize defuzzification, and pixel label is determined:

$$l_i=argmax\left\{u_{ij};j=1,2,\dots ,c\right\}$$

(8)

where l_i represents the cluster where pixel i belongs (implementation of the label of pixel i). This means that the label of the cluster corresponding to the maximum fuzzy membership of the pixel i is assigned to the label field.

In the label field, the potential energy function V_c is established as follows:

$$V_c\left(l_{i\prime },l_i\right)=\{\begin{array}{l}0,l_{i\prime }=l_i\\ 1,l_{i\prime }\ne l_i\end{array}$$

(9)

where i′ is the neighborhood pixel index of pixel i, and l_i′ is the label of the neighborhood pixel index. When the neighborhood pixel has the same label as the central pixel, it reaches a steady-state with a potential energy of 0; otherwise, the potential energy is 1.

In constructing the prior probability, a 3 × 3 pixel-square neighborhood window around pixel i is referred to as the eight-neighborhood pixel set, as shown in Figure 2a. As example, Figure 2b presents a label field implementation of pixel i’s neighborhood window. In this label field, the neighborhood potential energy distributions of the central pixel with different labels are shown in Figure 2c–e. When the label of the central pixel is 1, the potential energy corresponding to its neighborhood is lowest.

The prior probability is defined as the e index of the sum of the negative potential energy functions. In order to increase the corresponding prior probability of low potential energy state, the probability is normalized to satisfy the constraint condition ${\displaystyle {\sum }_{j=1}^c{w}_{ij}}=1$. Then, the prior probability of the pixel i belonging to cluster j is estimated by:

$$w_{ij}\triangleq \frac{\exp \left[-b{\displaystyle \sum _{i\prime \in N_i}{V}_c\left(l_{i\prime },l_i\right)}\right]}{{\displaystyle \sum _{k=1}^c\exp \left[-b{\displaystyle \sum _{i\prime \in N_i}{V}_c\left(l_{i\prime },l_k\right)}\right]}}$$

(10)

where b is a parameter representing the label interaction intensity of the neighborhood, with values ranging from any real number between [0, 1]. The larger the b value, the more the central pixel is affected by its neighborhood, which is suitable for images with more noise. Smaller b values indicate smaller neighborhood effects, which is suitable for images with more detailed information or more linear features.

2.5. Solution of Segmentation Model

In order to obtain the optimal fuzzy segmentation, the objective function is minimized by calculating the partial derivatives of each parameter and making the partial derivatives to be zero. Since the constraint of u_ij is ${\displaystyle {\sum }_{j=1}^c{u}_{ij}}=1$, the Lagrange multiplier β_i is introduced into the objective function of Equation (3) to construct the Lagrange equation:

$$L_{mq}\triangleq {\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^m}}{d}_{ij}+\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q}}-1}{q-1}+{\displaystyle \sum _{i=1}^N{\beta }_i\left({\displaystyle \sum _{j=1}^c{u}_{ij}-1}\right)}$$

(11)

The partial derivative of Equation (11) is determined and taken to be equal to zero, such that:

$$\frac{\partial L_{mq}}{\partial u_{ij}}=m{u}_{ij}^{m-1}{d}_{ij}+\frac{q}{q-1}{u}_{ij}^{q-1}+{\beta }_i=0$$

(12)

where, m is the fuzzy factor to describe the fuzzy extent of the pixel i belonging to cluster j, and q is the nonextensive parameter to represent the nonextensive degree of image. A large m-value indicates a large fuzzy extent, while an m-value close to one suggests a small fuzzy extent. According to Equation (12), the condition where m > q and m < q is similar to the mathematical phenomenon of large number + decimal, such that the influence of the decimal part in Equation (11) becomes negligible. Therefore, only when m = q can the exponentially weighted measure term and the fuzzy Tsallis entropy term in Equation (11) both impact the objective function simultaneously. In the image segmentation application, the fuzzy factor and the nonextensive parameter in Equation (3) can be uniformly represented by q, and the Equation (3) can be rewritten as:

$$J_q={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q{d}_{ij}^{}}}+\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q}}-1}{q-1}$$

(13)

When the GMM dissimilarity measure is introduced into the Equation (13), expression becomes:

$$J_{qGMM}\triangleq {\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q\left\{\frac{p}{2}\ln \left(2\pi \right)+\frac{1}{2}\ln \left[\det \left({\mathit{\Sigma }}_j\right)\right]+\frac{1}{2}\left[{\left({\mathit{x}}_i-{\mathit{v}}_j\right)}^{\mathrm{T}}{\mathit{\Sigma }}_j^{-1}\left({\mathit{x}}_i-{\mathit{v}}_j\right)\right]-\ln w_{ij}\right\}}}+\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^c{u}_{ij}^q}}-1}{q-1}$$

(14)

where, q has a dual physical meaning, which can represent the degree of ambiguity in the algorithm and can also describe the nonextensive parameters in statistics. The value of q directly controls the quality of the segmentation results, so we can select it according to the statistical characteristics of the images (the variance and mean of the clusters). The value of q is selected from the range of (0, 1) to segment images with small noise and highly detailed information, which has the effect of enlarging small differences. The value of q is selected from the range of (1, ∞) to segment images with more noise and continuous regions, which can effectively remove spot noise. The fuzzy Tsallis entropy provides the statistical characteristics of the images, which can be used to describe the average fuzziness of the image and the correlation between the gray levels of the pixels.

According to the Lagrange multiplier method, the membership degree u_ij, the cluster center v_j and the covariance matrix Σ_j are expressed as follows:

$$u_{ij}=\frac{{\left[\left(q-1\right)d_{ij}+1\right]}^{\frac{-1}{q-1}}}{{\displaystyle \sum _{j\prime =1}^c{\left[\left(q-1\right)d_{ij\prime }+1\right]}^{\frac{-1}{q-1}}}}$$

(15)

$$v_j^{}=\frac{{\displaystyle \sum _{i=1}^N{u}_{ij}^q{x}_i}}{{\displaystyle \sum _{i=1}^N{u}_{ij}^q}}$$

(16)

$${\Sigma }_j=\frac{{\displaystyle \sum _{i=1}^N{u}_{ij}^q\left(x_i-v_j^{}\right){\left(x_i-v_j^{}\right)}^{\mathrm{T}}}}{{\displaystyle \sum _{i=1}^N{u}_{ij}^q}}$$

(17)

In summary, the specific steps of implementing the proposed algorithm are as follows:

Step 1: Set the constants, including: the number of cluster c, nonextensive parameter q, neighborhood interaction intensity b, terminal condition Ɛ, and the total iterations. Also, set the initial U⁽⁰⁾.

Step 2: Solve the label field according to Equation (8).

Step 3: Compute the cluster center v_j^(z) of j^th cluster using Equation (16).

Step 4: Solve the covariance matrix Σ_j^(z) of j^th cluster according to Equation (17).

Step 5: Compute the prior probability w_ij^(z) combined with the label field using Equation (9) and Equation (10).

Step 6: Substitute w_ij^(z), v_j^(z), and Σ_j^(z) into the Equation (7) to calculate the dissimilarity measure d_ij^(z).

Step 7: Update the membership degree u_ij^(z+1) of the i^th pixel belonging to the j^th cluster by using Equation (15).

Step 8: Evaluate the condition whether ||U^(z+1)-U^(z)|| < Ɛ. Upon confirmation that the mathematical statement is true, the iteration process is stopped, and the defuzzification is performed based on the final membership matrix. Otherwise, the second step is iterated.

2.6. Quantitative Analysis of q

In order to quantitatively analyze the effective value range of the nonextensive parameter q in the TGMM-FCM algorithm, we use the Fuzzy Tsallis Entropy-Clustering Algorithm (T-FCM) with the Euclidean distance and Fuzzy Entropy-Clustering (FEC) algorithm [21] to segment the test image with 128 × 128 sized pixels, as shown in Figure 3. In

Table 1. Comparison of clustering results with different parameters of each algorithm.

FEC			T-FCM
m	t*	Jm	q	t*	Jq
0.8	28.94	204,850.4770	0.8	42.60	2,906,106.3785
0.9	56.78	204,835.4503	0.9	53.81	2,588,948.8127
0.95	28.45	204,826.6858	0.95	54.36	2,432,005.7475
1.05	28.44	204,808.9289	1.05	20.23	602,248.8115
1.1	56.73	204,801.1249	1.1	26.57	367,476.4695
1.2	47.54	203,781.8025	1.2	9.93	282,691.2020
1.3	24.81	204,761.0106	1.3	10.48	250,463.9400
1.4	43.78	203,738.7498	1.4	21.85	229,831.9548
1.5	37.76	204,714.9567	1.5	19.83	212,864.9024
1.75	29.06	203,648.7930	1.75	21.30	174,910.9655
2	28.15	204,574.0016	2	17.46	140,534.3012
2.25	43.23	204,483.2611	2.25	21.87	110,561.9344
2.5	41.51	274,732.6831	2.5	4.12	144,011.5837
2.75	41.22	204,278.3709	2.75	5.38	101,697.2255

* Time (t) is in seconds.

, the records of the objective function value (J_q) and convergence time (t) of TFCM under different parameters (q) are tabulated, as well as the objective function value (J_m) and convergence time (t) of FEC under different fuzzy factors (m).

When evaluating Table 1 and Figure 4a, the FEC is seen to be unaffected by changes in m when m < 1, suggesting that the algorithm fails at this condition. Theoretically, this is consistent with the fact that the fuzzy factor should be greater than one. When m > 1, the objective function value produces small fluctuations with changes in the fuzzy factor, which means that the FEC algorithm is stable, and the value of the fuzzy factor has little effect on segmentation results. In Figure 4a, as the blue line indicates, the objective function value of T-FCM undergoes an abrupt change when the q is equal to 1.1 and then falls steadily with increasing q at the range of [1.1, 2.25]. Past the q value of 2.25, the T-FCM experiences large fluctuations with small changes in q. Thus, the effective range of q for image segmentation using T-FCM is [1.1, 2.25], and the selection of nonextensive parameters (q) can significantly influence the segmentation results.

From Figure 4b, the convergence time t required for segmentation with T-FCM is clearly smaller than the alternative algorithm. Because the Tsallis entropy is in the exponential form, the change is more pronounced, while the Shannon entropy is in the logarithmic form, such that the change is more gradual. Thus, the fuzzy Tsallis entropy has the advantage of high efficiency compared with the fuzzy Shannon entropy, which can reduce the temporal redundancy to some extent.

3. Experimental Result and Discussion

In this study, the operating environment was a personal computer with 4G memory and an intel(R) i5 3rd generation processor. MATLAB programming was used in performing the segmentation experiments of multispectral images. To examine the feasibility and effectiveness of the algorithm, two other algorithms were used for comparison: the T-FCM and the Kullback–Leibler Gaussian Fuzzy C-means (KLG-FCM). The KLG-FCM adopts the negative logarithm of the Gaussian Distribution Function as the dissimilarity measure and uses the Kullback–Leibler (KL) divergence information as the regularization term of the objective function, achieving the fuzzification of objective function and representing the neighboring relationship with HMRF [49,50]. In the experiment, the different algorithms were used to segment the simulation images and multispectral images. Qualitative and quantitative evaluation of segmentation accuracy was employed to analyze the effectiveness of the TGMM-FCM algorithm.

3.1. Simulation Image Segmentation

Figure 5b is a simulation image generated according to a template image (shown in Figure 5a) at 128 × 128 pixel size. The template image has five unequal-sized homogeneous regions of irregular shapes, with the hexagonal shape (region (3)) accounting for the largest area. Figure 5b displays different scenes taken from IKONOS at a four-meter spatial resolution. As shown in

Table 2. Mean and variance of simulation image.

Parameter		Homogeneous Region
		①	②	③	④	⑤
Mean	Red band	146.34	58.45	13.49	129.83	30.12
	Green band	177.05	125.55	29.71	130.72	122.33
	Blue band	72.63	110.22	28.92	101.54	26.79
Variance	Red band	378.46	321.87	122.19	322.16	157.95
	Green band	211.74	314.99	282.61	132.02	47.48
	Blue band	126.88	452.49	155.63	170.79	121.66

, some regions of the simulation image have small differences in mean, but all the homogeneous regions have significant variances.

Figure 6 shows the segmentation results and contour overlays obtained by segmenting the simulation image using T-FCM, KLG-FCM, and the proposed algorithm. Figure 6(b1) shows that the T-FCM algorithm is able to execute the segmentation for regions (1) and (3) with similar cluster mean values, while the segmentation for regions (2) and (4) is not ideal where the texture features were more complicated. In Figure 6(b2), because the KLG-FCM algorithm introduced neighborhoods, the internal regions are smooth and continuous. However, since the Gaussian distribution, as the dissimilarity measure, can only solve the characteristics independently, the boundaries became blurred, and the segmentation effect is not ideal. In Figure 6(a3), the internal regions are smooth and continuous without error pixels, and the outlines are clear and linear with few jagged lines. The GMM dissimilarity measure in this algorithm appears to accurately fit the statistical features of each region in the image and has satisfactory performance in segmentation of images with similar means of the interregion and significant variances of the intraregion.

To explain how the TGMM-FCM algorithm compares with other algorithms in terms of how it can accurately fit the data distribution characteristics, we analyzed the segmentation results of each algorithm from a microscopic perspective. The 3D distribution of segmentation results using the T-FCM (shown in Figure 7b) indicates that the points belonging to the region and distant from the cluster center are easily segmented by mistake. The 3D distribution of segmentation results by KLG-FCM is shown in Figure 7c. In Figure 7c, the Gaussian distribution dissimilarity measure could completely match the statistical properties of regions (2), (4) and (5); but since the correlation between regions is neglected, the pixels belonging to region (3) are segmented into region (1). In Figure 7d, the TGMM-FCM algorithm assigns the GMM with weights characterizing the neighborhood relationship to each pixel. The color space distribution of each region could be restored in the original image by accurately fitting the statistical features of each region and allocating remote discrete points into the correct clusters. Based on the experimental results, the proposed algorithm is able to successfully segment the complex images accurately.

In order to verify the robustness and anti-noise of the TGMM-FCM algorithm, another set of simulation images, in which the spectral measure of each pixel follows a consolidated Gaussian distribution, is constructed following the method as proposed by Bioucas-Dias et al. [51]. Figure 8b is the simulation image generated according to the template image (shown in Figure 8a) at 256 × 256 pixel size. The template image has five homogeneous regions with regular shapes. The pixels of homogeneous region follow the same Gaussian mixture distribution as shown in

Table 3. Mean and variance of the simulation image.

Parameter		Homogeneous Region
		①	②	③	④	⑤
Mean	Red band	50	165	170	30	224
	Green band	96	191	170	44	32
	Blue band	80	70	170	26	32
Variance	Red band	10	160	10	60	10
	Green band	100	138	10	102	30
	Blue band	50	101	10	33	100

. As shown in Figure 8c,d, some Gaussian noise is added into the initial simulation image to form new simulation images with different signal-to-noise ratio (SNR).

The T-FCM, KLG-FCM, and the proposed algorithms were used in this section to segment the simulation images with different SNR, to verify the anti-noise performance of the TGMM-FCM proposed in this study. The segmentation results and contour overlays are shown in Figure 9. In Figure 9(a1–a4), the quality of the segmentation results obtained by the T-FCM is inferior, because the Euclidean distance cannot adequately recognize probability distributions of interregions. In Figure 9(b1–b4), the segmentation results of the KLG-FCM are better than the T-FCM due to the neighborhood and the Gaussian dissimilarity measure. However, many pixels have been erroneously grouped in regions (2) and (5), with large variance of intraregion and small difference in interregion means. Noise content has significant influence on the segmentation result in the T-FCM and KLG-FCM. As shown in Figure 9(c1–c4), the internal regions are smooth and continuous without error pixels, and the outlines are clear-cut and smooth with few jagged lines. And the segmentation results of TGMM-FCM are scantily influenced by noise.

In order to quantitatively evaluate the accuracy of the segmentation results, the template images in Figure 5a and Figure 8a were used as the criterion to calculate the confusion matrix generated by the segmentation results (shown in Figure 6 and Figure 9). The product accuracy, user accuracy, overall accuracy, and Kappa value were calculated according to the confusion matrix, as shown in

Table 4. Segmentation results’ accuracy of simulation images.

Algorithms	Accuracy	Homogeneous Region (Figure 5b/Figure 8c/Figure 8d)
		①	②	③	④	⑤
T-FCM	Product (%)	97.60/99.64/89.55	92.42/98.95/89.06	98.77/100.00/84.61	96.57/99.45/89.03	95.54/59.30/45.55
	User (%)	98.82/95.64/85.02	96.64/92.21/80.97	93.22/77.83/70.97	98.01/87.43/78.66	96.81/99.32/70.95
	Overall (%)	96.37/89.46/77.44
	Kappa	0.95/0.87/0.72
	Balanced Accuracy	0.98/0.93/0.86
KLG-FCM	Product (%)	98.01/98.63/95.05	99.13/99.42/95.81	89.32/99.51/96.79	96.07/99.38/91.95	93.87/73.45/80.23
	User (%)	84.07/94.50/93.86	100/95.08/91.53	96.53/84.00/93.27	100.00/92.39/89.26	99.96/99.54/88.71
	Overall (%)	95.65/92.79/91.23
	Kappa	0.95/0.91/0.89
	Balanced Accuracy	0.97/0.95/0.94
TGMM-FCM	Product (%)	98.14/100.00/99.90	96.85/99.99/99.95	100.00/100.00/99.97	97.67/100.00/99.94	97.64/99.94/99.49
	User (%)	100.00/99.96/99.89	100.00/100.00/99.89	93.65/99.87/99.91	100.00/99.98/99.50	99.83/99.99/99.93
	Overall (%)	98.20/99.98/99.79
	Kappa	0.98/0.99/0.99
	Balanced Accuracy	0.97/0.99/0.99

[52]. Indices with larger values correspond to higher accuracy segmentation, while accuracy often suffers from imbalanced data sets. Balanced accuracy (BACC) overcomes this problem by normalizing true positive and true negative predictions with the use of the number of positive and negative samples, respectively [53]. As presented in Table 4, the balanced accuracy was calculated using the confusion matrix. The overall accuracy using the T-FCM algorithm is less than 95.65%, and the Kappa value is less than 0.95. For the T-FCM algorithm, the product accuracy in region (5) of the simulation image with SNR = 10 is less than 50%, while the user accuracy is lower than 70%. As shown by the results, the performance of the T-FCM algorithm is worse with the presence of more noise. Compared with the T-FCM, the KLG-FCM algorithm has higher accuracy, but the product accuracy in region (5) of the simulation image with large noise is still less than 80%. The BACC demonstrates that KLG-FCM is also influenced by noise. For the TGMM-FCM algorithm, the overall accuracy is more than 98.20%, and the Kappa value and BBCC are more than 0.98, which are higher than the other algorithms. The balanced accuracy of simulation images with different SNR are all close to 100%. This means that TGMM-FCM algorithm has higher segmentation accuracy than the other algorithms, is suitable for segmenting complex images accurately and efficiently, and has satisfactory anti-noise performance.

3.2. Multispectral Image Segmentation

To verify the segmentation effect in actual multispectral images, we used the three algorithms in segmenting four sets of four-band (red, green, blue, and near-infrared) multispectral remote sensing images (as shown in Figure 10). The images are displayed in natural color composite (Figure 10a–d) and in near-infrared gray images (Figure 10e–h). In Figure 10a,e, the delineation of seawater and forest is complicated, given that their mean values are similar in the natural-color image, while the demarcation in the near-infrared image is recognizable and distinct given the unique spectral reflectance curve of vegetation. In Figure 10b,f, the infrastructure in the area contains a number of overpasses, which appears as long and narrow features. In the background, two types of features with small differences in gray level are presented, which would entail higher requirements in the algorithm. In Figure 10c,g, given that the pathways in this area are few, they could easily be omitted during segmentation. Moreover, the other three homogeneous regions are covered by different types of vegetation with similar spectral characteristic curves, so the difference in the near-infrared images is small. In Figure 10d, the color of sandy soil and asphalt road is similar, which is hard to distinguish visually and challenging to segment.

As shown in Figure 11(a1), the waves and seawater have obvious color differences. Since the T-FCM algorithm distinguishes dissimilarity using Euclidean distance, each band has the same contribution to the segmentation results. Therefore, a number of pixels that should belong to seawater are grouped together with the beach (see Figure 11(a1)). In Figure 11(a2), the shape of the overpass is roughly delineated, with very apparent discontinuities. In addition, the T-FCM algorithm grouped shaded parts of the overpass together with the lake. From the contour overlay in Figure 11(b2), there are a large number of discrete spots in the segmentation result. In Figure 11(a3), the flower nursery-1 is completely divided, but the pathways and some parts of the lawn are erroneously apportioned into the flower nursery-2. From the delineated overlay image in Figure 11(b3), the segmentation result of the whole area is not continuous. In Figure 11(a4), a small portion of the lawn is assigned to the sandy soil region. Based on Figure 11(b4), the segmentation result roughly describes the distribution of features in the original image.

In Figure 11(c1), even when combined with the neighborhood effect, parts of the wave are still combined with the beach. In Figure 11(c2), the narrow overpass and water are grouped in the same area. There are many misappropriated pixels in the upper left vegetation area, as shown in Figure 11(d2). In Figure 11(c3,d3), only the flower nursery-1 is completely segmented, while the other parts of the image are segmented by mistake. As shown in Figure 11(c4,d4), the segmentation result of the sandy soil is incomplete, and the bright part of lawn is mistakenly segmented into the rubber track.

Figure 11(e1) shows the segmentation results obtained using the TGMM-FCM algorithm with b = 0.7 and q = 1.1. The fuzzy Tsallis entropy can better characterize the spatial correlation between grayscales by adjusting the nonextensive parameter q. The GMM dissimilarity measure can accurately get the mean feature and covariance feature of each region, while the prior probability characterizing the neighborhood effect to be used as the weight of the Gaussian component, can further remove noise effects on the segmentation results. In Figure 11(f1), TGMM-FCM is able to neutralize the brightness change in seawater and accurately divide the seawater region. Figure 11(e2) shows the segmentation result obtained by the TGMM-FCM algorithm at b = 0.05 and q = 1.1. The transport infrastructure is correctly segmented, while the vegetation region showed few erroneously partitioned pixels. Figure 11(e3) shows the segmentation result obtained by the TGMM-FCM algorithm when b = 0.2 and q = 1.2. In this segmentation result, the flower nursery-1, nursery-2, and lawn are completely partitioned. Figure 11(e4) shows the segmentation results obtained by TGMM-FCM algorithm when b = 0.5 and q = 1.1. The result shows that each region is smooth and continuous with good segmentation effect. Simply put, the algorithm can segment remote sensing images with large regional correlation, and the segmentation results would be generally acceptable.

4. Conclusions

In this study, a fuzzy Tsallis entropy-clustering algorithm combined with GMM was proposed. The proposed algorithm utilizes the fuzzy entropy form to fuzzify the Tsallis entropy, defines the fuzzy Tsallis entropy function based on fuzzy membership, and then uses the fuzzy Tsallis entropy as regularization item. The objective function is defined as the difference between the objective function of FCM and the fuzzy Tsallis entropy. Based on mathematical analysis, the fuzzy factor and the nonextensive parameter in the objective function are combined, and the nonextensive parameter q is used to describe the fuzzy degree and nonextensive characteristics of the image. Additionally, the image label field is defined using the HMRF. Prior probability of center pixel with the eight-neighborhood system is computed as the weight of the GMM component to characterize information from neighboring pixels. The dissimilarity measure is defined in the negative logarithmic form.

The experimental results showed that the nonextensive parameter q directly affects image segmentation, and its effective range is from 1.1 to 2.25. The value of q can be adjusted with image statistical features to achieve the ideal segmentation results. Using the proposed algorithm and two other algorithms for comparison, segmentation experiments of simulated and actual multispectral images were carried out. The results from the qualitative and quantitative analyses indicate: (a) the proposed algorithm can quickly segment multispectral images with small color differences; and, (b) the algorithm can accurately calculate the statistical features of remote sensing images and can effectively remove outliers and noise effects from the segmentation results.