Sparse Fuzzy C-Means Clustering with Lasso Penalty

Abstract

Clustering is a technique of grouping data into a homogeneous structure according to the similarity or dissimilarity measures between objects. In clustering, the fuzzy c-means (FCM) algorithm is the best-known and most commonly used method and is a fuzzy extension of k-means in which FCM has been widely used in various fields. Although FCM is a good clustering algorithm, it only treats data points with feature components under equal importance and has drawbacks for handling high-dimensional data. The rapid development of social media and data acquisition techniques has led to advanced methods of collecting and processing larger, complex, and high-dimensional data. However, with high-dimensional data, the number of dimensions is typically immaterial or irrelevant. For features to be sparse, the Lasso penalty is capable of being applied to feature weights. A solution for FCM with sparsity is sparse FCM (S-FCM) clustering. In this paper, we propose a new S-FCM, called S-FCM-Lasso, which is a new type of S-FCM based on the Lasso penalty. The irrelevant features can be diminished towards exactly zero and assigned zero weights for unnecessary characteristics by the proposed S-FCM-Lasso. Based on various clustering performance measures, we compare S-FCM-Lasso with the S-FCM and other existing sparse clustering algorithms on several numerical and real-life datasets. Comparisons and experimental results demonstrate that, in terms of these performance measures, the proposed S-FCM-Lasso performs better than S-FCM and existing sparse clustering algorithms. This validates the efficiency and usefulness of the proposed S-FCM-Lasso algorithm for high-dimensional datasets with sparsity.

1. Introduction

Cluster analysis is an important tool in data analysis. In multiple disciplines, including artificial intelligence, pattern recognition, geology, biology, psychology, and information retrieval, clustering is an essential method for analyzing data. As a result, unlabeled input vectors are grouped into clusters or “natural” groups so that points within the clusters are more similar than points belonging to other clusters, i.e., to maximize intra-cluster similarity while minimizing interclass similarity. Cluster analysis aims to discover patterns, relationships, or structures in datasets by partitioning them into subsets or clusters, where objects within the same cluster are more similar to each other than to those in other clusters. There are many applications for clustering in numerical taxonomy, machine learning, data mining, social network analysis, image segmentation, market segmentation, anomaly detection, and industry. In general, there are two main approaches to clustering: probability model-based [1,2] and nonparametric clustering [3,4]. In this paper, we focus on the nonparametric clustering approach. There are many popular nonparametric clustering algorithms in the literature, such as hierarchical clustering [5,6], mean-shift [7,8], spectral clustering [9,10,11], k-means [12,13,14], fuzzy c-means [15,16,17], and possibilistic c-means [18,19,20]. However, these clustering algorithms may be effective for low-dimensional data, but they do not work well for high-dimensional datasets. The development of mathematical models that emphasize the variable and feature selections of high-dimensional data is a fascinating and developing research topic [21]. Technology has facilitated and encouraged data collection, leading to massive, complex datasets with a wide variety of unique characteristics and dimensions. In general, data with a high number of dimensions may include characteristics that are seen as unimportant or sparse. As a result, with high-dimensional data, the number of dimensions generally becomes insignificant, and so feature selection techniques are being effectively and successfully applied to maximize cluster quality.

In regression analysis, variable and feature selection was well-established by penalizing the $L_1$ norm because it leads to a sparsity property. Tibshirani [22] first proposed a popular one, called Lasso for the “least absolute shrinkage and selection operator”. The coefficients of regression variables may be forced to shrink toward zero via Lasso. Later, Witten and Tibshirani [23] produced the sparse k-means (S-KM) clustering framework as an approach for feature selection in k-means. Witten and Tibshirani [23] considered the S-KM algorithm by using the Lasso constraint in k-means as a mechanism for selecting features. We know that fuzzy c-means (FCM) clustering is a fuzzy extension of k-means and that possibilistic c-means (PCM) clustering is a possibilistic extension of FCM. Subsequently, Qiu et al. [24] expanded S-KM to sparse FCM (S-FCM) clustering by using the Lasso constraint as a mechanism for selecting features. Moreover, Yang and Benjamin [25] expanded S-KM and S-FCM clustering to sparse PCM (S-PCM) clustering in which they introduced two S-PCM algorithms, called S-PCM1 and S-PCM2. These S-PCM1 and S-PCM2 algorithms are subject to the Lasso constraint and Lasso penalty for feature weights, respectively.

Motivated by S-KM, S-FCM, and S-PCM, we propose a new type of S-FCM clustering, called S-FCM-Lasso. For S-FCM-Lasso, we use the FCM objective function with a Lasso (i.e., L1) penalty for feature weights. That is, the S-FCM proposed by Qiu et al. [24] uses the Lasso constraint for selecting features, which is similar to the S-KM algorithm proposed by Witten and Tibshirani [23]. However, our proposed S-FCM-Lasso is a new type of S-FCM under the Lasso penalty, making it different to the S-FCM algorithm. The performance effectiveness of the proposed S-FCM-Lasso algorithm is compared to S-FCM and other existing sparse clustering algorithms by considering both simulated and real datasets. The results demonstrated that our proposed S-FCM-Lasso algorithm has better performance than S-FCM and most existing sparse clustering algorithms by using several important evaluation measures. Furthermore, S-FCM2 gives excellent performance compared to S-FCM1. The proposed S-FCM-Lasso algorithm not only has the capability to produce quick and efficient results, but also has a feature selection ability that works by throwing irrelevant features toward zero. The rest of the paper is organized as follows. Section 2 contains the related literature and the structures of existing algorithms. The proposed S-FCM-Lasso algorithm is presented in Section 3. Section 4 includes the evaluation and comparison of performance of the proposed S-FCM-Lasso and existing clustering algorithms. Section 5 sums up the paper and contains some future recommendations, especially in terms of considering a point symmetry-based distance rather than Euclidean distance for presenting more cluster symmetry behaviors [26,27,28].

2. Related Work

In this section, we review the clustering algorithms proposed in the literature that are related to our work. In 1965, Zadeh [29] first proposed the idea of a fuzzy set with partial memberships in a fuzzy set. Later, Ruspini [30] successfully applied a fuzzy set in clustering. In fuzzy clustering, the fuzzy c-means (FCM) algorithm proposed by Dunn [31] and Bezdek [15] is the most well-known and used method. Fuzzy clustering has been widely studied and applied in a variety of substantive areas [32,33,34,35]. Let $X=\{x_1,x_2,\dots ,x_n\}$ be a dataset in a d-dimensional Euclidean space ${ℝ}^d$. Let $V=\left\{v_1,v_2,\dots ,v_c\right\}$ be the set of cluster centers for the dataset $X$, where $v_{kj}$ represents the $k^{\mathrm{th}}$ cluster with the $j^{\mathrm{th}}$ feature component, $n$ is the number of data points in $X$, $c$ is the number of clusters, and $d$ is the number of feature components. Let the membership matrix be $U={[u_{ik}]}_{n\times c},$ $i=1,2,\dots ,n,$ $k=1,2,\dots ,c$ where $u_{ik}$ is a fuzzy membership of the $i^{\mathrm{th}}$ data point in the $k^{\mathrm{th}}$ cluster of dataset X with ${\displaystyle {\sum }_{k=1}^c{u}_{ik}}=1$ and $u_{ik}\in [0,1]$ for $i=1,2,\dots ,n,$ $k=1,2,\dots ,c$. The FCM objective function [15] was defined as

$$J(U,V)={\displaystyle \sum _{k=1}^c{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^d{u}_{ik}^m{(x_{ij}-v_{kj})}^2}}}$$

(1)

where the weighting exponent $1<m<+\infty$ presents the degree of fuzziness. The FCM algorithm is iterated through the necessary conditions for minimizing $J(U,V)$, with the updating equations for membership function and cluster centers as: For $1\le i\le n$ and $1\le k\le c$, $u_{ik}={\left({\displaystyle \sum _{j=1}^d{(x_{ij}-v_{kj})}^2}\right)}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}/{\displaystyle \sum _{t=1}^c{\left({\displaystyle \sum _{j=1}^d{(x_{ij}-v_{tj})}^2}\right)}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}}$ and $v_{kj}={\displaystyle \sum _{i=1}^n{u}_{ik}^m{x}_{ij}}/{\displaystyle \sum _{i=1}^n{u}_{ik}^m}$. The fuzziness coefficient, or fuzzifier, is denoted by the m, which controls the level of fuzziness or softness in the cluster assignments.

In 1996, Tibshirani [22] first developed the Lasso (least absolute shrinkage and selection operator) regression technique for variable feature selection in regression analysis. The Lasso technique has the ability to shrink some redundant variables/features towards exactly zero so that only useful variables/features are retained. Based on the Lasso constraint in k-means, Witten and Tibshirani [23] proposed the sparse k-means (S-KM) clustering algorithm. The S-KM objective function was given by Witten and Tibshirani [23] as follows:

$$\underset{C_1,C_2,\dots ,C_k,w}{\max }\left\{{\displaystyle \sum _{j=1}^d{w}_j\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{i^{\prime }=1}^n{d}_{i,i^{\prime },j}-{\displaystyle \sum _{k=1}^c\frac{1}{n_k}}{\displaystyle \sum _{i,i^{\prime }\in C_k}{d}_{i,i^{\prime },j}}}}\right)}\right\}$$

(2)

subject to ${\Vert w\Vert }_2^2=1 , {\Vert w\Vert }_1\le \mathrm{s} , w_j\ge 0 \forall j$, where $d_{i,i^{\prime },j}={\left(x_{ij}-x_{i^{\prime }j}\right)}^2$. The weights ${w_j}^{\prime }s$ make a contribution to the clustering and sparsity of the feature $j$ by applying a suitable tuning parameter $s$ selection, where the value of $s$ is with $1\le s\le \sqrt{d}$, which is obtained through the guidance of the gap statistic, and the $d$ under the square root denotes the total number of features. In 2015, Qiu et al. [24] expanded S-KM to sparse FCM (S-FCM) clustering by using the Lasso constraint in FCM. Qiu et al. [24] considered the S-FCM objective function as:

$$\underset{U,V,w}{{\displaystyle \max }}\left[{\displaystyle \sum _{j=1}^d{w}_j\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(x_{ij}-{\overline{x}}_j\right)}^2}-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{i,k,j}^m{d}_{i,k,j}}}\right)}\right]$$

(3)

subject to ${\Vert w\Vert }_2^2\le 1 , {\Vert w\Vert }_1\le s , w_j\ge 0 \forall j$, where $w_j$ is a feature weight for the $j^{\mathrm{th}}$ feature, $j=1,\cdots ,d$, and $d_{i,k,j}={\left(x_{ij}-v_{kj}\right)}^2$ is the Euclidian distance, and ${\overline{x}}_j={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{x}_{ij}}$. The value $s$ lies between $1\le s\le \sqrt{d}$, where $s$ is the tuning parameter. The membership matrix is $U={\left[u_{ik}\right]}_{n\times c}$ and $v_{kj}$ are cluster centers. The choice of features is decided through the tuning parameter $s$ the according to the Lasso constraint. Gap statistics works as a technique for its selection. More weights of features will be zero as the $s$ value decreases. In the process of applying sparse weights for feature selection, S-FCM is able to sustain FCM’s capability to demonstrate fuzzy behavior, resulting in optimal clustering. In this way, the data clusters can be presented in a more concise and meaningful manner while still maintaining accuracy and sparsity. S-FCM extends FCM by introducing Lasso constraints into the membership function matrix, resulting in a more interpretable and robust clustering solution.

Recently, Yang and Benjamin [25] proposed two sparse methods of possibilistic c-mean (PCM) for feature selection in the use of the Lasso concept. The first proposed sparse PCM (S-PCM), called S-PCM1, consists of a framework for feature selection based on Witten and Tibshirani [23] and Qiu et al. [24] by considering the Lasso constraints. The second version of S-PCM, called S-PCM2, includes the (Lasso) $L_1$-regularization of feature weights as a penalty term for the PCM objective function. The S-PCM1 objective function [25] is as follows:

$$\underset{U,V,w}{\max }\left\{{\displaystyle \sum _{j=1}^d{w}_j\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{i^{\prime }=1}^n{(x_{ij}-x_{i^{\prime }j})}^2-\left[{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2+\lambda {\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\left(1-u_{ik}\right)}^m}}}}\right]}}\right)}\right\}$$

(4)

subject to ${\Vert w\Vert }_1\le s,{\Vert w\Vert }^2\le 1,w_j\ge 0\forall j$, where $u_{ik}$ are the possibilistic membership matrix, $w_j$ is a feature weight for the $j^{\mathrm{th}}$ feature, and $s$ is the tuning parameter with $1\le s\le \sqrt{d}$. The S-PCM2 objective function [25] is given below:

$$J_{S-PCM2}\left(U,V,W\right)={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^p{w}_j{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}}+\lambda {\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\left(1-u_{ik}\right)}^m}}+\alpha {\Vert w\Vert }_1$$

(5)

where ${\Vert w\Vert }_1$ is $L_1$-regularization of feature weights as a penalty term and $V=\left[v_1,v_2,\dots ,v_k\right]$ and $U={\left[u_{ik}\right]}_{n\times c}$ are the cluster centers and membership matrix, respectively.

3. The Proposed Sparse FCM Clustering Algorithm

In Section 2, we review the S-FCM proposed by Qiu et al. [24], which is an extension of the S-KM base on the FCM objective function. We mention that the Lasso constraint term is difficult to handle for S-FCM. In particular, the selection of the tuning parameter by using the gap statistic needs to be performed separately from the S-FCM algorithm. In general, it costs too much computation time. To avoid the complex estimate of the tuning parameter in S-FCM, in this section we propose a new type of S-FCM based on the Lasso penalty, called S-FCM-Lasso. The proposed S-FCM-Lasso should be much simpler than the S-FCM by Qiu et al. [24]. We should mention that our proposal is motivated by the S-PCM2 by Yang and Benjamin [25]. The Lasso penalty term of feature weights is added to the FCM objective function. Thus, the proposed S-FCM-Lasso objective function is as follows:

$$J_{S-FCM-Lasso}(U,V,W)={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2+\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}}}}$$

(6)

subject to ${\Vert w\Vert }_2^2=1$ where $u_{ik}$ is a fuzzy membership of the $i^{\mathrm{th}}$ point in the $k^{\mathrm{th}}$ cluster with ${\displaystyle {\sum }_{k=1}^c{u}_{ik}}=1$ and $u_{ik}\in [0,1]$ for $1\le i\le n$ and $1\le k\le c$. $W=\left[w_1,\dots ,w_d\right]\in R^d$ represents the feature weight vector, $U={\left[u_{ik}\right]}_{n\times c}$ is the fuzzy membership matrix, and $V=\left[v_1,v_2,\dots ,v_c\right]$ represents the c cluster centers. The updated equations for minimizing the proposed S-FCM-Lasso objective function $J_{S-FCM-Lasso}\left(U,V,W\right)$ are shown in Theorem 1.

Theorem 1.

The necessary conditions for minimizing the S-FCM-Lasso objective function $J_{S-FCM-Lasso}\left(U,V,W\right)$ are given below:

$$v_{kj}={\displaystyle \sum _{i=1}^n{u}_{ik}^m{x}_{ij}}/{\displaystyle \sum _{i=1}^n{u}_{ik}^m}, 1\le k\le c,1\le j\le d$$

(7)

$$u_{ik}={\displaystyle \frac{{\left[{\displaystyle \sum _{j=1}^d{w}_j{\left(x_{ij}-v_{kj}\right)}^2}\right]}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}}{{\displaystyle \sum _{k^{\prime }=1}^c{\left[{\displaystyle \sum _{j=1}^d{w}_j{\left(x_{ij}-v_{k^{\prime }j}\right)}^2}\right]}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}}}}, 1\le k\le c,1\le i\le n$$

(8)

$$w_j=\{\begin{array}{l}{\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}-\psi }{{\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}-\psi \Vert }_2^2}} , \mathrm{when} {\eta }_j<-\psi \\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}, \mathrm{when} \left|{\eta }_j\right|\le \psi \\ {\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+\psi }{{\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+\psi \Vert }_2^2}} , \mathrm{when} {\eta }_j>\psi \end{array}$$

(9)

where ${\eta }_j={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}$.

Proof. 

We first consider the necessary condition for minimizing the S-FCM-Lasso objective function $J_{S-FCM-Lasso}\left(U,V,W\right)$ with respect to $v_{kj}$ by fixing $u_{ik}$ and $w_j$. We take the partial derivative of $J_{S-FCM-Lasso}\left(U,V,W\right)$ with respect to $v_{kj}$ and set it to be 0 with ${\displaystyle \frac{\partial J_{S-FCM-Lasso}}{\partial v_{kj}}}={\displaystyle \frac{\partial }{\partial v_{kj}}}\left[{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2+\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}}}}\right]=0.$ We have ${\displaystyle \sum _{i=1}^n{u}_{ik}^m{x}_{ij}}-{\displaystyle \sum _{i=1}^n{u}_{ik}^m}{v}_{kj}=0,$ and then $v_{kj}={\displaystyle \sum _{i=1}^n{u}_{ik}^m{x}_{ij}}/{\displaystyle \sum _{i=1}^n{u}_{ik}^m},1\le k\le c,1\le j\le d$ which is Equation (7). We next consider the necessary condition for minimizing $J_{S-FCM-Lasso}\left(U,V,W\right)$ with respect to $u_{ik}$ by fixing $v_{kj}$ and $w_j$. We need to use the Langrange multiplier with the Lagrangian $J_{S-FCM-Lasso}^{\mathrm{L}}\left(U,V,W\right)={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2}}}$$+\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}+{\lambda }_1\left[{\displaystyle \sum _{k=1}^c{u}_{ik}-1}\right]$. Taking the partial derivative of $J_{S-FCM-Lasso}^{\mathrm{L}}\left(U,V,W\right)$ with respect to $u_{ik}$ and setting it to be 0, we obtain ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{\mathrm{L}}}{\partial u_{ik}}}={\displaystyle \sum _{j=1}^{d}m{u}_{ik}^{m-1}{w}_j{\left(x_{ij}-v_{kj}\right)}^2}+{\lambda }_1=0$ with ${\displaystyle \sum _{k=1}^c{u}_{ik}=1}$. Thus, we obtain $u_{ik}={\left[{\displaystyle \sum _{j=1}^d{w}_j{\left(x_{ij}-v_{kj}\right)}^2}\right]}^{\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}/{\displaystyle \sum _{k^{\prime }=1}^c{\left[{\displaystyle \sum _{j=1}^d{w}_j{\left(x_{ij}-v_{k^{\prime }j}\right)}^2}\right]}^{\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}}$, which is Equation (8). Similarly, we consider the necessary condition for minimizing $J_{S-FCM-Lasso}\left(U,V,W\right)$ with respect to $w_j$ by taking the partial derivative of the Lagrangian $J_{S-FCM-Lasso}^{*}\left(U,V,W\right)={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2+\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}+{\lambda }_2\left[{\Vert w\Vert }_2^2-1\right]}}}$ with respect to $w_j$ and set it to be 0, i.e., ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=0.$ The equation can be divided into two parts that separate the penalty expression $\psi {\displaystyle \sum _{j=1}^d{\left|w\right|}_j}$ from the other term so that we can deal both parts of ${\displaystyle \frac{\partial }{\partial w_j}}\left[{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2+{\lambda }_2\left[{\Vert w\Vert }_2^2-1\right]}}}\right]$ and ${\displaystyle \frac{\partial }{\partial w_j}}\left[\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}\right]$ separately. Let $Z={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m{w}_j{\left(x_{ij}-v_{kj}\right)}^2+{\lambda }_2\left[{\Vert w\Vert }_2^2-1\right]}}}$. To differentiate Z partially with respect to $w_j$ and set it to be zero, we have ${\displaystyle \frac{\partial Z}{\partial w_j}}={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m}}{\left(x_{ij}-v_{kj}\right)}^2+2{\lambda }_2{w}_j=0$ with ${\Vert w\Vert }_2^2=1$. Thus, we obtain $w_j=-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}/2{\lambda }_2$, and so ${\Vert w\Vert }_2^2={\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}/2{\lambda }_2\Vert }_2^2={\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}\Vert }_2^2/{\Vert 2{\lambda }_2\Vert }_2^2=1$. We obtain $2{\lambda }_2={\Vert 2{\lambda }_2\Vert }_2^2={\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}\Vert }_2^2.$ Thus, we have $w_j={\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}}{{\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}\Vert }_2^2}}$. For another term ${\displaystyle \frac{\partial }{\partial w_j}}\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}$, we let $L_1=\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}$, which is not differentiable with respect to $w_j$. However, we can use sub-differentiation for this penalty term $L_1=\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}$. We know that a sub-differential is a slope value that is less than the actual slope value between two points. Although the slope cannot be infinite, an infinite number of sub-differentials exists for any two points on a function curve. If we give the function $f\left(w_j\right)=\gamma \left|w_j\right|$ with $\gamma >0$, then the sub-differential of $f$ with respect to $w_j$ should be $\gamma sign\left(w_j\right).$ Thus, to sub-differentiate $L_1$ with respect to $w_j$, we can obtain the term $\psi sign\left(w_j\right)$, and so by using both the equations ${\displaystyle \frac{\partial Z}{\partial w_j}}=Z^{\ast }$ and $\psi sign\left(w_j\right)$, we have ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}}{\partial w_j}}=Z^{\ast }+\{\begin{array}{l}-\psi , when w_j<0 \\ [-\psi ,\psi ], when w_j=0\\ \psi , when w_j>0\end{array},$ i.e., ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}}{\partial w_j}}=\{\begin{array}{ll}{Z}^{\ast }-\psi ,& {\mathrm{when} \mathrm{w}}_{\mathrm{j}}<0\\ \left(Z^{\ast }-\psi ,Z^{\ast }+\psi \right),& {\mathrm{when} \mathrm{w}}_{\mathrm{j}}=0\\ Z^{\ast }+\psi ,& {\mathrm{when} \mathrm{w}}_{\mathrm{j}}>0\end{array}.$ For ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}={\displaystyle \frac{\partial Z}{\partial w_j}}+{\displaystyle \frac{\partial }{\partial w_j}}\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}=0$, we have the following three possibilities to solve this expression:

• Case 1 for $w_j<0$: Let ${\eta }_j={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}$. Since ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=0.$ When $w_j<0$, we have ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=Z^{\ast }-\psi ={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2+2{\lambda }_2{w}_j-\psi }}={\eta }_j+2{\lambda }_2{w}_j-\psi =0$. We obtain $w_j={\displaystyle \raisebox{1ex}{$-{\eta }_j+\psi $}\!\left/ \!\raisebox{-1ex}{$2{\lambda }_2$}\right.}$. And so, when ${\eta }_j>\psi ,$ we have $w_j={\displaystyle \frac{-{\eta }_j+\psi }{2{\lambda }_2}}={\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+\psi }{{\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+\psi \Vert }_2^2}}<0$.
• Case 2 for $w_j>0$: Let ${\eta }_j={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m}}{\left(x_{ij}-v_{kj}\right)}^2$. Since ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=0.$ When $w_j>0$, we have ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=Z^{\ast }+\psi ={\eta }_j+2{\lambda }_2{w}_j+\psi =0.$ We obtain $w_j={\displaystyle \frac{-{\eta }_j-\psi }{2{\lambda }_2}}$. And so, when ${\eta }_j<-\psi ,$ we have $w_j={\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}-\psi }{{\Vert -{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}-\psi \Vert }_2^2}}>0$.
• Case 3 for $w_j=0$: It is necessary to have an interval to contain zero according to sub-differentiation for $L_1=\psi {\displaystyle \sum _{j=1}^d\left|w_j\right|}$ so that $w_j=0$ can be an optimum. In this case, we have $Z^{\ast }-\psi ={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+2{\lambda }_2{w}_j-\psi ={\eta }_j+2{\lambda }_2{w}_j-\psi =0$ with $w_j={\displaystyle \frac{-{\eta }_j+\psi }{2{\lambda }_2}}$. Thus, we can obtain ${\eta }_j\le \psi$ for $w_j\ge 0$. On the other have, we have $Z^{\ast }+\psi ={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}+2{\lambda }_2{w}_j+\psi ={\eta }_j+2{\lambda }_2{w}_j+\psi =0$ with $w_j={\displaystyle \frac{-{\eta }_j-\psi }{2{\lambda }_2}}$. Thus, we can obtain ${\eta }_j\ge -\psi$ for $w_j\le 0$. For these cases where $w_j<0$, we have ${\displaystyle \frac{\partial J_{S-FCM-Lasso}^{*}\left(U,V,W\right)}{\partial w_j}}=Z^{\ast }+\psi ={\eta }_j+2{\lambda }_2{w}_j-\psi =0.$ Since both sides with ${\eta }_j\ge -\psi$ and ${\eta }_j\le \psi$ imply $\left|{\eta }_j\right|\le \psi$, we obtain that $w_j=0$ when $\left|{\eta }_j\right|\le \psi$.

According to cases 1, 2, and 3, the updated weighted equations in the form of Equation (9) are obtained. □

Thus, the S-FCM-Lasso algorithm (Algorithm 1) is as follows:

Algorithm 1. The S-FCM-Lasso Algorithm
Step 1:	Initialize the weights of the features as $W^{(0)}={\left[w_j\right]}_{1\times p}{,\mathrm{w}}_1=w_2=\dots =w_d=1/\sqrt{d}$ and the cluster centers $V^{(0)}$ randomly.
Step 2:	Update the initial membership function $U^{(0)}={\left[u_{ik}\right]}_{n\times c}$ by using $V^{(0)}$ and Equation (8) and $W^{(0)}.$ For a fixed $\epsilon >0$ with $\epsilon =1\times {10}^{-5}$, this is the threshold value for the stopping condition in Step 8 of the algorithm, and $c$ is the cluster size with $2\le c\le n$.
Step 3:	Set t = 1 and update $V^{(t)}$ with an initial $U^{(0)}$ by using Equation (7).
Step 4:	Update $U^{\left(t\right)}$ by using the $V^{(t)}$ of Step 3 and $W^{(0)}$ with the help of Equation (8).
Step 5:	Using the updated $V^{(t)}$ and $U^{\left(t\right)}$ of Steps 3 and 4, respectively, compute w*(t) with the help of the following equation:
	$w_j^{\ast }={\displaystyle \frac{-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{u}_{ik}^m{\left(x_{ij}-v_{kj}\right)}^2}}}{\left[-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{k=1}^c{\displaystyle \sum _{j=1}^d{u}_{ik}^m}{\left(x_{ij}-v_{kj}\right)}^2}}\right]}}$
Step 6:	Select $\psi$ using a binary search technique between $\max \left[w^{*\left(t\right)},0\right]$ and the weighted bound interval $\left[0,\sqrt{d}\right]$.
Step 7:	Update weights $w_j^{\ast (t)} with \psi$ by using Equation (9).
Step 8:	If ${\displaystyle \frac{{\displaystyle \sum _{j=1}^d\left|w_j^{(t)}-w_j^{(t-1)}\right|}}{{\displaystyle \sum _{j=1}^d\left|w_j^{(t-1)}\right|}}}<\epsilon$, while $w_{\mathrm{j}}^{\left(\mathrm{t}\right)}$ is the ${\mathrm{t}}^{\mathrm{th}}$ updated step for $w_{\mathrm{j}}$, then stop;
	ELSE setting t = t + 1 and return back to Step-3.

We next examine how to pick the value of $\psi$. In S-FCM-Lasso, if feature weights and cluster centers are initialized, then membership values can be calculated: we might be able to update the weighted equations, cluster center, and membership function by using them. To identify the parameter that could produce sparsity, a binary search technique is applied. To find out the value of $\psi$, we may use the updated weighted equation after having selected $w^{*}=\max \left(w_j^{*},0\right)$. We finally choose the optimal value of $\psi$ between $\left[w^{\ast },\left(a,\sqrt{d}\right)\right]$ by the binary search method according to Yang and Benjamin [25], where ‘a’ is the positive number. The optimal value of $\psi$ can be obtained by using this strategy and then weights can be updated using the cluster center and membership updating equations. The algorithm terminates after the condition specified in Step-8 is satisfied in accordance with a threshold value $\epsilon$, which is usually set at $1\times {10}^{-5}$. We mention that the binary search technique, also known as half-interval search, logarithmic search, or binary chop, is used to find the position of a target value within a sorted set. The middle element of the array is compared to the target value in a binary search. Here, we obtain $w^{*}=\max \left(w_j^{*},0\right)$ by using the updated weighted equation, and then we pick the optimal value of $\psi$ between the intervals defined by $\left[w^{\ast },\left(a,\sqrt{d}\right)\right]$. Thus, we present the optimal selection procedure for the value of $\psi$ with the binary search technique as follows:

1. 

Parameter range:

Let the parameter $\psi$ of S-FCM-Lasso have the lower and upper bounds $w_{min}^{\ast }$ and $w_{max}^{\ast }$.

2. 

Binary search initialization:

Set the initial search bounds with $w_{\mathrm{low}}^{\ast }=w_{min}^{\ast }$ and $w_{\mathrm{high}}^{\ast }=w_{max}^{\ast }$.

3. 

Iterative binary search:

Let us consider $w_{\mathrm{high}}^{\ast }-w_{\mathrm{low}}^{\ast }>\epsilon$.

i. 

Midpoint computation: $w_{\mathrm{mid}}^{\ast }={\displaystyle \frac{w_{\mathrm{low}}^{\ast }+w_{\mathrm{high}}^{\ast }}{2}}$

ii. 

Execute the S-FCM-Lasso method: Tune the S-FCM-Lasso algorithm using the current $w_{\mathrm{mid}}^{\ast }$. To assess clustering performance, use a suitable measure (validation metric, etc.).

iii. 

Evaluation of results: Comparing the clustering performance to previous iterations. Continue the investigate whether $w_{\mathrm{mid}}^{\ast }$ produces a better result.

iv. 

Set the search parameters:

IF the performance indicates that a higher $w^{\ast }$ can generate better results:

define $w_{\mathrm{low}}^{\ast }=w_{\mathrm{mid}}^{\ast }$; ELSE $w_{\mathrm{high}}^{\ast }=w_{\mathrm{mid}}^{\ast }$.

4. 

Finally, convergence:

The search continues until the gap $w_{\mathrm{high}}^{\ast }-w_{\mathrm{low}}^{\ast }\le \epsilon$ denotes convergence.

5. 

The optimal result for the parameter $\psi$:

The value of $w_{\mathrm{mid}}^{\ast }$ at the stage of convergence is considered to be optimal $\psi$.

In the next section, we present some experimental results and comparisons to evaluate the performance of the proposed algorithm.

4. Clustering Measures, Results, and Comparisons

In this section, we demonstrate the capabilities and efficiency of the proposed S-FCM-Lasso algorithm via comprehensive experiments on both synthetic and real data. In our experiments, we use four synthetic datasets by Qiu et al. [24] and nine real datasets. We compare our proposed S-FCM-Lasso algorithm with FCM, S-KM, S-FCM, S-PCM1, and S-PCM2. In order to determine the efficiency of the proposed S-FCM-Lasso algorithm, a comparison between the true labels (the labels assigned to each data point) and the predicted labels (assigned by the algorithm) is conducted. As evaluation indexes to measure the performance and strength of the proposed algorithm, we use accuracy rate (AR), the rand index (RI) [36], normalized mutual information (NMI) [37], the Jaccard index (JI) [38], the Fowlkes–Mallows index (FMI) [39], and running time (RT). The formal description of these measures is as follows.

Accuracy Rate (AR): Suppose that $C=\left\{C_1,C_2,\cdots ,C_c\right\}$ is the set of c clusters in the actual dataset and that $C^{\prime }=\left\{C_1^{\prime },C_2^{\prime },\cdots ,C_c^{\prime }\right\}$ is the set of c clusters produced by the relevant algorithm. The accuracy rate (AR) is defined as $AR={\displaystyle {\sum }_{k=1}^c{h}_k/n}$, where $h_k$ denotes the number of data points in $C_k$, which are also clustered to $C_k^{\prime }$, and $n$ is the total number of data points in the dataset. The evaluation measures accuracy rate (AR), which measures the percentage of algorithm performance with correctly identified data points.

Rand Index (RI) [36]: The rand index (RI) [36] evaluation measure is another evaluation criterion that measures the similarity between the two distinct clustering datasets. The RI is a metric used to evaluate the correct decisions by an algorithm to assess the overall effectiveness and efficiency of that algorithm. The rand index evaluates the percentage of similarities and disagreements between pairs of data points in two distinct clustering datasets and then generates a similarity score between 0 and 1, where 0 represents no agreement and 1 denotes complete agreement. Let $\left(X_i,X_j\right)$ be the given data points in the form of pairs. Let $a$ be the number of points in a pair that are members of the same cluster in both $C \mathrm{and} {\mathrm{C}}^{\prime }$ and let $b$ be the total number of points that can be paired if both are a part of two distinct clusters in $C$. If the two points are part of the same cluster in $C$ but different clusters in $C^{\prime }$, then the number of pairs of points is $c$, and if they are part of two different clusters in C but the same cluster in $C^{\prime }$, then the number of pairs of points is $d$. The total number of possible point pairs in a given dataset is $M=n\left(n-1\right)/2$ and $n$ is the number of total data points that are used to determine the RI, where RI is computed by $RI=\left(a+d\right)/M$. The RI can measure how accurate the algorithm is.

Normalized Mutual Information (NMI) [37]: In the fields of information theory and data clustering, normalized mutual information (NMI) [37] is a metric of similarity between two sets, A and B, of data. The NMI metric varies between 0 and 1, with 1 denoting complete agreement between the two sets and 0 representing no mutual information. The joint and marginal probabilities are used to measure the level of agreement between clusters. It evaluates how accurately the clustering technique is able to reconstruct a dataset’s original label. We calculate NMI by applying the evaluation measure $NMI=I(A:\mathrm{B})/([H (A)+H (B)] / 2)$, where $H\left(A\right)$ and $H\left(B\right)$ are the marginal entropies of A and B, respectively. $I\left(A:B\right)$ is the mutual information between $H\left(A\right)$ and $H\left(B\right)$.

Jaccard Index (JI) [38]: The Jaccard index (JI) [38], usually referred to as the Jaccard similarity coefficient, is a metric that evaluates how similar or dissimilar two sets are with respect to one another. The JI analyzes members of the two sets in order to identify which members are similar and which are dissimilar [38]. It provides an easy approach to calculating the level of agreement or overlap in sets. The Jaccard index varies from 0 to 1. An evaluation score of 0 denotes that there are no similarities between the sets, demonstrating their full dissimilarity. A score of 1 represents full resemblance, with the sets exactly equivalent. We calculate the JI score by $JI=J\left(A,B\right)={\displaystyle \frac{\left|A\cap B\right|}{\left|A\cup B\right|}}$, where B is a number from either set, while A is a number that exists in both sets. The intersection of the sets A and B is symbolized by $\left|A\cap B\right|$. The magnitude of the unions of A and B is symbolized by $\left|A\cup B\right|.$ According to a, b, c, d, and M, described in the RI, the JI is computed by $JI=a/M$.

Fowlkes–Mallows Index (FMI) [39]: To measure the extent of agreement across two clusterings, Fowlkes and Mallows [39] proposed the Fowlkes–Mallows index (FMI) in 1983. Three kinds of outcomes are taken into consideration by the FMI: false positives (FPs), true positives (TPs), and false negatives (FNs). TP is a count of pairs of points that are presented in the same cluster in both clusterings $C$ and $C^{\prime }$. FP denotes the number of pairs of points that are presented in the same cluster in $C$, but not in $C^{\prime }$. FN is the number of pairs of points that are presented in the same cluster in $C^{\prime }$, but not in $C$. The FMI investigates TPs, FPs, and FNs, which are combined for constructing the FMI with $FMI={\displaystyle \frac{TP}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)}}}$. The FMI has a range from 0 to 1, with a value of 1 indicating perfect agreement between both clusterings $C$ and $C^{\prime }$, and a value close to 0 indicating little to no agreement between $C$ and $C^{\prime }$. Moreover, it illustrates the existence of compact clusters.

Running Time (RT): The total average running time (RT), expressed in seconds, is how long it takes for an algorithm to achieve convergence. Based on the above measures of AR, RI, NMI, JI, FMI, and RT, we next evaluate the performance of our proposed S-FCM-Lasso algorithm in comparison with the existing algorithms.

Evaluation and Comparison with Synthetic Data

In this section, the proposed S-FCM-Lasso algorithm takes advantage of sparsity, and selects the relevant features with nonzero weights and discards the irrelevant features exactly towards zero. This can overcome the drawbacks associated with traditional clustering techniques that have no sparsity behavior. The proposed S-FCM-Lasso algorithm works in two phases. In the first phase, it finds and picks the most relevant attributes, hence reducing the dimensions of the datasets. It benefits from sparsity to enhance computing power as well as interpretation, providing an efficient algorithm to both researchers and practitioners to quickly identify the most important characteristics in a shorter time. For the comparisons, we implemented FCM, S-KM, S-FCM, S-PCM1, S-PCM2, and the proposed S-FCM-Lasso algorithm repeatedly with 100 simulations using various initial values to obtain their averages of AR, RI, NMI, JI, FMI, and RT. These findings are displayed in the below tables. We evaluated the performance and effectiveness of these algorithms using both synthetic and real datasets.

Example 1

(data generated from Qiu [24]). Let the data matrix be $X_{n\times p}$ having dataset size $n\times p$, where $n$ represents the number of data objects and $p$ is the number of attributes. The dataset has an independent and normal distribution and it is generated by an indicator function with three groups; namely, $C_1,C_2{ \& \mathrm{C}}_3$ by Qiu, et al. [24]. The means of the respective generated distributions are $\mu ,-\mu ,0$, which differ exclusively in regard to the first $q\left(q<p\right)$ characteristic, and the irrelevant attributes have a zero mean for the remaining $\left(q-p\right)$ attributes. We have constructed three unique normal distributions with a mean shift only in the first $q$ characteristics among the three clusters $C_1,C_2{ \& \mathrm{C}}_3$. Applying the above method, we constructed a dataset with a number of data items, $n=60 \& p=25$ is the number of attributes, respectively. The values of the parameters as a fuzzifier are $m=2$ and $\mu =2.5$, with $\left(q=5\right)$ being the number of relevant attributes and $\left(q-p=20\right)$ remaining irrelevant attributes. The binary search approach with the proposed S-FCM-Lasso algorithm was applied to determine the value of the tuning parameter psi $\left(\psi \right)$ with respect to the number of iterations.

We initialized the weights through the equation $w_j=1/\sqrt{p}$. The performance of our S-FCM-Lasso algorithm, with comparisons, is shown in

Table 1. Performance comparisons between the classic FCM, S-FCM, S-PCM1, and S-PCM2 and S-FCM-Lasso based on average AR, RI, NMI, JI, FMI, and RT.

Algorithms	Evaluation Criteria
	AR (Avg.)	RI (Avg.)	NMI (Avg.)	JI (Avg.)	FMI (Avg.)	RT (Avg.)
FCM	0.9167	0.9444	0.7218	0.8462	0.9000	0.0510
S-FCM	0.9841	0.9894	0.9423	0.9688	0.9841	2.4758
S-PCM1	0.9650	0.5480	0.0000	0.5480	0.7400	0.0690
S-PCM2	0.9670	0.8100	0.7230	0.6240	0.7730	0.0630
S-FCM-Lasso	1.0000	1.0000	0.9999	0.9999	0.9999	0.0401

. The accuracy rate (AR), rand index (RI), and average running time (RT) were applied as evaluation indicators to investigate the performance and efficacy of our proposed S-FCM-Lasso method. As shown in Table 1, we apply our proposed S-FCM-Lasso algorithm to the datasets generated from a Gaussian distribution to illustrate its efficacy, effectiveness, and performance. In Figure 1, we have shown bar graphs comparing the S-FCM-Lasso algorithm with existing algorithms, i.e., FCM, S-FCM, S-PCM1, and S-PCM2. In Figure 2, the clustering structure is also shown for all algorithms and their performances are also shown in Table 1.

The proposed S-FCM-Lasso algorithm has a small bar after minimization of the objective function (the algorithm converges exactly towards the zero). We can see that the bar graphs of S-FCM-Lasso show that a smaller number of iterations are required to converge the algorithm comparatively, although the exiting S-PCM2 also has a smaller bar (as it has a smaller number of parameters) than the other algorithms; therefore, we can say that our proposed S-FCM-Lasso is faster and converges more efficiently with a smaller number of iterations. In Table 1, which is shown below, we have applied different algorithms, such as FCM, S-FCM, S-PCM1, S-PCM2, and our proposed S-FCM-Lasso, to show their performances and how efficiently they convergence.

The results and comparisons of the above three algorithms in Table 1 have shown that the S-FCM-Lasso algorithm surpasses FCM comparatively and the existing sparse S-FCM, S-KM2, and S-PCM2 clustering algorithms in terms of the average (avg) evaluation indices such as AR, RI, NMI, JI, and FMI. The outcomes in Table 1 demonstrate that S-FCM-Lasso shows excellent results compared to the other algorithms, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right)$, i.e., $1.000>0.9833, 0.9841, 0.9650, \mathrm{a}\mathrm{n}\mathrm{d} 0.9670$. Furthermore, for (RI), $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right)$, i.e., 1.0000 > 0.3333, 0.9894, 0.5480, and 0.8100. A similar behavior of averages (NMI, JI, and FMI) can be observed in Table 1. The avg (RT) is also provided in Table 1; that is, the average total running time. The avg (RT) of FCM (0.0510) is small but the avg (RT) of the proposed S-FCM-Lasso algorithm is the lowest, as compared to S-FCM, S-PCM1, and S-PCM2, i.e., 0.0401 < 2.4758, 0.0690, and 0.0630. We can conclude that the performance of our proposed S-FCM2 algorithm is much better than all the other algorithms. The clustering structure of FCM, S-FCM, S-PCM1, and S-PCM2 is shown in Figure 2a–d and that of S-FCM-Lasso is shown in Figure 2e,f, with respect to true labels and with respect to predicted labels, respectively; that is, this is similar to the results obtained for the real dataset. We can say that our proposed S-FCM-Lasso algorithm also has an excellent ability in terms of good clustering structures.

Example 2

(dataset generated at varying dimensions p). The data points were formed with a Gaussian distribution but categorized into four different groups or clusters, including the generation of data elements with a normal distribution. Similarly to experiment 1, the dataset for this experiment has been generated using numerous numbers increasing in size that vary from 100 to 1000. There are the four clusters in the generated dataset, these being $C_1,C_2,C_3 \mathrm{and} C_4$, the means of the distribution are $-\mu ,\mu ,2\mu \mathrm{and} 0,\left(\mu =2.5\right)$, respectively, which first consist of $q$ significant characteristics. The remaining (q-p) characteristics with a mean of zero indicate that they are all insignificant features. The numerical value of the fuzzifier is $m=2$. We obtained the clustering results for our proposed S-FCM-Lasso algorithm in experiment 2 with an increasing number of dimensions from 100 to 1000, and its comparative analysis with the classic FCM, S-FCM, S-PCM1, and S-PCM2 is shown in Figure 3. Multiple bar graphs illustrating the averages of the parameters AR, RI, NMI, JI, FMI, and RT (total running time in seconds) are shown in Figure 3. In Figure 3, the multiple bar graphs of our proposed S-FCM-Lasso show its superiority over all of the other algorithms, i.e., the classic FCM and the competing S-FCM, S-PCM1, and S-PCM2 clustering algorithms. We conclude that our proposed S-FCM-Lasso algorithm outperforms all the other algorithms in Figure 3a–f, respectively.

Example 3

(based on simulated data for n = 100, 200… 1000). This example is based on the generated dataset of three clusters by using the method of Qiu, et al. [24] for an increasing number of sample sizes, n = 100, 200… 1000, and a fixed number of dimensions p = 1000. The first 25 dimensions are considered as relevant and the remaining 975 as irrelevant features with parametric values mu= 3 and m = 2. The resulting values of the evaluation indices AR, RI, NMI, JI, and FMI for the proposed S-FCM-Lasso clustering method along with the existing S-FCM, S-PCM1, S-PCM2, and classic FCM are shown in Figure 4, based on the averages of these evaluation measures, using multiple bar graphs. In Figure 4, the multiple bar graphs represent the averages of evaluation indices, i.e., AR, RI, NMI, JI, FMI, and RT, respectively. We observe that our proposed S-FCM-Lasso algorithm surpasses all the other algorithms in Figure 4a–f, respectively. S-FCM-Lasso has been shown to have a low average running time (RT) in multiple bar graphs in Figure 4f. Therefore, we conclude that our proposed S-FCM-Lasso algorithm has shown good clustering performance and excellent clustering results. We can say that S-FCM-Lasso outperforms all the other competing algorithms.

Example 4

(dataset generated from Gaussian mixture distribution). The dataset is generated from multivariate Gaussian mixture distribution ${\displaystyle {\sum }_{k=1}^3{\alpha }_{k}N\left({\mu }_k,{\Sigma }_k\right)}$ with parameters and mean vectors being ${\alpha }_k={\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}, k=1,2,3, {\mu }_1=[3,3,3]; {\mu }_2=[6,6,6]; {\mu }_3=[9,9,9]$ and the identity variance covariance sigma matrices being ${\Sigma }_k=0.30I_3$. The generated dataset of 450 values is distributed into three clusters $C_1,C_2,C_3$, each of a size of 150 with 15 features or dimensions. Among these features, three features are considered relevant features. The remaining twelve features are generated from a uniform distribution with zero mean. The value of the fuzzifier is $m=2$. We generated the above dataset using the famous Monte-Carlo simulation technique with 100 simulations to apply our proposed S-FCM-Lasso algorithm and compare it with the classic FCM and the existing S-FCM, S-PCM1, and S-PCM2 algorithms. The resulting values of the different evaluation measures used to compare these algorithms are provided in

Table 2. Comparisons of algorithms FCM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso through AR, RI, NMI, JI, FMI, and RT.

Algorithms	Evaluation Criteria
	AR (Avg.)	RI (Avg.)	NMI (Avg.)	JI (Avg.)	FMI (Avg.)	RT (Avg.)
FCM	0.9547	0.5639	0.9548	0.1889	0.3178	0.0246
S-FCM	0.9987	0.9991	0.9940	0.9974	0.9987	3.3712
S-PCM1	0.9990	0.9720	0.9700	0.9500	0.9720	0.0900
S-PCM2	0.9950	0.8550	0.8210	0.7410	0.8470	0.1230
S-FCM-Lasso	1.0000	1.0000	1.0000	1.0000	1.0000	0.0212

. The bar graph is shown in Figure 5 and the clustering patterns of the different algorithms are shown in Figure 6.

The results in Table 2 demonstrate that our proposed S-FCM-Lasso algorithm outperforms the classic FCM and the existing sparse S-FCM, S-PCM1, and S-PCM2 algorithms in terms of averages (Avg.) of the evaluation measures AR, RI, NMI, JI, and FMI, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right),$ $\mathrm{i}.\mathrm{e}.,1.0000>0.9547, 0.9987,0.9990, \mathrm{a}\mathrm{n}\mathrm{d} 0.9950$. In addition, for RI, NMI, JI, and FMI, respectively, we can see that 1.0000 > 0.5639, 0.9991, 0.9720, and 0.8550 for NMI, 1.0000 > 0.9548, 0.9940, 0.9700, and 0.8210, and for JI 1.0000 > 0.1889, 0.9974, 0.9500, and 0.7410, and also for FMI 1.0000 > 0.3178, 0.9987, 0.9720, and 0.8470. The average total running time, avg. (RT), of our proposed S-FCM-Lasso algorithm (0.0212) is lower than the avg. (RT) of all the other competing algorithms such as the classic FCM and the existing S-FCM, S-PCM1, and S-PCM2, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{T}}_{S-FCM-Lasso}\right)<\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{T}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{T}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{T}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}.,$0.0212 < 0.0246, 3.3712, 0.0900, and 0.01230. Therefore, we concluded that S-FCM-Lasso has superior results compared to all of the other competing algorithms mentioned above.

Moreover, the bar graph of S-FCM-Lasso in Figure 5 is the smallest due to its simplicity, with a smaller number of iterations required for the algorithm to converge. The clustering structures of the classic FCM and the competing S-FCM, S-PCM1, and S-PCM2 algorithms are shown in Figure 6a–d, which further illustrates the clustering structure of our proposed sparse S-FCM-Lasso algorithm, as shown in Figure 6e,f, with respect to true and predicted labels. We concluded that the overall structure was consistent across the actual and predicted labeling for all the algorithms mentioned above. Thus, it is obvious that the proposed clustering S-FCM-Lasso algorithm can produce superior clustering patterns. As a result, we concluded that our proposed S-FCM-Lasso performed well based on good clustering structures and results that are very efficient, leading us to claim that our proposed S-FCM-Lasso algorithm is superior to all other algorithms.

Example 5

(iris dataset). In the fields of machine learning and statistics, the iris dataset [40,41] is a widely utilized dataset. Since British statistician and biologist Sir. Fisher first published it in 1936 [36], it has grown to be a standard dataset for exploring pattern recognition and classification approaches. The Setosa, Versicolor, and Virginica varieties of iris flowers are the three different species that collectively make up the iris dataset. There are four characteristics that each flower sample has: sepal length (SL), sepal width (SW), petal length (PL), and petal width (PW). In Figure 7, we illustrate that our proposed S-FCM-Lasso algorithm exhibits the best number of iterations compared to other existing algorithms. The results of feature weights based on S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso for the iris dataset are shown in

Table 3. Results of feature weights based on S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso for iris dataset.

Algorithms	Feature Weights
	PL	PW	SL	SW
S-FCM	0.0000	0.0000	0.4113	0.9115
S-PCM1	0.1180	0.0000	0.9810	0.1560
S-PCM2	0.0150	0.0000	0.2400	0.0000
S-FCM-Lasso	0.0000	0.0000	0.1434	0.8566

.

Table 4. Performances of FCM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso with nonzero weights.

Algorithms	Evaluation Criteria
	AR (Avg.)	RI (Avg.)	NMI (Avg.)	JI (Avg.)	FMI (Avg.)	RT (Avg.)
FCM	0.9580	0.9500	0.8377	0.8867	0.9370	0.1028
S-FCM	0.9700	0.9600	0.8642	0.9223	0.9500	3.3404
S-PCM1	0.9690	0.7630	0.7190	0.5780	0.7580	0.0850
S-PCM2	0.9620	0.8100	0.6950	0.6500	0.7930	0.0780
S-FCM-Lasso	0.9997	0.9733	0.8705	0.9234	0.9600	0.0742

shows the performances of FCM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso with the top-ranked values marked in bold and the second-ranked values underlined.

The results in Table 4 show that the S-FCM-Lasso algorithm outperforms the classic FCM algorithm and the existing S-FCM, S-PCM1, and S-PCM2 algorithms based on the average (Avg.) evaluation indicators AR, RI, NMI, JI, and FMI, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}.,$0.9997 $>0.9580, 0.9700,0.9690, \mathrm{a}\mathrm{n}\mathrm{d} 0.9620$. For the average (Avg.) rand index (RI), $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}.,$0.9997 $>0.950, 0.9600,0.7630,0.8100, \mathrm{a}\mathrm{n}\mathrm{d} 0.9620$. The other average evaluation measures, NMI, JI, and FMI, have revealed similar behavioral trends with $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM2}\right)$, i.e., 0.8705 $>0.8377,0.8642, 0.7190, \mathrm{a}\mathrm{n}\mathrm{d} 0.6950$, and similarly for the other JI and FMI measures, i.e., 0.9234 $>0.8867,0.9223, 0.5780, \mathrm{a}\mathrm{n}\mathrm{d} 0.6500,$ and $0.9600>0.9370,0.9500, 0.7580, \mathrm{a}\mathrm{n}\mathrm{d} 0.7930$. The average total running time (RT) of S-FCM-Lasso (0.0742) is the lowest but the avg(RT) of S-PCM2 (0.0780) is also low.

Consequently, we conclude that our proposed S-FCM-Lasso has shown better performance than the other algorithms; namely, the classic FCM and the competing S-FCM, S-PCM1, and S-PCM2 algorithms. The proposed S-FCM-Lasso algorithm has the ability to retain relevant features by assigning nonzero weights and discard the irrelevant features by assigning zero weights to them, as shown in Table 3. The bar graph of S-FCM-Lasso is also the smallest bar graph, as shown in Figure 7. The proposed S-FCM-Lasso algorithm consumes a small number of iterations to achieve minimization (algorithm convergence). The clustering structures of the classic FCM and competing S-FCM, S-PCM1, and S-PCM2 algorithms is shown in Figure 8a–d. The graphical clustering structure of the proposed S-FCM-Lasso is further illustrated in Figure 8e,f with respect to the true and predicted labels. We ultimately concluded that, in general, the structure of the algorithms was similar for actual and predicted labeling. As a consequence, it seems obvious that the proposed S-FCM-Lasso clustering algorithm has the potential of generating effective clustering structures. Consequently, we ultimately concluded that our proposed S-FCM-Lasso algorithm performed well in terms of clustering structure and produced excellent results that were successful and efficient, leading us to claim that it is superior to all other competing algorithms.

Example 6

(abalone datasets). Abalone is the common name for any group of small to very large sea snails, commonly found along the coasts across the world, used as a delicacy in cuisines, and its leftover shell is fashioned into jewelry due to its iridescent luster. Due to its demand and economic value, it is often harvested on farms. There are many species of sea snails known as abalone, which can range in size from small to very large, and they are found across the world. Their shells are used as delicacies and fashioned into jewelry because of their iridescent color. Farms often harvest them due to their high economic value and demand. This dataset is available on open access data source Kaggle. The total number of observations in the abalone dataset is 4177 and the total number of features is eight, which includes sex, length, diameter, height, whole weight, shucked weight, viscera weight, and shell weight. We applied our proposed S-FCM-Lasso algorithm to this dataset and compared it with the other competing algorithms, i.e., the FCM, S-FCM, S-PCM1, and S-PCM2 clustering algorithms.

Table 5. Comparisons of algorithms FCM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso based on average (Avg.) evaluation measures AR, RI, NMI, JI, FMI, and RT for the abalone dataset.

Algorithms	Evaluation Criteria
	AR (Avg.)	RI (Avg.)	NMI (Avg.)	JI (Avg.)	FMI (Avg.)	RT (Avg.)
FCM	0.7524	0.8242	0.4452	0.5543	0.7524	0.1380
S-FCM	0.9562	0.8390	0.4389	0.5546	0.7535	2.0369
S-PCM1	0.9582	0.5560	0.0420	0.5440	0.7410	2.0272
S-PCM2	0.9600	0.5030	0.0400	0.0530	0.6720	0.1273
S-FCM-Lasso	0.9997	0.8392	0.4484	0.6092	0.7571	0.1261

compares the S-FCM-Lasso algorithm with the other algorithms and shows the following performance results:

The results in Table 5 show that the S-FCM-Lasso algorithm demonstrated excellent performance compared to the classic FCM algorithm and the competing S-FCM, S-PCM1, and S-PCM2 algorithms based on the average (Avg.) evaluation indicators AR, RI, NMI, JI, and FMI, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right)$, i.e., 0.9997 $>0.7524, 0.9562,0.9582, \mathrm{a}\mathrm{n}\mathrm{d} 0.9600$. For the average (Avg.) rand index (RI), $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right),$ i.e., 0.8392 $>0.8242, 0.8390,0.5560, \mathrm{a}\mathrm{n}\mathrm{d} 0.5030$. A similar behavioral pattern was observed in the other average evaluation measures, NMI, JI, and FMI i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM2}\right)$, i.e., $0.4484>0.4452,0.4389, 0.0420, \mathrm{a}\mathrm{n}\mathrm{d} 0.0400$, and similarly for the other JI and FMI measures, i.e., 0.609 $>0.5543,0.5546, 0.5440, \mathrm{a}\mathrm{n}\mathrm{d} 0.0530$ and 0.757 $>0.7524,0.7535, 0.7410, \mathrm{a}\mathrm{n}\mathrm{d} 0.6720$. The average total running time (RT) of S-FCM-Lasso (0.1261) is the lowest, but the avg(RT) of S-PCM2 (0.1273) is also low.

Example 7

(diabetes datasets). The original source of this dataset is the National Institute of Diabetes and Digestive and Kidney Diseases. The objective is to determine if a patient has diabetes or not using diagnostic parameters. From a large database, these instances were selected based on several constraints. All patients here are females of Pima Indian heritage who are at least 21 years old. The dataset has one target variable, outcome, and other medical explanatory factors. a patient’s age, BMI, insulin level, number of pregnancies, and other factors are all considered predictor variables. The data used in this experiment is publicly available on Kaggle. The total number of observations in the diabetes dataset is 768 and the total number of features is eight, which includes pregnancies, glucose, blood pressure, skin thickness, insulin, BMI, diabetes pedigree, age, and outcome. We used this dataset to apply our proposed S-FCM-Lasso method and compare it to all other competing clustering algorithms, such as FCM, S-FCM, S-PCM1, and S-PCM2. The performance of our proposed S-FCM-Lasso with an algorithmic comparison in

Table 6. Comparisons of algorithms FCM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso based on average (Avg.) evaluation measures AR, RI, NMI, JI, FMI, and RT for the diabetes dataset.

Algorithms	Evaluation Criteria
	AR (Avg.)	RI (Avg.)	NMI (Avg.)	JI (Avg.)	FMI (Avg.)	RT (Avg.)
FCM	0.7524	0.8349	0.4376	0.4988	0.7524	0.1482
S-FCM	0.9200	0.8090	0.4389	0.5546	0.7135	2.0369
S-PCM1	0.9560	0.5599	0.0510	0.5583	0.7500	2.0590
S-PCM2	0.9600	0.5533	0.0400	0.0530	0.6720	0.1293
S-FCM-Lasso	0.9999	0.8381	0.4589	0.6092	0.7591	0.1240

illustrates the effectiveness of the proposed S-FCM-Lasso algorithm:

The results are summarized in Table 6, which shows that the S-FCM-Lasso approach outperforms the existing S-FCM, S-PCM1, S-PCM2, and traditional FCM methods when considering the average (Avg.) evaluation indicators AR, RI, NMI, JI, and FMI, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}.,$0.9999 $>0.7520, 0.9200,0.9560, \mathrm{a}\mathrm{n}\mathrm{d} 0.9600$. For the average (Avg.) rand index (RI), $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}.,$0.8381 $>0.8349, \mathrm{0.8090,0.5599}, \mathrm{a}\mathrm{n}\mathrm{d} 0.5533$. The other average evaluation measures, NMI, JI, and FMI, reveale similar behavioral trends: $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM1}\right), \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM2}\right)$, i.e., 0.4589 $>0.4376,0.4389, 0.0510, \mathrm{a}\mathrm{n}\mathrm{d} 0.0400$ and similarly for the other JI and FMI measures, i.e., 0.6092 $>0.4988,0.5546, 0.5583, \mathrm{a}\mathrm{n}\mathrm{d} 0.0530$ and $0.7591>0.7524,0.7135, 0.7500, \mathrm{a}\mathrm{n}\mathrm{d} 0.6720$. The average total running time (RT) of S-FCM-Lasso (0.1240) is the lowest but the avg(RT) of S-PCM2 (0.1293) is also low.

Example 8

(real datasets). The experiment incorporates different real-world datasets including breast cancer, seeds, vertebrae, sonar, diabetes, abalone, and lungs taken from the UCI Machine Learning Repository [41] and ODDS Library [42]. These real datasets were subjected to the exiting FCM and our proposed S-FCM-Lasso, to evaluate their algorithmic effectiveness as well as their potential for identifying the relevant features and discarding those that are irrelevant. The performance of both the existing S-FCM and the proposed S-FCM-Lasso clustering algorithms regarding feature selection (the number of relevant features with nonzero weights) is illustrated in

Table 7. Based on real datasets, the performance evaluation and potential of S-FCM1 and S-FCM2 for the remaining features.

Real Datasets	Data Points (n)	Features (p)	Clusters (c)	No. of Remaining Features
				S-FCM	S-FCM-Lasso
Breast cancer	569	30	2	10	12
Vertebral	310	6	2	1	1
Seeds	210	7	3	2	2
Sonar	208	60	2	33	35
Lung	309	13	4	11	10
Diabetes	768	8	2	3	3
Abalone	4177	8	3	4	4

. Moreover, Table 6 shows that both algorithms can select relevant features with nonzero weights and can shrink irrelevant features because the Lasso penalty on the feature weights is included.

The two sparse algorithms, S-FCM and our proposed S-FCM-Lasso algorithm, are compared and examined in Table 7. We demonstrate that both algorithms are capable of generating sparse feature weights and, by discarding unnecessary features due to the inclusion of the lasso penalty, giving them exactly zero weights, and both algorithms have the ability to select the most appropriate features.

Example 9

(other real datasets). In this experiment, we used a variety of other real datasets, such as breast cancer, seeds, vertebrae, cardiac activity, sonar and lungs [38,39]. In this study, we reviewed a few authentic real datasets from the health and medical sciences using a variety of dimensions. We subsequently applied our proposed method to these real datasets to analyze its usefulness, efficiency, and proficiency in selecting important features. These real datasets have been generated by scientists and researchers in a variety of domains of life, i.e., science, research, machine learning projects, and numerous other fields. We performed 100 simulations with different initial values to investigate the clustering results. In our investigation, we obtained the value of tuning parameter s with fuzzifier $m=2$, considering the statistics gap in S-FCM. We applied our proposed S-FCM-Lasso algorithm on these real datasets and compared it with the classic FCM and competing S-FCM, S-KM, S-PCM1, and SPCM2 clustering algorithms in order to evaluate their performance as well as their feature selection capability. In

Table 8. A comparison of the performance of FCM, S-KM, S-FCM, S-PCM1, S-PCM2, and S-FCM-Lasso based on average AR, RI, NMI, JI, and FMI using different real datasets.

Datasets	Evaluation Indicators	AR	RI	NMI	JI	FMI
Breast Cancer	fuzzy c-means (FCM)	0.8541	0.4306	0.4371	0.0787	0.1459
	sparse k-means (S-KM)	0.9940	0.8950	0.6800	0.8270	0.9050
	S-FCM	0.9700	0.8090	0.4389	0.5546	0.7135
	S-PCM1	0.9960	0.5280	0.0440	0.5190	0.7090
	S-PCM2	0.9970	0.8840	0.7160	0.8030	0.8910
	S-FCM-Lasso	0.9999	0.9228	0.7372	0.8454	0.9541
Vertebral	fuzzy c-means (FCM)	0.8635	0.4402	0.4332	0.0726	0.1654
	sparse k-means (S-KM)	0.9920	0.5100	0.0990	0.4710	0.8830
	S-FCM	0.9920	0.6882	0.2168	0.3626	0.5323
	S-PCM1	0.9940	0.7800	0.0000	0.7800	0.8830
	S-PCM2	1.0000	0.5540	0.1550	0.1310	0.6690
	S-FCM-Lasso	1.0000	0.8570	0.5461	0.6671	0.7852
Sonar	fuzzy c-means (FCM)	0.8667	0.4773	0.0513	0.0923	0.1660
	sparse k-means (S-KM)	0.9810	0.5200	0.0280	0.0500000	0.6890
	S-FCM	0.9820	0.7179	0.0357	0.4054	0.5769
	S-PCM1	1.0000	0.8990	0.0000	0.05300	0.9480
	S-PCM2	1.0000	0.5030	0.0400	0.0530	0.6720
	S-FCM-Lasso	1.0000	0.7201	0.0482	0.5833	0.6993
Arrhythmia	fuzzy c-means (FCM)	0.8533	0.4677	0.4302	0.0952	0.1703
	sparse k-means (S-KM)	1.0000	0.5340	0.0140	0.1190	0.6460
	S-FCM	0.9922	0.8723	0.4460	0.4872	0.4502
	S-PCM1	0.9920	0.4980	0.0000	0.4980	0.7060
	S-PCM2	1.0000	0.6570	0.0110	0.1010	0.7750
	S-FCM-Lasso	1.0000	0.8813	0.4632	0.4896	0.9332
Seeds	fuzzy c-means (FCM)	0.7524	0.8312	0.4476	0.5031	0.7524
	sparse k-means (S-KM)	0.9650	0.8270	0.6420	0.5900	0.7420
	S-FCM	0.9564	0.8123	0.6220	0.5762	0.7232
	S-PCM1	0.9732	0.8317	0.4233	0.5972	0.7476
	S-PCM2	0.9720	0.7410	0.5680	0.5220	0.7000
	S-FCM-Lasso	0.9753	0.8381	0.6400	0.6092	0.7571
Cardio	fuzzy c-means (FCM)	0.8224	0.8042	0.4375	0.4031	0.7424
	sparse k-means (S-KM)	0.9980	0.5040	0.0070	0.4570000	0.6460
	S-FCM	0.9980	0.8090	0.4389	0.4546	0.7135
	S-PCM1	1.0000	0.8260	0.0000	0.0550	0.9090
	S-PCM2	1.0000	0.8260	0.0000	0.0960	0.9090
	S-FCM-Lasso	1.0000	0.8421	0.4484	0.6092	0.9071
Lungs	fuzzy c-means (FCM)	0.8552	0.6117	0.4375	0.2638	0.4175
	sparse k-means (S-KM)	0.9610	0.7530	0.6070	0.5650	0.7210
	S-FCM	0.9960	0.6780	0.0042	0.5172	0.4822
	S-PCM1	0.9610	0.4950	0.0000	0.4950	0.7030
	S-PCM2	0.9360	0.6150	0.3130	0.4070	0.5790
	S-FCM-Lasso	0.9986	0.6785	0.0288	0.5178	0.4822

, we display the performance of S-FCM-Lasso with the other competing algorithms based on different criteria of evaluation: average accuracy rate (AR), RI, NMI, JI, FMI. Table 8 summarizes the results, with bold results denoting the highest values and highlighted results denoting the second-highest values.

In Table 8, we applied the classic FCM and proposed S-FCM-Lasso algorithms and the competing S-FCM, S-KM, S-PCM1, and S-PCM2 algorithms to these real datasets. Moreover, we compared and analyzed our proposed S-FCM-Lasso algorithm with the other existing algorithms. Several evaluation measures (average AR, RI, FMI, JI, MNI, and RT) assessed when the proposed S-FCM-Lasso algorithm achieves convergence, having been obtained using different real datasets. Moreover, we found, from the above table, that our proposed S-FCM-Lasso algorithm outperformed the other algorithms and displayed superior effectiveness and efficient findings compared with those obtained using the classic FCM and existing S-FCM, S-KM, S-PCM1, and S-PCM2 algorithms, i.e., for cancer data: $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{FCM}\right), \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-KM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM1}\right)$, $\mathrm{a}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-PCM2}\right) \mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h} 0.9999>0.8541,0.9940,0.9700,0.9960, \mathrm{a}\mathrm{n}\mathrm{d} 0.9970.$ In addition, we have shown that our proposed S-FCM-Lasso outperforms the classic FCM and existing S-KM, S-FCM, S-PCM1, and S-PCM2 algorithms based on the average (Avg.) RI: $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{A}\mathrm{R}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-KM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM1}\right)$, $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{R}\mathrm{I}}_{S-PCM2}\right), \mathrm{i}.\mathrm{e}., 0.9228>0.4306,0.8950,0.4389,0.8090, \mathrm{a}\mathrm{n}\mathrm{d} 0.5280$.

Similarly, for the other evaluation measures, NMI, JI, and FMI, i.e., $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-KM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM1}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM1}\right),$ $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{N}\mathrm{M}\mathrm{I}}_{S-PCM2}\right)$ with $0.7372>0.4371,0.6800,0.8950,0.4389,0.0440, \mathrm{a}\mathrm{n}\mathrm{d} 0.7160$ and $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{S-KM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{S-PCM1}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{J}\mathrm{I}}_{S-PCM2}\right),$ i.e., $0.8454>0.0787,0.8270,0.5546,0.5190, \mathrm{a}\mathrm{n}\mathrm{d} 0.8030$ and for FMI we have $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{S-FCM-Lasso}\right)>\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{S-KM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{S-FCM}\right),\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{S-PCM1}\right),$ $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathrm{F}\mathrm{M}\mathrm{I}}_{S-PCM2}\right),$ i.e., $0.9541>0.1459,0.9050,0.7135,0.7090, \mathrm{a}\mathrm{n}\mathrm{d} 0.8910.$ We found similar behavior for the other datasets; namely, arrhythmia, cardio and lungs, based on average (Avg.) AR, RI, NMI, JI, and FMI evaluation measures. Furthermore, we can observe the effectiveness of our proposed S-FCM-Lasso algorithm on seeds and sonar datasets in Table 7. Therefore, we can say that the proposed S-FCM-Lasso method is faster than the other classic and competing algorithms mentioned above. Similarly, we analyzed the behavior of our proposed S-FCM-Lasso algorithm for other datasets and compared it with the competing algorithms mentioned above. In conclusion, as shown in Table 8, our proposed S-FCM-Lasso algorithm performed better than the other algorithms across numerous real datasets. It demonstrated highly efficient and successful results compared to the other algorithms. For S-FCM to obtain a tuning parameter value by using a gap statistic consumes more time than for S-FCM-Lasso. The S-FCM-Lasso method performed better than the FCM, S-KM, S-FCM, S-PCM1, and S-PCM2 algorithms.

5. Conclusions

In this article, we proposed our clustering algorithm S-FCM-Lasso, which not only efficiently and successfully seeks out significant features but is also capable of eliminating insignificant ones by adding sparsity to feature weights. On both simulated and real datasets, the algorithm demonstrated superior clustering results. The S-FCM-Lasso algorithm has a good capacity to identify and discard redundant features since it can pick out key features by assigning nonzero weights to relevant features and zero weights to irrelevant features. Using the Lasso penalty, the proposed algorithm is capable of achieving sparsity with feature weights by shredding irrelevant features down to exactly zero. In S-FCM, sparsity depends on the value of the tuning parameter $\psi$, which can be achieve by gap statistics and is more complicated and time-consuming. The competing S-FCM algorithm applies a more complex gap statistics procedure, while our proposed S-FCM-Lasso applies a much simpler algorithmic convergence. The sparse feature weights cannot be produced with a high $\psi$ value. A tuning parameter that is suitably small is required to achieve the optimum results.

As a result, we can conclude that S-FCM-Lasso has shown superior performance over all existing algorithms via both synthetic and real datasets, performs excellently, and has shown efficient clustering results with a feature selection ability. We are able to say that proposed S-FCM-Lasso outperforms the other traditional FCM and existing S-KM, S-FCM, S-PCM1, and S-PCM2 algorithms. Moreover, the proposed S-FCM-Lasso performed better on both synthetic and real datasets and also showed good clustering structures. In fuzzy c-means, data points can be grouped into multiple clusters with varying degrees of membership, while k-means performs hard clustering, where each data point is assigned exclusively to one cluster. On the other hand, S-FCM-Lasso and S-PCM2 have greater flexibility in accommodating data than S-KM, which leads us to conclude that they are more flexible than S-KM. Furthermore, our proposed S-FCM-Lasso algorithm has shown an excellent ability to pick out the relevant features by assigning them nonzero weights and discarding the irrelevant features towards exactly zero by taking advantage of the Lasso penalty. In summary, S-FCM-Lasso has shown powerful clustering results and excellent cluster structures with a good capability for feature selection that performs more efficiently than S-KM, S-FCM, S-PCM1, and S-PCM2 on both synthetic and real datasets. Since a point symmetry-based distance [25] can represent better behavior for cluster symmetry [26] than Euclidean distance, in our further work, we will extend the S-FCM-Lasso by using a point symmetry-based distance so that it can present better cluster symmetry.