Hybrid Intelligent Pattern Recognition Systems for Mass Segmentation and Classification: A Pilot Study on Full-Field Digital Mammograms

Abstract

Governments and health authorities emphasize the importance of early detection of breast cancer, usually through mammography, to improve prognosis, increase therapeutic options and achieve optimum outcomes. Despite technological advances and the advent of full-field digital mammography (FFDM), diagnosis of breast abnormalities on mammographic images remains a challenge due to qualitative variations in different tissue types and densities. Highly accurate computer-aided diagnosis (CADx) systems could assist in the differentiation between normal and abnormal tissue and the classification of abnormal tissue as benign or malignant. In this paper, classical, advanced fuzzy sets and fusion techniques for image enhancement were combined with three different thresholding methods (Global, Otsu and type-2 fuzzy sets threshold) and three different classifying techniques (K-means, FCM and ANFIS) for the classification of breast masses on FFDM. The aim of this paper is to identify the performance of the advanced fuzzy sets, fuzzy sets type-2 segmentation, decisions based on K-means and FCM, and the ANFIS classifier. Sixty-three combinations were evaluated on ninety-seven digital mammographic masses (sixty-five benign and thirty-two malignant). The performance of the sixty-three combinations was evaluated by estimating the accuracy, the F1 score, and the area under the curve (AUC). LH-XWW enhancement method with Otsu thresholding and FCM classifier outperformed all other combinations with an accuracy of 95.17%, F1 score of 89.42% and AUC of 0.91. This algorithm seems to offer a promising CADx system for breast cancer diagnosis on FFDM.

1. Introduction

Breast cancer is currently the most common cancer globally and the second leading cause of cancer death in women today [1]. Supporting physician’s diagnostic decisions has become an important application of artificial intelligence (AI), the role of which is expected to expand in the future [2]. Hybrid intelligent systems (HIS) in the field of medical pattern recognition have been widely applied for data classification. HIS combines intelligence tools and methods that make sense on a theoretical basis and can be applied practically. The unique characteristic of a HIS is that it can integrate two or more intelligent techniques that maintain their operating principles and attributes despite being fused or transformed. HIS performance depends on the selected intelligent systems and their architecture. In this paper, the HIS adaptive neuro-fuzzy inference system (ANFIS) is used to classify breast masses from digital mammograms that have also been processed with various enhancement and segmentation methods. A short review of related published work segmentation and classification follows below.

There are many CADx systems that involve several processing steps and different methodologies for enhancement, segmentation, feature selection and classification. Previous works have accomplished great results, but in real testing data, there is always the question of whether they will be able to carry on. Therefore, the methods that are used on systems that detect pathologies and help the physician perform their work more efficiently have to become more advanced to overcome the demands that new images may present.

This work focuses on the HIS ANFIS classification of digital mammographic masses that were previously enhanced by advanced fuzzy sets, as reported in our previous work [3]. K-means and Fuzzy C-means classifiers are also implemented for comparison purposes. The steps of the proposed classification algorithm are:

• Advanced fuzzy sets for image enhancement.
• Fused images based on OWA operators.
• Global, Otsu and type-2 fuzzy sets for thresholding.
• Feature extraction.
• Feature selection.
• Classification using K-means, Fuzzy C-means and ANFIS.

Image enhancement contains fuzzy methods in order to enhance the image contrast. Five types of fuzzy sets have been used: linguistic hedges; a fuzzy enhancement function; and advanced fuzzy sets that are intuitionistic, Pythagorean and Fermatean fuzzy sets [4]. Two types of OWA operators have been used, with the target being the fusion of the fuzzy techniques and the creation of an image that would provide better results.

Thresholding techniques are used so the enhanced image can undergo segmentation. Global and Otsu thresholding are classic methods for image segmentation [5]. Fuzzy sets type-2 is a thresholding method where a membership function is used, and the threshold index is extracted out of the ultrafuzziness of the image.

The features that are extracted are based on the geometry of the thresholded image, the texture of the initial image and the fuzzy features from the enhanced fuzzified image.

Classification is performed with the classic method of K-means and additionally more complicated methods of Fuzzy C-means (FCM) [6,7] and ANFIS technique that combines neural network and fuzzy inference engine methods.

The novelty and contribution of this work rely on the combination of various elements for the optimum classification of mammographic masses. The proposed methods include:

• An adaptive neuro-fuzzy inference system (ANFIS) for classification.
• A hybrid classification procedure of mammographic masses based on FCM/K-means clustering algorithms using a decision tree.
• Advanced fuzzy sets for contrast enhancement.

In addition, the proposed methods are trained and tested on masses detected with FFDM, which is the current state-of-the-art in mammography. FFDM images differ significantly from digitized screen-film mammography (SFM) images and require either modified or new CADx approaches.

The study is structured as follows: Section 2 contains the mathematical formulas of fuzzy image enhancement, mammography image segmentation, feature extraction and classification. It also contains a description of the training and testing datasets. Section 3 presents the results of this study, followed by the discussion in Section 4 and conclusions in Section 5.

Related Works

In papers [8,9,10], the authors analyze, evaluate and review existing artificial intelligence-based classification and segmentation techniques proposed in the last years for early detection and diagnosis of breast cancer using medical images. Yazdanbakhsh et al. [11] created a deep neuro-fuzzy network based on definitions of Takagi–Sugeno–Kang and tested the classifier on three datasets. The highest accuracy that they achieved was 99.58%. Gao et al. [12] represented a review of deep and machine learning techniques for mammography, assuming that the most frequent supervised methods for classification are artificial neural networks (ANNs), support vector machine (SVM) and random forest (RF). The most frequent unsupervised methods are the clustering algorithms of K-means, principal component analysis (PCA) and singular value decomposition (SVD). In deep learning, convolutional neural networks (CNNs) are one of the most used algorithms. Currently, computers that try to simulate the human brain are far from the learning capabilities of it. Almutairi et al. [13] proposed the deep reinforcement learning-based Deep Q learning (DQL) method as a classifier and the gorilla troops optimization (GTO) algorithm for feature selection. The method GTO-DQL was tested on three datasets with an accuracy up to 99.02%. Malebary et al. [14] proposed an algorithm named breast mass classification system (BMC) that is based on K-means, long short-term memory network of recurrent neural network (RNN), CNN and random forest. The proposed BMC system was tested on DDSM and MIAS dataset. For DDSM, the accuracy and AUC were 96% and 94–97%, respectively. For MIAS, the accuracy and AUC were 95% and 94–98%, respectively. Li et al., in [15], introduced the Dual Core Net (DCN), which computes mass segmentation and classifies simultaneously. The proposed algorithm was tested on DDSM and INbreast datasets, achieving 92.27% DI coefficient and 85% AUC for DDSM and 93.69% DI coefficient and 93% AUC for INbreast dataset. Parah et al., in [16], proposed a procedure for breast cancer masses that includes the Watershed segmentation and a hybrid machine learning classifier that is based on multi-layer perceptron (MLP), J48 and K-means. The methods were tested on the database from the UCI Machine Learning Repository and achieved an accuracy of over 88%. A paper by Sarvestani et al. focused on the decision tree classification algorithm [17]. In [18], a novel mass detection process that includes enhancement, characterization and classification was described. The classification mechanism is based on Bayesian regularization back-propagation networks and ANFIS techniques. Hosseini et al. [19] present a review of the application of the adaptive neuro-fuzzy inference system as a classifier in medical image classification during the past 16 years.

The most common deep networks that are used for image processing are convolutional neural networks, recurrent networks, and generative adversarial models [20]. The authors [21] proposed a novel method of an ensemble classifier set and depthwise separable convolution to extract shallow features. Abdullah et al. [22] represented and developed a convolutional neural network model for brain tumor segmentation. Abdel Rahman et al. [23] introduced a modified version of InceptionV3 and ResNet50 classifiers that are similar to CNN and tested the procedure on the DDSM dataset. They managed to obtain an 85.7% accuracy. Salama et al. [24] tested segmentation and classification methods in order to identify masses as malignant or benign. Fully, CNNs were introduced. The technique where applying data augmentation with a modified U-Net model and classifying the data with the InceptionV3 method achieved 98.87% accuracy and 98.88% AUC. Nemade et al. [25] represented two deep learning-based ensemble models. They used the VGG16, InceptionV3 and VGG19 as base classifiers and the two ensemble models were trained. The first ensemble model used a linear meta-learner in form of logistic regression and the second ensemble model used a neural net as the meta-learner for classification. The second ensemble model achieved 98.1% accuracy.

There is significant published work on the segmentation and classification of mammographic images. The fuzzy sets are notably useful and often used to address uncertainty issues and quantify uncertainty. Lin et al. [26] proposed a fuzzy-based novel quality control algorithm for the segmentation of medical images. Salih et al. [27] presented an algorithm based on fuzzy set methods for segmenting the boundaries of breast masses. Chaira T. [28] refers to thresholding techniques in general and proposes a method where intuitionistic fuzzy sets can be used as a segmentation method for images that contain blood vessels and blood cells. Tahoun et al. [29] used the BGWO feature selection technique and analyzed the percentage of features that the algorithm could reduce in order to achieve a high ROC area. The researchers achieved an accuracy of 78% and AUC 0.871 for the classification as benign or malignant. Catro-Tapia et al. [30] analyzed 14 classifiers of deep learning methods that were categorized into five classes. Their study showed that the CNN GoogLeNet classifier achieved the best results with an accuracy of 91.92% and AUC of 99.29%. Mobark et al. [31] proposed a CNN model, namely CoroNet, to perform automated breast cancer detection. It attained an accuracy of 94.92% on a four-class classification (benign/malignant mass and calcification). Alshehri et al. [32] studied CNN with and without attention mechanisms (AMs). The implementation of a deep learning algorithm on infrared thermal breast images showed an accuracy of over 99% with AMs and 92.32% without. Du et al. [33] presented a procedure that classifies architectural distortion, a feature of breast cancer. Enhancing the images with top–bottom hat and exponential transformation, reducing the noise with NSCT and finding a threshold for the segmentation of the image using the improved PCNN achieved an accuracy of 93.16% and an AUC of 0.93.

2. Materials and Methods

2.1. Fuzzy Image Enhancement

Image enhancement methods were previously developed and evaluated based on fuzzy sets and the fusion of them on mammograms [3]. They provided the starting point for the development of a fully automated pattern recognition and classification algorithm. The enhancement methods were linguistic hedge concentration (LH-CON); a linguistic hedge that was proposed by Xie, Wang and Wu (LH-XWW); and intuitionistic, Pythagorean and formation fuzzy sets (IFS, PFS and FFS). The methods created images that were fused with ordered weighted average (OWA) operators, OWA with genetic algorithm (OWA-GA) and OWA where Hong and Kim (OWA-HK) presented a way of finding weights. Genetic algorithms were used in order to optimize parameters for LH-XWW, IFS, PFS, FFS, and OWA-GA.

2.2. Advanced Fuzzy Sets and OWA Fusion Images

LH-CON at fuzzy sets creates a linear correlation between the input and the enhanced image. LH-XWW also has a linear correlation, but the output is different for every image. The use of a genetic algorithm allowed for the adaptive parameter tuning of each image. Advanced fuzzy sets such as IFS, PFS and FFS gain attention because of their non-linear character. They also use genetic algorithms to find the optimal values for parameters that are not constant. The five new images that were created by the advanced fuzzy sets are fused by OWA operators. The weights of OWA operators are obtained with two methods. The first method uses a genetic algorithm (OWA-GA), and the second uses a function that Hong and Kim proposed (OWA-HK). The OWA aggregation method is applied at every pixel of the image.

2.3. Mammography Image Segmentation

The grey level thresholding plays a vital role in pattern recognition in medical images. It follows the process of enhancement, and its output improves feature extraction for the subsequent pattern recognition techniques. Three types of thresholding methods were used in this work: global, Otsu and type-2 fuzzy sets thresholding.

2.3.1. Segmentation via Global Thresholding

The technique of global thresholding is a common method that does not need many computational resources. The algorithm aims to find the optimal threshold value by searching in the spatial domain. An iterative process is followed that analyzes the parameters of the enhanced image and finds the global threshold. Parameter λ is a pre-defined constant [34]. The iterative process is:

• Initializing the threshold value using the function $T=0.5·\left(I_{max}+I_{min}\right)$, where $I_{max}$ and $I_{min}$ are the maximum and minimum values of the image.
• Thresholding the image with the value $T$. This way, the pixels are separated into two groups: $I_1$, which contains the pixels with values $\ge T$, and $I_2$, which contains the pixels with values $<T$.
• Finding the average $m_1$ and $m_2$ from the groups $I_1$ and $I_2$ that were calculated previously.
• Finding a new threshold value using the function $T^{\prime }=0.5\left(m_1+m_2\right)$ to compare it with the previous thresholding value.
• If $\left|T-T{}^{\prime }\right|>\lambda$, then the steps 2 to 4 must be repeated. Otherwise, the threshold value is $T^{\prime }$ [34].

2.3.2. Otsu Thresholding

• A classic method for thresholding an image is Otsu’s method. Otsu threshold is calculated in order to minimize the overlapping of class distributions. Otsu threshold is found when the minimum entropy of the sum of background and foreground is calculated [35,36]. For our work, we used the command of MATLAB “graythresh” to find the threshold value and “imbinarize” to create the thresholded image with the threshold value [37].

2.3.3. Interval Type-2 Fuzzy Sets for Thresholding

Another approach to image segmentation is using the type-2 fuzzy sets method. This algorithm also has an iterative procedure. Before the beginning of the iterative procedure, the Tizhoosh membership function must be applied to the enhanced image. The Tizhoosh function is:

$$\mu (g)=\left\{\begin{array}{cc}0,& g\le g_{min} or g\ge g_{max}\\ L\left(g\right)={\left(\frac{g-g_{min}}{T-g_{min}}\right)}^{\alpha },& g_{min}<g\le T\\ R\left(g\right)={\left(\frac{g_{max}-g}{g_{min}-T}\right)}^{\beta },& T<g<g_{max}\end{array}\right.$$

(1)

The min and max indexes are used in the scanning procedure, which is described next [38].

The next step is to find two new membership functions for each pixel. There are two ways of creating these functions.

The first way is:

${\mu }_U\left(x\right)={\left[\mu \left(x\right)\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\alpha $}\right.}$ and ${\mu }_L\left(x\right)={\left[\mu \left(x\right)\right]}^{\alpha }$.

The second is:

${\mu }_U\left(x\right)={\left[\mu \left(x\right)\right]}^{1-\alpha }$ and ${\mu }_L\left(x\right)={\left[\mu \left(x\right)\right]}^{1+\alpha }$.

That means:

${\mu }_U\left(x\right)={\left[\mu \left(x\right)\right]}^{0.5}$ and ${\mu }_L\left(x\right)={\left[\mu \left(x\right)\right]}^2$

or

${\mu }_U\left(x\right)={\left[\mu \left(x\right)\right]}^{0.75}$ and ${\mu }_L\left(x\right)={\left[\mu \left(x\right)\right]}^{1.25}$.

In this paper, the first way was used, but with different exponents:

${\mu }_U\left(x\right)={\left[\mu \left(x\right)\right]}^{0.25}$ and ${\mu }_L\left(x\right)={\left[\mu \left(x\right)\right]}^4$ [38].

The lower and upper functions can be seen in Figure 1.

The next action is to extract the ultrafuzziness of the system. The ultrafuzziness index acts as a means of comparison to find the optimal threshold value. The ultrafuzziness of the system is extracted for the spatial domain by the function [38]:

$$\tilde{\gamma }\left(\tilde{A}\right)=\frac{1}{MN}{\sum }_{i=1}^{M-1}{\sum }_{j=1}^{N-1}\left[{\mu }_U\left(g_{ij}\right)-{\mu }_L\left(g_{ij}\right)\right]$$

(2)

and using the histogram of the image by the function:

$$\tilde{\gamma }\left(\tilde{A}\right)=\frac{1}{MN}{\sum }_{g=0}^{L-1}h\left(g\right)\times \left[{\mu }_U\left(g\right)-{\mu }_L\left(g\right)\right]$$

(3)

where $M$ and $N$ are the rows and columns of the image, and $h$ is the histogram of the image.

The general structure of the algorithm of type-2 fuzzy sets and the calculation of the ultrafuzziness includes the following:

• Selection of the membership function. In this paper, it was the Tizhoosh function. It can also be used as a sigmoid membership function.
• Calculation of the histogram if the histogram ultrafuzziness (3) is used.
• Initialize membership functions start.
• Scanning the function across the entire grayscale of the image.
• Finding the functions ${\mu }_U\left(x\right)$ and ${\mu }_L\left(x\right)$ for every step of the scanning procedure.
• Finding maximum ultrafuzziness and the optimal $T$.
• Thresholding the image using $T$ of the maximum ultrafuzziness.
• Finally, defuzzification of the result using the function $x_{mn}^{\prime }={\mu }_{mn}^{\prime }·\max value$ [38].

The thresholding methods that were used can change the optimal threshold value in every image. In Figure 2, we present the thresholded images that the thresholding methods created from an ROI of a mammogram.

2.3.4. Flood Fill Operation

A thresholded image may present several areas that are above the threshold value, as shown in Figure 3. MATLAB commands “regionprops”, “ismember” and “imfill” were used to remove unconnected regions and fill the voids to extract a solid region with the largest area [37].

In Figure 4, we can see a general overview of the segmentation process.

2.4. Feature Extraction

A classification system aims to achieve optimum classification by extracting the best statistical scores. The methods that were described create images that are used for feature extraction. The features of an image show different perspectives of it. In this paper, features were extracted that describe the geometry, the texture and the fuzziness of the masses. Nine features were extracted, two of which were geometric, four were texture and three were fuzzy features. The geometric features were the circularity and the minimum to maximum axial distance of the thresholded mass. The texture features were the contrast between the mass and the background, homogeneity, correlation and energy of the thresholded mass. The fuzzy features were the area, the perimeter and compactness.

2.4.1. Geometric Features

MATLAB command “regionprops” was used to extract the circularity of the minor and major axial distances [37]. Circularity indicates how round a mass is and how close it is to an ideal circle. If the index is close to one, the more circular it is. If it is close to zero, the more irregular it is. The minor and major axial distances were merged into one feature by dividing the minor by the major axial distance as a way to represent the elongation of a mass. The lower the value of this index, the more elongated it is, as the minimum axial distance is small, and the maximum axial distance is bigger.

2.4.2. Texture Features

Image Masks

The method of extracting masks from the images is helpful for the next feature extraction techniques. With the thresholding techniques and the creation of masks, matrices were created that contain exclusively the mass and exclusively the background. The matrix that contained the mass was created by inserting the value of the initial image if the pixel was above the threshold value, and the value of the initial image was ignored if it was below the threshold value. The opposite procedure was followed in order to create the background matrix.

GLCM Matrix

The process of extracting texture features is performed indirectly with the image. These features need a gray-level co-occurrence Matrix (GLCM) to be extracted. A GLCM matrix has lines and columns equal to the gray levels of the image. That means if an image has 256 gray levels, the dimensions of the GLCM matrix will be $256\times 256$. Each element of the table represents the combination of the row and the column of it. A GLCM matrix usually has large dimensions, and this is why much temporary data are stored, i.e., a $256\times 256$ for each image. Such a matrix is very sensitive to extracting texture features. The gray levels of an image are usually decreased because of the large dimensions and sensitivity of a GLCM matrix. The composition of a GLCM matrix is described in Figure 5 for four gray levels. By scanning, from left to right, the image from which the GLCM matrix is generated, which means a degree of 0°, the upper left element of the GLCM matrix fills with the number of times that the combination 0,0 occurs. This indicates how many times a gray level 0 pixel (neighbor pixel) is to the right of another gray level 0 pixel (reference pixel). If the degree of creating the GLCM matrix is different, i.e., 45°, then the upper left element of the GLCM matrix will fill with the number of times that the combination 0,0 occurs; however, it would indicate how many times a pixel with gray level 0 (neighbor pixel) is diagonally at the upper right of another pixel with gray level 0 (reference pixel) [39].

The MATLAB command “graycomatrix” was used for the creation of the GLCM matrix with a degree of 0° [37].

Extraction of Texture Features

The features that were extracted with the help of the GLCM matrix were homogeneity, correlation and energy of the mass. The application of the GLCM matrix was performed on the mass mask of every image. For the contrast between the mass and the background, the GLCM matrix was not used [40]. The MATLAB command “graycoprops” may be applied for the extraction of homogeneity, correlation and energy [37]. Alternatively, the following functions may be used [41].

For homogeneity, the function is:

$$homogeneity=\sum _{i,j=1}^n\frac{p_{ij}}{1+{(i-j)}^2}$$

(4)

where $p_{ij}$ is the element of the GLCM matrix at row I and column j, and n is the maximum value of the gray levels.

For correlation:

$$correlation=\sum _{i,j=1}^n{p}_{ij}\frac{(i-{\mu }_i)(j-{\mu }_j)}{{{\sigma }_i}^2}$$

(5)

where ${\mu }_k=\sum _{ij}k·p_{ij}$ and ${\sigma }_k^2=\sum _{ij}{p}_{ij}{\left(i-{\mu }_k\right)}^2$.

For energy:

$$energy=\sum _{i=1}^n\sum _{j=1}^n{p_{ij}}^2$$

(6)

The extraction of the contrast between the mass and the background is performed with the help of the masks and without using the GLCM matrix. It was based on Weber’s law, where the ratio of the background and interest point difference intensity to the background intensity is constant [42]. The function is as follows [43]:

$$contrast=\frac{I_b-I_f}{I_b}$$

(7)

where $I_b$ is the intensity of the background, and $I_f$ is the intensity of the interest point.

For the intensity of the background and the interest point to be found, the background and interest point mask were used, respectively. The average value of the background mask was defined as the background intensity, and the average pixel value of the interest point mask was defined as the intensity of the interest point.

2.4.3. Fuzzy Features

The extraction of the fuzzy features was based on the fuzzified version of the enhanced images. The thresholding technique was useless for this kind of feature. The functions have a fuzzy matrix as an input and a value as an output. The features that were extracted during this method are the area, the perimeter and the compactness [44].

For the fuzzy area, the function is:

$$fuzzy Area\left(\mu \right)=\sum _{m=1}^M\sum _{n=1}^N{\mu }_{m,n}$$

(8)

where the matrix has dimensions $M\times N$ and ${\mu }_{m,n}$ is membership participation of each pixel of the image.

For the fuzzy perimeter:

$$fuzzy Perimeter\left(\mu \right)=\sum _{m=1}^M\sum _{n=1}^{N-1}‖{\mu }_{m,n}-{\mu }_{m,n+1}‖+\sum _{m=1}^{M-1}\sum _{n=1}^N‖{\mu }_{m,n}-{\mu }_{m+1,n}‖$$

(9)

For the compactness of the image, the function was proposed by Rosenfeld:

$$compactness\left(\mu \right)=\frac{fuzzy Area\left(\mu \right)}{{\left[fuzzy Perimeter\left(\mu \right)\right]}^2}$$

(10)

Compactness obtains its maximum value when the interest area is a circle [45].

2.5. Data Normalization and Feature Selection

2.5.1. Normalization

The procedure of feature extraction is performed for all the mammograms, creating a matrix that contains all of them: the type of enhancement, the type of thresholding and an index for each image if it contains a malignant or a benign mass. If the mass is malignant, then the index is “1”, and if the mass is benign, then the index is “0”.

The normalization method is a necessary procedure and must be performed before the data are inserted into the classifier because the range of the features’ values may vary significantly. For example, one feature may have values ranging from 0 to 1, while another feature may have values from 0 to 5000. A normalization method maps all features to the same range, either from 0 to 1 or from −1 to 1. The value of 1 maps the maximum feature value, while 0 or −1 maps the minimum value. MATLAB’s command “normalize” was used to map all features’ values from 0 to 1 [37].

The normalization can also be performed with the following function [46]:

$${x^{\prime }}_i=\frac{x_i-x_{min}}{x_{max}-x_{min}}$$

(11)

where $x_{min}$ and $x_{max}$ are the minimum and maximum value of a feature, respectively; $x_i$ is the i-th value from the total values that were extracted from the images; and ${x^{\prime }}_i$ is the new normalized value.

2.5.2. Feature Selection

The feature selection and ranking for the training of an automated decision system is a crucial and challenging process [39,40]. Feature selection methods are applied to different fields, including the pharmaceutical and oil industry, voice and pattern recognition, biotechnology, etc. [47]. The aim of these methods is to reduce the features that are inserted into the classifier by keeping out unnecessary ones and, by extension, decreasing noise in multidimensional data. A small number of features allows for better visualization and understanding of the data, fewer operations and storage requirements, and more efficient training and testing processes. Overtraining of the classifier can also be avoided while improving its performance, giving faster results and a better understanding of the training process. An exhaustive search was used in this study for feature selection. This method investigates all feature combinations and yields optimum results [48]. The relatively small dataset of our pilot study allowed for the application of the method that is usually computationally intensive and almost impractical to use with a large number of features and large datasets [41,42].

2.6. Classification

2.6.1. K-Means

Classification with K-means clustering is widely used because of its simplicity to understand and apply. We initialize how many clusters we want to have (k clusters), and the algorithm starts with random k clusters. Then, k centers are chosen from the clusters that are the centroids of them. New clusters are created according to the distance of the data from the centers. This procedure is repeated until the centers do not change [49,50]. Many researchers have applied clustering techniques to solve medical image segmentation problems. K-means is a well-known clustering algorithm that is widely applied for segmenting and classifying medical images. The training data create the centers that the testing data use to classify the masses [51,52,53]. MATLAB’s command “kmeans” was applied to create clusters and “pdist2” for the test data [37]. Two groups are created (“G1” and “G2” in Figure 6) with labels “1” and “2”, without knowing which is malignant or benign. Two truth tables are created, the first using the labeled data with “1” as malignant and the second truth table as “2” as malignant. Two accuracies are extracted (“Acc1” and “Acc2” in Figure 6). We use the truth table of the accuracy that is greater than the other.

2.6.2. Fuzzy C-Means

Since fuzzy sets were introduced, the concept of data having a membership grade to different membership functions has been used frequently. At the FCM algorithm, clusters are represented as the membership functions, and data points are assigned to several clusters. The membership degree to an input belonging to a particular cluster is evaluated. In the end, a data point consists of multiple membership functions, and the objective function is computed by the Euclidian distance [54,55]. Fuzzy C-means is a well-known clustering algorithm that is widely applied for segmenting and classifying medical images [56]. MATLAB commands were used to create the FCM classifier. Commands “fcm” and “genfis” were used to create the clusters [37]. The command “evalfis” was used for the test data [37]. Due to the nature of the method, the test data are labeled from a range (in Figure 7, the labels are extracted as “Y”). The ROC curve is extracted, and the optimum threshold is found (in Figure 7, T is the threshold). If a label of data is greater than the threshold, Y = 1; otherwise, Y = 0. As conducted at the K-means method, two groups are created (“G1” and “G2” in Figure 7) with labels “0” and “1”, without knowing which is the malignant or benign. Two truth tables are created, the first using the labeled data with “0” as malignant and the second truth table with “1” as malignant. Two accuracies are extracted (“Acc1” and “Acc2” in Figure 7). We use the truth table of the accuracy that is greater than the other.

2.6.3. ANFIS

ANFIS is a hybrid intelligent system that includes a TSK (Tagaki–Sugeno–Kang) model with a multi-layer neural network structure. The parameters of the TSK systems are determined by learning methodologies. Learning methodology (back-propagation method) automatically defines the parameters of fuzzy sets, such as membership functions and rules, without an operator’s intervention. The difference between ANFIS and neural networks is that the neural networks are created by training while ANFIS is formed by fuzzy rules, and then the learning method optimizes the membership functions and rule parameters [19,40,41]. Although ANFIS shows that it has a problem with the curse of dimensionality, it has a significant improvement in classification accuracy [57].

ANFIS is based on the Sugeno fuzzy model, and the structure can be represented by the following first-order rules:

$$\begin{gathered} Rule 1:If \left(x is A_1\right) and \left(y is B_1\right) \\ Then f_1=p_{1}x+q_{1}y+r_1 \\ Rule 2:If \left(x is A_2\right) and \left(y is B_2\right) \\ Then f_2=p_{2}x+q_{2}y+r_2 \end{gathered}$$

(12)

where $x$ and $y$ are the inputs; $A_i$ and $B_i$ are the fuzzy sets; and $f_i$ are the outputs set by the fuzzy rules with parameters $p_i$, $q_i$ and $r_i$.

The parameters of fuzzy sets and the $p_i$, $q_i$ and $r_i$ are those determined by the learning process. The learning method is hybrid as it uses the least squares technique and back propagation. The process is performed in two passes. The first pass is the forward direction; the fuzzy set parameters are kept constant while the parameters $p_i$, $q_i$ and $r_i$ are determined using the least squares technique. The output y of ANFIS is extracted by using the new rule values. The desired output with the new output has an error, which is calculated by subtracting them from each other. The next pass is the back propagation, where the parameters of the fuzzy sets are updated. The parameters of the rules remain constant. Figure 8 shows an ANFIS system with three inputs, two fuzzy rules at each input and one output [41,58]. MATLAB’s command “anfis” was used for the training process and “evalfis” for the testing process [37].

The general procedure is shown in Figure 9.

2.7. Evaluation Metrics

For the evaluation of the classifiers, five different performance measures are applied. The measures are the accuracy, sensitivity, specificity, F1 score and AUC of the results. The measures are known by finding the number of true/false positive and true/false negative outputs:

• True positive (TP) denotes the malignancy correctly predicted as malignancy.
• True negative (TN) denotes the benignity correctly predicted as benignity.
• False positive (FP) denotes the benignity mistakenly predicted as malignancy.
• False negative (FN) denotes the malignancy mistakenly predicted as benignity.

The measures are defined as:

• $\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}\times 100\mathrm{\%}$
• $\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}}\times 100\mathrm{\%}$
• $\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}\times 100\mathrm{\%}$
• $\mathrm{F}1 \mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{2\times \mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}\times 100\mathrm{\%}$

Sensitivity is the ability of the classifier to detect that malignancy is present, given that malignancy is present.

Specificity is the ability of the classifier to detect that benignity exists, given that benignity exists.

Accuracy is the ability of the classifier to correctly detect benignity and malignancy.

The criterion to find the best combination of the features is to find the highest accuracy of the system. Once the highest accuracy is found, then the sensitivity and specificity are calculated [41,59].

For the classifiers FCM and ANFIS, ROC curves were extracted, helping to extract the area under the ROC curve (AUC) that is an objective index [33]. AUC has values between 0 and 1. The closer to 1, the better the classifier. ROC curve can enhance the optimum performance of a classifier only when the classifier has as an output a range of numbers. ROC curve tests different thresholds for classifying the range of the outputs and suggests the best threshold value. The K-means method has as an output only two values if the clusters are two, and the ROC curve cannot be applied to it.

The F1 score also provides an objective evaluation of a classifier [30,33].

2.8. Mammography Images (Dataset)

Direct digital mammograms with 97 masses, 65 benign and 32 malignant, were used for training and testing the classifier. This is an in-house FFDM database, the images of which were acquired with a Senographe 2000D or a Senographe Essential FFDM system (GE Healthcare, Chicago, IL, USA). The for-presentation images were used in this study that are either 1914 × 2294 or 2394 × 3062 pixels with 16 bits per pixel (12 bits were allocated). All but nine of the masses came from different mammograms. Nine of the masses were observed on both the mediolateral oblique and the craniocaudal view of the same mammogram and were considered 18 independent findings. Regions of interest (ROIs) were extracted from the full mammograms based on the ground truth files defined by expert mammographers. Figure 10 shows a representative example of ROI extraction. ROIs varied in size because each one was adapted to the mass size so as to encompass the mass and the local background (red circle in Figure 10).

The FFDM dataset was provided by Dr. Maria Kallergi and the Smart Hospital Research Laboratory of the Department of Biomedical Engineering of the University of West Attica. All cases were confirmed by biopsy or annual follow-up for at least three years. This is a secondary use of unidentified and non-coded data, and the work does not constitute research with human subjects because there was no interaction with any individual, and no identifiable private information was used. This work was IRB-exempt.

3. Results

To extract the results of the methods, we used computer resources with the following characteristics: Dell PE 350, CPU: Intel Xeon E-2314 2.8 GHz, 8 M Cache, DELL Memory 16 GB—DDR4 UDIMM 3200 MHz, Windows server 2019 standard edition 16 Cores 2 VMs. At the ROIs of the mammograms, median filtering and the methods that were proposed in our previous work were performed. The methods provided seven enhanced images, LH-CON, LH-XWW, IFS, PFS, FFS, OWA-GA and OWA-HK, with one initial ROI mammography as an origin [3].

Furthermore, in this paper, we performed thresholding, feature extraction, feature selection and classification of the mammograms. Each enhanced image is thresholded by three different methods of thresholding. Global, Otsu and type-2 fuzzy sets thresholding provide a total of twenty-one thresholded images. The images that we created with the initial images will provide the features that we need for the classifier. Geometrical, texture and fuzzy features are extracted according to the feature extraction section of this paper. This procedure happens to all the mammograms of the dataset. At the end of this step, there will be three three-dimension matrixes. The first matrix has rows of the ninety-seven mammograms; c columns of the nine features that are extracted from each mammography; and, as the depth of the matrix, the seven enhancement methods. The first three-dimensional matrix is about the features that are extracted with the method of global thresholding, the second three-dimensional matrix is about the method of Otsu thresholding and the third three-dimensional matrix is about the method of type-2 fuzzy sets thresholding. The data of the three matrixed are normalized at this stage. The normalization is performed on each feature of the mammograms. The feature selection is the exhaustive search where all the combinations of the features are tested with the K-means, FCM and ANFIS classifiers so the best combination can be found.

K-means, FCM and ANFIS classifiers have two up to nine inputs that depend on the combination that the exhaustive search gives. For the ANFIS classifier, we used two membership functions at the input, type “gbell”, and the rules are constant.

The dataset was divided into training and testing sets. The training set was 70% of the data, and the testing set was 30%. The classifiers were trained first with the training set, and the performance indicators were recorded from the testing set. The sets were randomly selected. The performance of the classifiers was tested based on the accuracy. The classification procedure was performed ten times so it can be statistically more accurate, and the results for each classifier are the average of the ten results. The following

Table 1. Indicators for global thresholding method (The best scores are bold).

Contrast Methods		Accuracy %	Sensitivity %	Specificity %	F1 Score %	AUC
LH-CON	K-means	83.79	61	91.46	64.61	-
	FCM	93.79	76.84	98.3	82	0.86
	ANFIS	92.07	71.12	98.68	79.91	0.83
LH-XWW	K-means	87.59	74.94	92.73	77.2	-
	FCM	94.14	79.44	98.21	85.27	0.84
	ANFIS	93.45	84.19	96.37	85.47	0.88
IFS	K-means	81.38	76.54	83.1	70.58	-
	FCM	92.41	78.77	96.88	82.9	0.87
	ANFIS	92.07	71.5	96.78	78.46	0.81
PFS	K-means	80.34	67.34	85.45	65.26	-
	FCM	92.41	72.9	96.98	77.87	0.8
	ANFIS	93.45	82.6	97.22	86.33	0.86
FFS	K-means	86.9	77.68	90.11	77.39	-
	FCM	95.17	83.35	98.2	86.98	0.91
	ANFIS	93.79	80.61	97.75	83.75	0.85
OWA-GA	K-means	83.79	77.72	85.05	70.61	-
	FCM	91.38	74.14	96.04	77.4	0.82
	ANFIS	92.07	64.39	98.77	74.76	0.75
OWA-HK	K-means	84.83	69.36	90.77	71.39	-
	FCM	92.76	73.98	97	77.83	0.81
	ANFIS	92.41	63.95	99.07	75.57	0.76

,

Table 2. Indicators for Otsu thresholding method (The best scores are bold).

Contrast Methods		Accuracy %	Sensitivity %	Specificity %	F1 Score %	AUC
LH-CON	K-means	86.55	79.03	88.7	75.78	-
	FCM	92.76	82.89	96	83.89	0.9
	ANFIS	91.03	61.53	98.21	72.17	0.73
LH-XWW	K-means	87.24	66.92	93.11	69.24	-
	FCM	95.17	88.41	97.32	89.42	0.91
	ANFIS	90.69	72.34	96.02	76.09	0.8
IFS	K-means	80.34	65.31	85.01	63.76	-
	FCM	92.76	73.79	97.81	79.39	0.83
	ANFIS	91.03	67.63	96.43	73.76	0.77
PFS	K-means	81.38	70.09	83.74	64.25	-
	FCM	92.76	73.96	97.01	79.74	0.79
	ANFIS	93.1	80.26	97.59	85.98	0.86
FFS	K-means	80.69	74.41	83.59	65.43	-
	FCM	91.38	74.06	97.07	81.12	0.8
	ANFIS	92.07	69.26	97.82	76.77	0.75
OWA-GA	K-means	81.03	82.45	79.64	73.22	-
	FCM	92.07	79.83	95.9	82.29	0.87
	ANFIS	92.41	64.79	98.71	75.39	0.77
OWA-HK	K-means	78.62	43.31	89.66	45.84	-
	FCM	92.76	74.82	97.3	81.64	0.84
	ANFIS	92.41	70.25	98.67	79.83	0.83

and

Table 3. Indicators for type-2 fuzzy sets thresholding method (The best scores are bold).

Contrast Methods		Accuracy %	Sensitivity %	Specificity %	F1 Score %	AUC
LH-CON	K-means	86.55	63.68	93.15	64.81	-
	FCM	92.41	73.73	97.29	79.96	0.84
	ANFIS	90.34	72.07	95.28	76.8	0.82
LH-XWW	K-means	89.31	70.05	96.37	76.44	-
	FCM	94.14	83.54	96.99	83.67	0.89
	ANFIS	91.72	79.39	95.35	81.34	0.86
IFS	K-means	82.41	70.71	87.08	65.66	-
	FCM	91.72	69.89	97.29	77.46	0.81
	ANFIS	90.69	67.89	95.88	73.19	0.75
PFS	K-means	82.41	76.27	85.51	72.46	-
	FCM	93.45	73.59	97.87	79.84	0.82
	ANFIS	92.07	82.52	95.26	83.71	0.86
FFS	K-means	85.86	73.26	91.05	74.87	-
	FCM	91.72	81.19	94.69	82.36	0.84
	ANFIS	92.07	75.15	96.21	76.49	0.83
OWA-GA	K-means	85.17	83.97	84.98	77.77	-
	FCM	92.76	84.71	94.51	82.83	0.87
	ANFIS	90.69	70.96	95.74	73.54	0.79
OWA-HK	K-means	85.17	70.63	91.07	69.6	-
	FCM	93.1	84.79	96.3	86.22	0.88
	ANFIS	90.69	66.23	98.26	76.48	0.76

show the results. Table 1 shows the results of the global thresholding method, Table 2 shows the results of Otsu thresholding and Table 3 shows the type-2 fuzzy sets thresholding method. For each line, the first subline is the results of K-means clustering, the second subline of FCM and the third subline of ANFIS.

Based on Table 1, the enhancement method FFS with global thresholding and the FCM classifier scored the highest indicator values with an accuracy of 95.17%, F1 score of 86.98% and AUC of 0.91 (ROC curve in Figure 11a).

Based on Table 2, the enhancement method LH-XWW with Otsu thresholding and FCM classifier scored the highest indicator values with an accuracy of 95.17%, F1 score of 89.42% and AUC of 0.91 (ROC curve in Figure 11b).

Based on Table 3, the enhancement method LH-XWW with type-2 fuzzy sets thresholding and FCM classifier scored the highest indicator values with an accuracy of 94.14% and an AUC of 0.89 (ROC curve in Figure 11c). The best F1 score for type-2 fuzzy sets is achieved with the enhancement method OWA-HK and the classifier FCM, with a score of 86.22% (ROC curve in Figure 11d).

In general, the best scores were achieved by the FCM classifier. The highest accuracies and AUC were achieved by FFS enhancement, global thresholding, FCM classifier and LH-XWW enhancement, Otsu thresholding and FCM classifier, but the second combination reached a higher F1 score.

Advanced fuzzy sets (IFS, PFS and FFS) yielded satisfactory results, especially with global and Otsu thresholding methods.

4. Discussion

In this paper, we proposed the classification of breast masses as benign or malignant with an intelligent system that includes enhancement, thresholding, feature selection and classification with state-of-the-art methodologies [3]. The pilot evaluation was performed with 97 masses from FFDM for presentation images. Geometrical, texture and fuzzy features were computed for each mass ROI and used as inputs to the classifiers.

FCM and ANFIS classifiers achieved the best results compared to the K-means method. The best combination of methods overall was the LH-XWW enhancement with Otsu thresholding and FCM classifier, achieving an accuracy of 95.17%, F1 score of 89.42% and AUC of 0.91. The second best combination was the FFS enhancement with global thresholding and FCM classifier. The conventional K-means method was suggested as a simple method to classify the data but had the poorest results, leaving no room for future improvement. Otsu thresholding is usually considered a conventional method to threshold an image with average results. In the FFDM data of our study, however, it led to highly accurate classification when combined with the other proposed methodologies.

5. Conclusions

The benign/malignant classification of mammographic lesions depends on the radiologists’ experience and the limitations of mammography. FFDM brought significant improvements in breast imaging compared to SFM, particularly for dense breasts. However, it has not alleviated all issues. As a result, numerous biopsies are still performed annually for the diagnosis of breast cancer, and only about 20% of these biopsies yield a positive outcome [60]. The field of automatic detection of cancerous masses is a well-researched scientific topic for digitized SFM, but there is still room for improvement for FFDM.

This paper presented a pattern recognition methodology for FFDM data that included advanced fuzzy sets for contrast enhancement and a hybrid FCM/K-means clustering with a decision tree for classification. A pilot evaluation demonstrated promising performance, matching reports on digitized SFM at worst; classification accuracy was consistently above 90% independent of the enhancement or the thresholding approach.

Future work will include deep learning and the expansion of the digital image dataset for more efficient training and, furthermore, testing of the proposed intelligent system with additional features specific to the various mass types.