Positron Emission Tomography Image Segmentation Based on Atanassov’s Intuitionistic Fuzzy Sets

Abstract

In this paper, we present an approach to fully automate tumor delineation in positron emission tomography (PET) images. PET images play a major role in medicine for in vivo imaging in oncology (PET images are used to evaluate oncology patients, detecting emitted photons from a radiotracer localized in abnormal cells). PET image tumor delineation plays a vital role both in pre- and post-treatment stages. The low spatial resolution and high noise characteristics of PET images increase the challenge in PET image segmentation. Despite the difficulties and known limitations, several image segmentation approaches have been proposed. This paper introduces a new unsupervised approach to perform tumor delineation in PET images using Atanassov’s intuitionistic fuzzy sets (A-IFSs) and restricted dissimilarity functions. Moreover, the implementation of this methodology is presented and tested against other existing methodologies. The proposed algorithm increases the accuracy of tumor delineation in PET images, and the experimental results show that the proposed method outperformed all methods tested.

1. Introduction

Segmentation of digital images is the procedure of partitioning an image into disjoint parts, regions, classes, or subsets so that every part must fulfill an unmistakable and very characterized property and attribute. Image segmentation is an essential step towards the analysis of image information. Image segmentation plays an important role in digital image processing and is used in almost every field of science; for example, digital image processing, satellite imaging, computer vision, biometrics, medical images and other [1,2,3,4,5,6,7].

In this work, we use image segmentation to detect and delineate tumors in PET images.

Positron emission tomography, or PET, is a technique for the imaging of physiological processes in humans. This technique presents the distribution of a radioactive emitter monitored by surrounding detectors. With the aid of mathematical algorithms, the image is constructed based on the distribution of the marker. PET has become an indispensable tool to ensure more accurate treatment of patients and early diagnosis for the treatment of cancer [8,9,10,11,12,13,14,15,16,17]. PET is heavily used in medicine, biology, neurology, and pharmaceutical research of brain activity, blood, or glucose.

PET images are known for their high sensitivity and low spatial resolution. Furthermore, PET images have low signal-to-noise ratios and suffer from noise caused by random and scattered coincidences. With these conditions, the difficulties for a successful image segmentation increase [9,18,19,20,21,22,23,24,25]. The delineation of tumors in PET images is a crucial step because the determination of this boundary should be kept as small as possible to minimize damage to healthy tissue by future treatments, but the boundary must ensure the inclusion of the entire extent of the diseased tissue. Different methods have been proposed using several segmentation techniques [26,27,28,29,30]. Fixed threshold methods do not find a consensus value to correctly delineate the tumor [3,11]; adaptive threshold methods have better results, but most of the times the results depend on standardized uptake values (SUV) [20,31,32,33] or source-to-background ratio (SBR) [23,34,35] values, and these values are not always available; the iterative threshold methods depend on knowing ground truth metrics or limiting the number of iterations, which can change the output values [23,36]. The best region-based methods are region of interest (ROI)- or seed-dependent, and edge-based methods do not have good results because of the faded image edges, caused by the noisy characteristic of PET images. Stochastic-based methods with very different approaches presented several results [22,37,38,39,40]. The use of deep learning-based methods is limited both by the problem’s nature (precision of the delineation) and the number of labeled samples needed. Nevertheless, some deep learning-based methods have also been recently proposed [41,42,43].

We will introduce a new approach to PET image segmentation, using an iterative thresholding method based on Atanassov’s intuitionistic fuzzy sets (A-IFSs). Hereupon, the presented algorithm finds and delineates tumors in PET images in a non-supervised way. The method is invariant to the images size, seeds position, region shape, and SUV or SBR values, resulting in a better and more efficient tumor delineation procedure.

2. Fuzzy Logic-Based Image Thresholding Using A-IFSs

Atanassov’s intuitionistic fuzzy sets (A-IFSs) have been successfully used to determine the optimal threshold value for gray-level image segmentation. Atanassov’s intuitionistic fuzzy index values are used for representing the hesitance of an expert on determining whether a pixel of the image belongs to the background or the object of the image [44] and we use these values to determine if the pixel belongs to a non-healthy tissue or a healthy tissue.

Melo-Pinto et al. [44] proposed the following membership functions to represent the relationship between each pixel to the background ${\tilde{Q}}_{Bt}$ or the object ${\tilde{Q}}_{Ot}$:

$${\mu }_{{\tilde{Q}}_{Bt}}=F\left(d\left(\frac{q}{L-1},\frac{m_B\left(t\right)}{L-1}\right)\right)$$

(1)

$${\mu }_{{\tilde{Q}}_{Ot}}=F\left(d\left(\frac{q}{L-1},\frac{m_O\left(t\right)}{L-1}\right)\right)$$

(2)

For each t, the mean of the intensities of the pixels that belong to the background $m_{Bt}$ and the mean of the intensities of gray of the pixels that belong to the object $m_{Ot}$ are given by the following expressions:

$h\left(q\right)$ being the number of pixels of the image with the intensity q, and the function $F\left(x\right)=1-0.5x$ and the restricted dissimilarity function $d(x,y)=|x-y|$, constructed from the automorphisms ${\phi }_1\left(x\right)={\phi }_2\left(x\right)=x$ for all $x\in [0,1]$, where $m_{Bt}$ and $m_{Ot}$ are represented as follows:

$$m_B\left(t\right)=\frac{{\sum }_{q = 0}^{t}qh\left(q\right)}{{\sum }_{q = 0}^{t}h\left(q\right)}$$

(3)

$$m_O\left(t\right)=\frac{{\sum }_{q=t+1}^{L-1}qh\left(q\right)}{{\sum }_{q=t+1}^{L-1}h\left(q\right)}$$

(4)

With the proposal to split the object from the background, it is essential to accurately determine the property that must be fulfilled by the pixels that belong to the object. This property establishes the form of the membership function associated with the set that represents the object. Usually, this property is not positively known, and the selection of membership function is conditioned by the missing knowledge/ignorance of the expert who constructs the membership function.

Considering that we need a multi-thresholding algorithm to refine the threshold selection in tumor delineation in PET images, we will use the following expression to calculate the Atanassov’s intuitionistic index $\pi$.

$$\pi \left(q\right)=\wedge \left(1-{\mu }_{{\tilde{Q}}_{Bt}}\left(q\right),1-{\mu }_{{\tilde{Q}}_{Ot}}\left(q\right)\right)$$

(5)

Atanassov’s intuitionistic index $\pi$ represents the missing knowledge/ignorance of the expert in determining the membership value of a specific pixel to the background or the object of the image.

3. PET Image Segmentation with Iterative Thresholding Using A-IFSs

In this section, we present a general A-IFSs-based multi-level image thresholding that, besides its low computational cost, autonomously determines the thresholds based on the image pixels’ gray-levels homogeneity. The method selects iteratively, using the images characteristic, the best threshold value to identify the damaged tissues in PET images.

In order to compute the threshold, the proposed algorithm is applied an unspecified number of times to the image using a divide and conquer strategy. First, the algorithm is applied to the original image Q with its pixels’ gray-levels $\in [0,L-1]$, determining the threshold value $t_i$ corresponding to the smallest entropy of the image Q. This threshold value is then used to create two sub-images: the sub-image with intensities lower than $t_i$ with its pixels’ gray-levels $\in [0,t_i-1]$, and the sub-image with intensities greater than $t_i$ with its pixels’ gray-levels $\in [t_i+1,L-1]$. Finally, the algorithm is applied to the sub-image, which has a higher amplitude between gray-level entropy values. Each sub-image processed is marked and cannot be processed again.

In Figure 1 we represent the computational process resulting from the application of the algorithm to a given image, employing a binary tree where each tree node contains the description of the images’ gray-scale and the threshold value obtained through the application of the algorithm to that image.

The proposed algorithm consecutively divides the resulting sub-images by means of the threshold value $t_i$ obtained through the application of the general algorithm to each one of them. To enhance the algorithm with the capability to self-stop the process of determining new thresholds and consequently sub-dividing the images, we use the region’s homogeneity introduced by the following algorithm.

A New Approach to Perform Tumor Delineation in PET Image with Iterative Thresholding Using A-IFSs

Tumor delineation in PET images is a crucial step because the boundary has to ensure the inclusion of all of the damaged cells, but if the boundary is too large, it will endanger healthy tissues during future treatments [11]. A-IFSs find the best threshold value to split the object from the background, and therefore to refine the threshold to match the small tumor region, we apply the A-IFSs to a selected sub-image until we obtain the pretended pixel gray-scale intensities’ homogeneity.

In our iterative threshold algorithm, we use the homogeneity (H) value of the sub-image to determine when the algorithm stops searching for another threshold. We define H as:

$$H=\sum _{i,j}\frac{p(i,j)}{1+\left|i-j\right|}$$

(6)

We also define the entropy amplitude C with a value that will determine which sub-image will be processed in the next iteration. To determine the value of C we need to sub-divide the image $Q_i$ into two sub-images $Q_{i+1}$ with gray-levels $q(x,y)\in [Lo{w}_i,t_i-1]$ and $Q_{i+2}$ with gray-levels $q(x,y)\in [t_i+1,Hig{h}_i]$ and check if the image $Q_i$ was processed to prevent reprocessing the same image again. Then, for all of the sub-images $Q_{i+1}$ and $Q_{i+2}$ the value C is calculated as:

$$C_{Q_i}=MA{X}_{Q_i}-MI{N}_{Q_i}$$

(7)

with

$$MA{X}_{Q_i}=max\left\{{\epsilon }_T\left(q\right)\right\},\forall q(x,y)\in [Lo{w}_i,Hig{h}_i],$$

(8)

and

$$MI{N}_{Q_i}=min\left\{{\epsilon }_T\left(q\right)\right\},\forall q(x,y)\in [Lo{w}_i,Hig{h}_i],$$

(9)

The sub-image with the highest $C_{Q_i}$ is selected for forward processing.

The algorithm stops when H achieves the desired value. For tumor delineation in PET images, the algorithm stops when H is greater than or equal to 0.999. This value was obtained experimentally. We tested several numbers of iterations and from $H\ge$ 0.999 there were no significant improvements in the sub-image homogeneity (Figure 2). The PET 1–4 images issued in Figure 2 correspond to the PET 1–4 breast cancer images presented in Figure 3. These images were selected due to the different tumor-adjacent tissue characteristics they present, resulting in significant homogeneity differences after the first iteration. Despite these differences, after some iterations the method converges for all images and, since we used the maximum entropy amplitude to select the sub-image to be processed, the algorithm stops, and no other sub-image needs to be processed.

4. Performance Evaluation

In order to test the performance of the proposed methodology, we applied the proposed algorithms to a set of images available from the Cancer Image Archive (TCIA). TCIA is a service that de-identifies and hosts an extensive archive of medical images of cancer accessible for public download. The images are available on the TCIA website: http://www.cancerimagingarchive.net/, accessed on 1 March 2019. The data are organized as “Collections”, typically patients related by a common disease (e.g., lung cancer), image modality (MRI, CT, PET, etc.) or research focus. In this work, we used 40 gray-scale images with 512 × 512 pixels each. Some examples are shown in Figure 4. Other examples and respective ground truth results, obtained by medical experts, are shown in (Figure 5).

To evaluate the different segmentation methods, we use the intersection over union ($IoU$) and pixel accuracy measures. The $IoU$ measures the intersection over the union of the labeled segments and reports the average. The $IoU$ value can be calculated as follows:

$$IoU=\frac{target\cap prediction}{target\cup prediction}$$

(10)

Pixel accuracy is the ratio of the correctly classified elements over all available elements and can be calculated as follows:

$$accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(11)

Although the accuracy measure is more prone to misleading high values due to the classes’ imbalance (tumor background), it was used to establish the differences in the resulting evaluation when using $IoU$.

5. Results

The A-IFSs algorithm was tested with the data set introduced previously. Figure 3 shows examples of some original images, their corresponding results produced by the proposed A-IFSs-based methodology, and their corresponding ground truth. Using the $IoU$ and the pixel accuracy values to evaluate the segmentation results, we confirmed that the use of the homogeneity value, with $H\ge$ 0.999, to obtain the threshold value, resulted in the best segmentation for each image.

5.1. Fixed Threshold Comparison

Over the last years, several studies from different authors confirmed that the most accepted global threshold values, in lesion delineation, and the most suitable for segmenting lesions, are between $34\%$ and $50\%$ of the image max intensity plus the best threshold $58\%$ value from the fixed-threshold experimental evaluation performed. The results of the comparison of different fixed threshold values with the ground truth show that the best fixed threshold value is image-dependent (

Table 1. Fixed threshold IoU results.

T Max	PET 1	PET 2	PET 3	PET 4	...	Average
30%	0.33777	0.82292	0.21323	0.72679	...	0.29735
32%	0.39137	0.87486	0.24328	0.76301	...	0.32694
34%	0.47370	0.92075	0.2837	0.8021	...	0.36072
36%	0.57546	0.96107	0.33012	0.84333	...	0.40531
38%	0.65011	0.99367	0.38675	0.87995	...	0.46184
40%	0.72352	0.92911	0.45873	0.93108	...	0.52214
42%	0.77205	0.89241	0.49902	0.97316	...	0.56530
44%	0.79987	0.84684	0.53415	0.98113	...	0.60077
46%	0.82646	0.80127	0.57098	0.93904	...	0.63505
48%	0.86802	0.76456	0.60362	0.88824	...	0.66256
50%	0.90863	0.72911	0.63573	0.84615	...	0.68626
52%	0.95322	0.67975	0.66405	0.80697	...	0.69683
54%	0.99361	0.64684	0.68339	0.76633	...	0.70175
56%	0.96299	0.59494	0.70585	0.73295	...	0.70384
58%	0.91231	0.54937	0.73068	0.69231	...	0.70475
60%	0.86323	0.47722	0.75642	0.64586	...	0.70115

). Therefore, we used the most-used values in the literature ($40\%$ and the $50\%$) in our comparison studies [19,24,25,45,46,47,48,49].

5.2. Comparison with Different Segmentation Algorithms

In this section, the proposed algorithm is compared to existing algorithms using the IoU and pixel accuracy measures.

We evaluated our algorithm against existing similar methodologies, fuzzy C-means (FCM), K-means, and the affinity propagation (AP) algorithms. Since our algorithm detects and delineates the tumor, having as a final result a binary image, we adapted the existing algorithms to produce similar results. In each method, we also used different parameter values in order to choose the best performance setup.

The FCM and K-means algorithms need to define a seed and a number of clusters to perform the segmentation. With the objective to have a binary image as a result, we set the number of clusters at 2 and handcrafted the seed position to the max intensity of the image (assuming the best possible seed position), resulting in a binary image. To achieve a binary image using the AP, we tested different weighting parameters, m and n, achieving the best result when $m=1$ and $n=6$.

Figure 6 shows the results obtained with the $IoU$ method, and Figure 7 shows the pixel accuracy. Each algorithm output result is compared to the ground truth.

The obtained results show that the fixed threshold $t50\%$ had better performance than the fixed threshold $t40\%$ considering any of the two evaluation measures, and the fixed threshold $t40\%$ had the worst segmentation results of the tested methods. The fixed threshold $t58\%$ obtained the second-best $IoU$ and accuracy values (

Table 2. IoU algorithm comparison results.

Number of Images: 40
	Min	Average	Max
t40%	0.16199	0.52214	0.96322
t50%	0.28319	0.68626	0.98023
t58%	0.31577	0.70475	0.98688
K-Means	0.04167	0.65334	0.98688
FCM	0.02005	0.46292	0.98586
AP	0.28319	0.69397	1
A-IFSs	0.09229	0.76492	1

and

Table 3. Pixel accuracy algorithm comparison results.

Number of Images: 40
	Min	Average	Max
t40%	0.97498	0.99223	0.99988
t50%	0.98842	0.99693	0.99995
t58%	0.99693	0.99777	0.99997
K-Means	0.99092	0.99732	0.99997
FCM	0.86800	0.97388	0.99996
AP	0.98847	0.99696	1
A-IFSs	0.99235	0.99843	1

). These are expected results because this threshold was obtained for our data set. According to both measurement values, the FCM had the highest distribution of values (as can be seen in Figure 6) and the lowest $IoU$ values. The K-means and AP methods showed similar performance, with AP achieving better $IoU$ scores despite the higher distribution of the accuracy results. The proposed algorithm outperformed all of the other methodologies, and this result is even more noticeable (around at least $10\%$ increase in performance) when using an evaluation measure ($IoU$) not biased by the numerous background pixels, and with a low distribution of values in accuracy. Considering that our methodology is unsupervised, in the 40 images, some produced a wide range of $IoU$ values. However, it is the method that obtained the best global results. The observed results indicate that A-IFSs has more precision and repeatability.

6. Discussion

In this paper, a new A-IFSs-based segmentation algorithm is proposed to increase the accuracy of tumor delineation in PET images. This is a crucial step in order to minimize the damage by future treatments, but optimally including the entire extent of the diseased tissue. In the proposed methodology, we introduced an A-IFSs-based segmentation approach that, unlike the existing methodologies, works without any previous processing or human interaction defining a ROI, any seed position, or SBR value. Moreover, being an AIFSs-based methodology, and considering that AIFSs were proven to efficiently deal with image uncertainties in the past, the proposed methodology was able to better deal with the PET images’ uncertain tumor boundaries. Atanassov’s intuitionistic fuzzy index values are used for representing the hesitance of an expert on determining whether a pixel of the image belongs to a non-healthy tissue or a healthy tissue. In order to verify its effectiveness, five existing representative methods were used for comparison [25,45,50,51,52]. Considering solely the fixed thresholds methods, the fixed threshold $t58\%$, having been experimentally selected for our data set, obtained the second-best IoU and accuracy values. Regarding the other tested methods, FCM had the highest distribution of values and the lowest IoU values while the K-means and AP methods showed similar performance, with AP achieving better IoU scores despite the higher distribution of the accuracy results. Overall, the experimental results show that, despite not using previous information or making use of human interaction, the proposed method has achieved the best performance among all methods tested showing more precision and repeatability. Notably, the overlapping of the target area and the predicted area increases significantly with this method, representing an improvement, outperforming all of the other methodologies tested for tumor delineation in PET images. The hierarchical partition of the pixel intensity classes of the method together with a non-binary definition of class membership seems to be the main reason for the superior performance over the other methods. However, since the method relies on image intensity clustering, this may be understood as a limitation, and future work should include spatial feature information.

7. Conclusions

In this work a new A-IFSs-based segmentation algorithm is proposed for tumor delineation in PET breast cancer images. The proposed methodology is a low computational cost, fully unsupervised, and effective tool for tumor delineation in PET images. Future work is intended for tumor delineation of different human cancer types.