Optimal Thresholding for Multi-Window Computed Tomography (CT) to Predict Lung Cancer

Abstract

Lung cancer is the world’s second-largest cause of cancer mortality. Patients’ lives can be saved if this malignancy is detected early. Doctors, however, encounter difficulties in detecting cancer in computed tomography (CT) images. In recent years, significant research has been devoted to producing automated lung nodule detection methods that can help radiologists. Most of them use only the lung window in their analysis and generally do not consider the mediastinal windows, which, according to recent research, carry important information. In this paper, we propose a simple yet effective algorithm to analyze multi-window CT images for lung nodules. The algorithm works in three steps. First, the CT image is preprocessed to suppress any noise and improve the image quality. Second, the lungs are extracted from the preprocessed image. Based on the histogram analysis of the lung windows, we propose a multi-Otsu-based approach for lung segmentation in lung windows. The case of mediastinal windows is rather difficult due to irregular patterns in the histograms. To this end, we propose a global–local-mean-based thresholding technique for lung detection. In the final step, the nodule candidates are extracted from the segmented lungs using simple intensity-based thresholding. The radius of the extracted objects is computed to separate the nodule from the bronchioles and blood vessels. The proposed algorithm is evaluated on the benchmark LUNA16 dataset and achieves accuracy of over 94% for lung tumor detection, surpassing that of existing similar methods.

1. Introduction

According to the recent report of the American Institute of Cancer published on 14 February 2022, there are over 236,740 reported cancer infections, with 117,910 instances reported in males and 118,830 cases diagnosed in women [1]. Around 130,180 people have died, including 61,360 females and 68,820 males. Both small cell lung cancer (SCLC) and non-small-cell lung cancer (NSCLC) are represented among these patients. NSCLC is a far more typical type of respiratory tumor [1,2]. Lung tumor death rates in men have declined by 56% since the year 1990, whereas women’s death rates have declined by 32% since 2002. Male birth mortality increased by 5% annually from 2015 to 2019, whereas female birth mortality has increased by 4% annually [2]. According to statistical data, up to 80% of malignancies are NSCLC, while up to 20% among all malignancies are SCLC. The three most prevalent types of NSCLC are adenocarcinoma, squamous cell carcinoma, and large cell carcinoma [3]. Mortality rates in males decreased between 2015 and 2019 by 5% annually, while female death rates decreased by 4% annually within this period. As a result, many people have quit smoking, and new technology for lung nodule identification and treatment has been created, such as the dual-energy imaging approach [4]. Researchers have improved tumor identification by combining this method with fluoroscopy during radiotherapy [2,5].

Numerous indicators have been devised by the World Health Organization (WHO) to aid lung cancer prevention, which include avoiding the use of cigarettes and alcohol and exercising regularly to maintain a healthy weight [6]. On the other hand, early detection can help those who have the condition by enabling them to obtain the best care as soon as possible. Scientists struggle to identify minute nodules because the malignancy is so small at this stage, but it is treatable also. To this end, computer-based systems and devices have been proposed to help radiologists who analyze computed tomography (CT) images to detect lung cancer.

In order to design efficient computer-aided diagnosis (CAD) systems for the identification of lung cancer cells, image processing, machine learning, and deep learning have been extensively explored. Most image-processing-based methods use some form of thresholding to identify the nodules in lung CT images, and they have garnered a lot of interest since they allow for the modification of image pixels for more effective analysis. For instance, Li et al. [7] proposed an automated lung nodule detector that is based on thresholding. Six different features, including the effective diameter, degree of compactness, and irregularity, were computed from detected nodule candidates and used to filter the nodule and non-nodule regions. The method presented in [8] utilizes a watershed for lung segmentation. It uses a marker control watershed and region growing approach to segment potential nodule regions from CT images. Different features are then computed from the binarized segmented lung to identify nodules. A similar method is presented in [9], which uses a simple thresholding-based segmentation technique to extract the lung nodule region. Several features, e.g., area, perimeter, and eccentricity, are then computed to separate nodules. The method in [10] uses morphological procedures as a preprocessing approach; subsequently, they extracted the nodule by subtracting the thresholded image from an object removal mask. A technique presented in [11] denoises CT images using image enhancement techniques and uses histogram equalization to enhance the image quality. The images are then segmented using the watershed approach. Finally, the lung nodule candidates are categorized according to their area, perimeter, and eccentricity.

Machine learning (ML) systems can adapt from data, detect patterns, and make different choices without the need for human interaction. Machine learning techniques are widely used in medical diagnosis and the monitoring of patients [12,13,14,15,16,17]. This method aids in the recognition of patterns and the computation of visual attributes that are relevant to the diagnosis process. ML-based nodule detectors generally extract important features from CT images and use some classification techniques to separate the nodules. For example, the model presented in [18] segments the nodule area and extracts 15 texture features from both the segmented nodule and its surrounding region; it then trains a model using support vector machines (SVM) to detect nodules. In the method introduced in [19], gray-scale CT images were preprocessed using median filters to eliminate noise. To separate the image into several parts and retrieve useful information during segmentation, watershed segmentation was performed. The method extracts image characteristics such as the area, perimeter, diameter, etc., from the segmented regions. To determine whether the identified nodule was cancerous or benign, the authors ultimately employed an SVM classifier. Khehrah et al. [20] proposed a CAD system that separates the lungs using a histogram-based method and looks for potential nodules by computing different statistical and shape-based features and classifying them with SVM.

Deep learning (DL) tries to mimic human behavior to gain knowledge [21]. Due to its high performance and versatility, it helps to process a large amount of medical information and shows appreciable performance in tasks such as segmentation, classification, and detection [22]. Several lung cancer detection methods in the literature exploit deep learning. For example, in the method presented in [23], the authors used U-Net [24] and ResNet [25] models to extract the features of CT images and then used different classifiers to classify cancerous CT scans. Asuntha et al. [26] proposed a deep learning method by combining CNN and the fuzzy particle swarm optimization algorithm. The main advantage of this algorithm is that it lessens the computational intricacy of the CNN algorithm and gives better performance. The method introduced in [27] computes a set of features, such as geometric features, intensity features, texture descriptors, gradient features, and region descriptors, and classifies them using deep learning. Masood et al. [28] deployed a cloud-based 3D deep convolutional neural network (3DDCNN) to aid radiologists.

The DL-based lung nodule detection presented in [29] employed histogram equalization to enhance the image quality and applied an improved profuse clustering technique to separate the region of interest. A CAD system [30] that uses CT images as input and identifies regions of interest first, followed by nearby regions, to evaluate CT scans for positive or negative lung cancer is presented. To diagnose cancer at an early stage, it employs 2D and 3D convolutional neural networks, namely Google-net-based models, as classifiers. To identify and categorize lung cancer in CT scans, Wafa et al. [31] proposed a CAD that identifies a small nodule (less than 10 mm) in a 3D CT scan image. For better results, each image was converted into Hounsfield units during the preprocessing step, before being segmented using the Watershed approach. Following segmentation, linear scaling is used to normalize the 3D images, which are provided to the U-Net-based malignancy classifier to locate tiny boxes of the best candidates for nodules. Finally, an image composed of these small 2D slices of the best nodule candidates is obtained, and the layers are classified using the 3D-CNN classifier. The 2017 Kaggle Data Science Bowl was considered by Kuan et al. [32]. They used a modified ResNet for both nodule detection and malignancy detection after preprocessing the CT scans.

Most existing thresholding-based lung nodule detection techniques handle CT scan images consisting of lung windows only and do not consider the mediastinal windows in the detection of the disease. Research such as [33] has shown that the detection rate is improved if both windows are analyzed for lung nodules. On the other hand, deep learning requires a huge amount of data to train a model perfectly; it is challenging to obtain such an extensive dataset in the medical field, where imaging and tagging by experts is a tedious and expensive task. Time consumption is another factor that must be considered in regard to deep learning models. Therefore, a simple and fast yet effective method is needed. In this paper, we propose a thresholding-based, simple and accurate technique for lung nodule detection. The proposed technique can address both lung windows and mediastinal windows efficiently and gives appreciable accuracy in terms of cancer detection.

2. Proposed Method

As with most existing lung nodule detectors, the proposed method also consists of three steps. First, the images are preprocessed to eliminate noise and to improve the image contrast and quality so that the later stages of the algorithm can perform better. Second, the lung is segmented from the processed images, and, in the third and final step, nodules are detected using the shape characteristics.

2.1. Preprocessing

CT scan images can be noisy, which can adversely affect the segmentation accuracy. Therefore, almost every existing lung nodule detection technique implements some preprocessing method to improve the image quality. To reduce the noise effect and enhance the contrast, we propose to use image normalization and image smoothing as preprocessing steps. To increase the pixel intensity and improve the contrast, image normalization is used. The images in our collection have a range of pixel intensities from −1000 to 1000. Image normalization is employed to rescale the intensity values within the 0–255 range. Moreover, to reduce or eliminate the image noise, image smoothing is implemented. To this end, we use the Gaussian smoothing filter.

2.2. Lung Segmentation

After preprocessing, the lung in the image must be segmented to extract the region of interest for further processing. There are two types of CT scan images, i.e., lung windows and mediastinal windows, as shown in Figure 1. One may note that in a CT image of the lung window, there are three types of gray levels: a black background and gray regions for the chest wall and lung area. These three regions can be seen in the histogram of the image, as discussed in [20]. Figure 2a presents the histogram plot of the lung window image shown in Figure 1a. The first peak in the histogram corresponds to the black background in the image, which can be easily dropped. The second peak represents the lungs and the circular gray region, and the third peak separates the circular component from the lung region. Therefore, a threshold from the second valley can be used to extract the lungs.

The histogram of the mediastinal window, on the other hand, is straightforward; see Figure 2b. The background and the lung intensities are concentrated around the lower end of the histogram and the region around the lungs is gray. This means that in order to binarize the mediastinal window, we simply require an optimal threshold value from the first valley of the histogram.

2.2.1. Segmentation in Lung Window

For image segmentation, there are several existing segmentation techniques, such as flood fill, thresholding, and watershed. The flood fill algorithm is not suitable for segmentation since it can miss some important areas, as with thresholding. The watershed algorithm is widely used in lung segmentation and is more effective; however, in most cases it requires seeding the region of interest.

Extracting lungs from the lung window scan is indeed a tri-class pixel classification problem (Figure 1a). To achieve this pixel categorization, we use the multi-Otsu thresholding algorithm [34]. Otsu’s algorithm has been explored to solve many medical imaging problems, e.g., [35,36,37]. It finds the optimal thresholds by maximizing the between-class variance with an exhaustive search [38]. The multi-Otsu method returns two thresholds for three class problems. In our case, the first threshold comes from the valley between the first and the second peaks, and the second threshold separates the second class of pixels from the third class of pixels. The thresholds computed in the lung window shown in Figure 1a using the multi-Otsu method are shown in Figure 3 using the red markers. The second threshold separates the lungs from the rest of the image. Figure 4 shows a few lung window samples from the test dataset and the segmented lungs using the proposed method.

2.2.2. Segmentation in Mediastinal Window

We recall that mediastinal window images usually show two types of gray level intensities; however, in some cases, there are more than two groups, which results in inaccurate thresholds computed with the multi-Otsu method. For instance, lung segmentation on a sample mediastinal window using the multi-Ostu threshold is shown in Figure 5. Figure 5b shows the histogram of the mediastinal image shown in Figure 5a. The segmentation result using this threshold is shown in Figure 5c. Many patches can be noted in the segmented image, which have been incorrectly classified.

Analysis of the mediastinal window histograms shows that the incorrect threshold occurs when there are a few closely appearing peaks on the higher end of the histogram, instead of one large peak. The Otsu method in such cases sometimes results in converging the threshold between these peaks instead. A simple strategy to avoid an incorrect threshold for lung segmentation in the mediastinal window is to compute the image’s global mean and use it as a threshold value. Let I be an image of size $M\times N$; the global mean ${\mu }_g$ of I can be estimated as

$${\mu }_g=⌈\frac{1}{MN}\sum _{x=1}^M\sum _{y=1}^{N}I(x,y)⌉$$

(1)

This mean serves as a better threshold than that estimated with the Otsu method in many cases; however, it still can suffer from bias toward the larger part. Specifically, if there is a significant number of large values (gray) compared to the background (black), the mean can be found in a similar region to the Otsu threshold. In Figure 6, we present some results obtained using the global threshold for lung extraction. The top two rows show the results where segmentation using global-mean-based thresholding produced good results, and the bottom two rows show some failure cases.

To overcome this problem, we use the global mean to roughly separate the image into two regions, ${\mathrm{\Omega }}_1$ and ${\mathrm{\Omega }}_2$. The set ${\mathrm{\Omega }}_1$ consists of those image pixels that lie in the interval [0, ${\mu }_g$], and ${\mathrm{\Omega }}_2$ contains the rest of the pixels, i.e., the pixels with intensity lying within [${\mu }_g$, 255]. The local means of the two regions are computed and used to find a better threshold. The local mean ${\mu }_1$ of the region ${\mathrm{\Omega }}_1$ is then computed.

$${\mu }_1=\frac{1}{\left|{\mathrm{\Omega }}_1\right|}\sum _{x,y}{\mathrm{\Omega }}_1(x,y)$$

(2)

where $\left|{\mathrm{\Omega }}_1\right|$ represents the size of the set. Similarly, ${\mu }_2$ is computed for ${\mathrm{\Omega }}_2$.

$${\mu }_2=\frac{1}{\left|{\mathrm{\Omega }}_2\right|}\sum _{x,y}{\mathrm{\Omega }}_2(x,y)$$

(3)

The threshold $\tau$ is then taken as the average of the two local means.

$$\tau =\frac{{\mu }_1+{\mu }_2}{2}$$

(4)

Since the threshold $\tau$ is taken as the average of the two local means, it overcomes the bias issue. In Figure 7, we present a few examples where the global mean thresholding technique produced poor results and the local mean thresholding produced ($\tau$) quite accurate results. From these lung images, it can be noted that a few lung pixels are incorrectly marked as non-lung and vice versa. To recover these pixels, we employ the watershed algorithm [39,40]. The obtained lung mask is used as the seed for the watershed algorithm. The third column in Figure 7 shows the result after applying the watershed algorithm. The resulting images present accurate lung segmentation.

2.3. Nodule Detection

After the lungs are segmented, the next step is the extraction of vessels from the lungs. We achieve this by simply applying thresholding to the segmented lungs. The intensity levels of objects inside the lungs are lower than the lung background. We observe that a threshold value −500 is a good choice to extract vessels and nodules from the segmented lungs, as shown in Figure 8a. Following vessel extraction, morphological techniques, namely binary opening and binary closing, are used to eliminate the smaller objects from the area. We calculate the region of all the objects included in the vessel mask. Almost all nodules are circular, with white pixels covering most of the circle. As a result, we can disregard the additional objects with the elongation feature. We compute the diameter of each lesion to detect the nodule. The diameter threshold that we set to detect nodules is 3 mm because, in our dataset, the radiologists annotated the nodules within the range of 3–30 mm and this is the general nodule size in lung cancer. A few radiologists considered a nodule size of 2 mm but they later disregarded these annotations [41]. Figure 8c shows the nodule detected using the extracted nodule candidates (shown in Figure 8b). Figure 8d shows the detected nodule in the original image. The pseudocode of the proposed algorithm is presented in Algorithm 1.

Algorithm 1 Proposed Nodule Detection Algorithm

Require: A CT Image I of size $M\times N$ Ensure: Detected nodule(s). 1: Normalize I to bring the intensity values within range 0–255 2: Suppress noise in I using Gaussian blur [$G(x,y)=\frac{1}{2\pi {\sigma }^2}{e}^{-\frac{x^2+y^2}{2{\sigma }^2}}$] 3: if I is a lung window then 4:    Compute histogram H of I 5:    Find the two local minima $l_a,l_b$ in H 6:    [${\tau }_1,{\tau }_2,{\tau }_3$] ← multi-Otsu($I,l_a,l_b$) 7:  Threshold I using ${\tau }_2$ to separate lungs from rest of the lung window image 8: else {I is a mediastinal window} 9:    ${\mu }_g=⌈\frac{1}{MN}{\sum }_{x=1}^M{\sum }_{y=1}^{N}I(x,y)⌉$ {Compute the global mean $m{u}_g$ of I} 10:    Use ${\mu }_g$ to roughly separate divide I into two regions, ${\mathrm{\Omega }}_1$ and ${\mathrm{\Omega }}_2$. 11: ${\mu }_1=\frac{1}{\left|{\mathrm{\Omega }}_1\right|}{\sum }_{x,y}{\mathrm{\Omega }}_1(x,y)$ {Compute the local mean ${\mu }_1$ of the region ${\mathrm{\Omega }}_1$} 12: ${\mu }_2=\frac{1}{\left|{\mathrm{\Omega }}_2\right|}{\sum }_{x,y}{\mathrm{\Omega }}_2(x,y)$ {Compute the local mean ${\mu }_2$ of the region ${\mathrm{\Omega }}_2$} 13: $\tau =\frac{{\mu }_1+{\mu }_2}{2}$ {The threshold $\tau$ is then taken as the average of the two local means} 14: Threshold I using $\tau$ to separate lungs from rest of the mediastinal window image 15: end if 16: Threshold I using threshold value $-500$ This extracts vessels and nodules from the segmented lungs I. 17: I = OPEN (I) {OPEN is morphological binary open operator.} 18: I = CLOSE (I) {CLOSE is morphological binary open operator.} 19: Drop the objects in I that have the elongation feature, i.e., those that are not circular 20: Compute the diameter d of each lesion in I 21: if $d\ge 3$ mm then 22:  Mark the region as nodule 23: else $\left\{d<3\right\}$ mm 24:  Mark the region as non-nodule 25: end if 26:Highlight the nodule regions, if any, using red circles

3. Experiments and Results

In this section, we report the performance of the proposed algorithm. We also compare the results with other existing nodule detectors to show the effectiveness of the proposed method.

3.1. Dataset

The experiments are performed on the benchmark LUNA16 (LUng Nodule Analysis 2016) dataset [42]. It is derived from the publicly available LIDC/IDRI database. This data collection comprises labeled data from 888 patients. The data for each patient are composed of CT scan data and labeling (0 for non-cancer and 1 for cancer). Each image is in 3D MHD format with 2D slices of 512 × 512 dimensions. For each pixel within a 2D slice, the range of gray levels is from −1000 to 1000. The lung region varies in 2D slices of an image. MHD is a MetaImage medical format that contains a description file of an image. Each $.MHD$ file comes with a $.RAW$ file that contains explicit data of that image. The dataset characteristics are summarized in

Table 1. LUNA16 dataset description.

Attribute	Property
Number of CT scans	888
Annotated lung nodule	1186
Format	3D MHD
2D slice size	512 × 512

. A 3D representation of a lung CT scan is shown in Figure 9. The dataset can be downloaded from (accessed on 5 June 2023) https://luna16.grand-challenge.org/Download/.

3.2. Performance Evaluation and Comparison

We compute numerous performance parameters to assess the effectiveness of the proposed lung nodule detection algorithm. There are four possible outcomes when an image is tested with any lung nodule detection algorithm: if the image is cancerous and is predicted as cancerous, it is a true positive ($TP$); if the image is non-cancerous and is predicted as non-cancerous, it is a true negative ($TN$); if the image is non-cancerous but is predicted as cancerous, it is a false positive ($FP$); and if the image is cancerous but is predicted as non-cancerous, it is a false negative ($FN$). Based on this formation, we calculate the accuracy, sensitivity or recall, and precision values to test the performance of the proposed method.

Accuracy reflects all true positive and true negative results in detecting nodules. It calculates the percentage of accurately predicted data from all data points as follows:

$$Accuracy=\frac{TN+TP}{TN+TP+FN+FN}$$

(5)

Precision reflects all positive results that are actually positive marked by the radiologist.

$$Precision=\frac{TP}{TP+FP}$$

(6)

Sensitivity or recall measures the percentage of successful results to determine the correct prediction.

$$Recall=\frac{TP}{TP+FN}$$

(7)

We computed all performance metrics on the test dataset and compared the results with existing similar approaches. Specifically, we selected the following methods for performance comparison: Nasser [43], Sang [44], Makaju [19], Xie [45], Alakwaa [31], Jin [46], and Khumancha [47]. The method proposed by Nasser et al. [43] employed an artificial neural network to determine whether or not lung nodules were present in the CT image. The method presented in [44] used U-Net [24], encoder–decoder [48], and MixNet [49] models. In Alakwaa [31], 3D-CNN is used to detect lung nodules from CT scans. They preprocessed the images first using the watershed algorithm for lung segmentation. The lung nodule detector presented in [47] also uses a 3D-CNN model to identify lung nodules using low-dose CT scan images. In [19], the watershed algorithm is used to segment lungs from CT scans and feature extraction is used to detect nodules and then send these features to an SVM for classification. In [45], the authors used R-CNN and 2D CNN models to detect lung nodules and classify true nodules, respectively.

The results of the proposed and the compared methods are presented in

Table 2. Nodule detection performance comparison of the proposed and the compared methods in terms of precision, recall, and accuracy. The dataset used in evaluation is listed under ‘Dataset’. ‘—’ indicates the unavailability of the score.

Method	Accuracy	Recall	Precision	Dataset
Nasser [43]	0.96	—	—	—
Sang [44]	—	0.94	—	LIDC-IDRI
Makaju [19]	0.92	—	—	—
Jin [46]	0.84	0.82	0.86	—
Xie [45]	—	0.86	—	LUNA16
Alakwaa [31]	0.86	—	—	LUNA16
Khumancha [47]	—	0.82	0.89	LUNA16
Proposed	0.94	0.97	0.92	LUNA16

. The proposed algorithm achieves accuracy of 0.94, which is better than all compared methods except the Nasser [43] algorithm, which performs marginally better. However, they do not report the other metrics, making it impossible to draw a fair comparison. In terms of the precision and recall measures, the proposed algorithm outperforms the compared methods by achieving scores of 0.92 and 0.97, respectively. In recent years, deep-learning-based methods have been extensively explored in medical image analysis and showed good performance, particularly for image segmentation and classification problems. These approaches, however, require huge amounts of tagged data for training. Preparing ground truth for medical data is challenging because it needs expert radiologists and it is a very time-consuming task. Therefore, these approaches are not very practical. The proposed method does not require any such training and yet it can produce appreciable results.

In Figure 10, we present a few results of the proposed algorithm on test images from the LUNA16 dataset. The results and statistics presented in Table 2 show that the proposed method is quite accurate and reliable. Moreover, it works for both lung window scans and mediastinal window scans and achieves better accuracy.

The proposed method was implemented in Python and its source code has been publicly released for research purposes at (accessed on 6 June 2023). https://github.com/Muflah-cloud/lung-cancer-detection. All experiments were performed on a laptop running macOS Big Sur with a 2.4 GHz Dual-Core Intel Core i7 and 16GB RAM. The execution time of the proposed method was computed on the entire dataset and averaged to analyze the computational efficiency. Our algorithm takes 6.25 seconds on average to process one CT scan, which shows that the method is computationally efficient.

4. Conclusions

In this paper, we present a lung nodule detection algorithm using CT images. The algorithm is a combination of two methods, one for detecting the nodules in the lung window and the other for analyzing the mediastinal window for nodules. The proposed algorithm is simple; it uses multi-Otsu-based thresholding and a novel global–local-mean-based segmentation to extract the lungs. The watershed algorithm and thresholding are used to find the nodule candidate regions, which are identified using their radii. The algorithm was tested on the LUNA16 dataset with 94% detection accuracy, making it superior to many existing techniques, because it handles multiple window types for segmentation and detects nodules solely through simple image processing operations. As we chose to extend our segmentation approach to a multi-modality context, the originality of our method has the potential to exert a significant effect in detecting lung nodules.