Artificial Intelligence Approach for Early Detection of Brain Tumors Using MRI Images

Abstract

Artificial intelligence (AI) is one of the most promising approaches to health innovation. The use of AI in image recognition considerably extends findings beyond the constraints of human sight. The application of AI in medical imaging, which relies on picture interpretation, is beneficial for automatic diagnosis. Diagnostic radiology is evolving from a subjective perceptual talent to a more objective science thanks to AI. Automatic object detection in medical images is an essential AI technology in medicine. The problem of detecting brain tumors at an early stage is well advanced with convolutional neural network (CNN) and deep learning algorithms (DLA). The problem is that those algorithms require a training phase with a big database of more than 500 images and time-consuming with a complex computational and expensive infrastructure. This study proposes a classical automatic segmentation method for detecting brain tumors in the early stage using MRI images. It is based on a multilevel thresholding technique on a harmony search algorithm (HSO); the algorithm was developed to suit MRI brain segmentation, and parameters selection was optimized for the purpose. Multiple thresholds, based on the variance and entropy functions, break the histogram into multiple portions, and different colors are associated with each portion. To eliminate the tiny arias supposed as noise and detect brain tumors, morphological operations followed by a connected component analysis are utilized after segmentation. The brain tumor detection performance is judged using performance parameters such as Accuracy, Dice Coefficient, and Jaccard index. The results are compared to those acquired manually by experts in the field. The results were further compared with different CNN and DLA approaches using Brain Images dataset called the “BraTS 2017 challenge”. The average Dice Index was used as a performance measure for the comparison. The results of the proposed approach were found to be competitive in accuracy to those obtained by CNN and DLA methods and much better in terms of execution time, computational complexity, and data management.

1. Introduction

One of the most promising methods of health innovation is artificial intelligence (AI). The application of AI to picture identification significantly expands research beyond the limitations of human vision. Automatic diagnosis benefits from the use of AI in medical imaging, which depends on image interpretation. Diagnostic radiology is becoming a more objective discipline with the use of AI. An essential use of AI in medicine is automatic segmentation and object detection in medical images. Picture segmentation is a fundamental and challenging task in image processing, pattern recognition, and computer vision. It is used in many different applications, such as medical image processing and ship target segmentation [1]. The main goal of segmentation is to divide an image into homogeneous classes. Each element in each class shares the characteristics of grayscale, feature, color, intensity, and texture [2,3,4,5]. In the literature, there are four common picture segmentation techniques that fall into four groups: clustering-based techniques, region-based techniques, graph-based techniques, and thresholding-based techniques.

An image analysis system’s success is dependent on the segmentation’s quality. It is crucial to recognize brain tumors as soon as possible in order to begin therapy. While examining brain images, segmentation is of utmost importance. Most medical image research focuses on MR images because they are a critical component of diagnostic imaging since they allow for the early detection of aberrant alterations in diseases and organs [6]. When there are only a few classes, brain MR pictures are piecewise constant [7]. With brain MRI, segmentation methods such as clustering, thresholding, region growth, and manual segmentation can be used.

Several segmentation techniques make use of the intensity-based methodology. They work with the pixel intensity of the image, classifying each pixel according to its level of intensity. The method of picture segmentation known as thresholding is both well-established and in use [8,9]. The intensity threshold values are computed using intensity histograms [7]. Although being a rapid and inexpensive technique, thresholding is susceptible to picture noise. A method called region growth is used to separate related areas of an image from a collection of pixels with comparable brightness. The classification technique employs data with known labels to divide the visual feature space into homogenous items [10,11].

A segmentation technique called clustering does not need training images or the help of a supervisor. Clusters of pixels with comparable brightness and/or texture are identified throughout the image. By calculating the mean intensity for each class, the k-means clustering technique separates input data into k classes using Euclidean distance. Fuzzy-based segmentation algorithms and hybrid segmentation methods are being used for MRI images [12]. According to the complexity of the scene in the image, level set techniques, which segment images based on specific levels, are a novel approach for image segmentation based on finding peaks and/or valleys in the image’s histogram. There are several ways to set the levels (LSM). Ref. [13] provides a multi-local statistical segmentation method that employs a weighted edge-based level to segment noisy images. In order to effectively separate objects of interest in a picture, [14] proposes a new level set approach that combines edge, region, and 2D histogram information.

One of the thresholding techniques that has achieved great success in medical imaging and Nanomedicine imaging [15] is Otsu’s segmentation methodology [8,16,17]. The two different kinds of threshold procedures are local and global threshold techniques. The various kinds of global groups include spatial methods, histogram shape-based methods, entropy-based methods, and object attribute-based approaches.

These techniques come in two varieties: bi-level thresholding and multi-level thresholding. Multilevel can produce several thresholds, such as tri-level or quad-level, and break pixels into different classes according to the intensity histogram’s global or local level when used to segment complicated images [18,19]. Multilevel divides the image into several classes based on the global or local intensity histogram, whereas bi-level divides the image into two classes.

In order to determine the threshold values for multilevel thresholding segmentation, many studies have used objective functions optimization techniques. Slime Mould Algorithm for image segmentation, an advanced multi-threshold technique, has been recently proposed [20]. A framework for efficient brain tumor classification using MRI images is proposed in [21]. It is based on three steps, first, image improvement of visual quality. Second, the locations of tumors are obtained based on a clustering-based method. Third is feature extraction. A recent deep learning approach [22,23] uses two different pre-trained deep learning models to extract features from images as vectors. The vectors are then combined using the partial least squares (PLS) method to form a hybrid feature vector. Then, the top tumor locations are revealed via agglomerative clustering. Those methods provide very good accuracy, meanwhile, are heavily time-consuming. An updated Unet algorithm was published recently [23]. The results obtained on brain segmentation are about 84% average Dice index.

When used with medical brain images and a real-coded genetic algorithm, the innovative technique of multilevel thresholding for brain segmentation has been demonstrated to yield accurate and promising results [24]. In 2017, multiple OTSU thresholding and cross-correlation were used to identify brain tumors, and in the same year, a multilevel thresholding method based on electromagnetics was created to accurately segment brain MRI data to detect white matter and gray matter [25]. Using modified multi-level Otsu Thresholding and cross-correlation of principal components, tumor identification in MRI images has recently been developed [26]. A comparison between the Patch-Levy Bees Algorithm and the Harmony Search Algorithm for Multilevel Thresholding of Brain Tumor MRI Images was published by [27].

The segmentation of retinal blood vessels using morphology and hybrid multilevel thresholding has been shown to be a reliable technique [28]. The brain MR image was segmented using multilevel thresholding, particle swarm optimization, the Otsu approach, and anisotropic diffusion [29].

This study compares a few CNN and DLA segmentation techniques used on MRI brain tumor pictures to a developed approach for multilevel segmentation of MRI brain tumor images based on an updated harmonic search optimization algorithm [30]. Performance evaluation indicators are used in the comparison, and the result is favorable and competitive in a time of execution and accuracy.

2. Materials and Methods

2.1. Materials

The vast publicly accessible collection of MRI scans of brain tumors from the Brain Tumor Segmentation Challenge BraTS 2017 and 2021 datasets [31,32] was used in this study. In our experiments, a set of 40 carefully selected images that had previously been utilized by other researchers participating in the challenge were used to compare our findings to those of other approaches [33,34,35].

Native (T1), post-contrast T1-weighted (T1Gd), T2-weighted (T2), and T2 Fluid Attenuated Inversion Recovery (T2FAIR) are the four grayscale contrast modalities used in each MRI scan.

In order to allow for model generalization, these images were collected using a variety of clinical protocols and scanners from different institutions.

Each image was manually segmented by a number of professionals using the same annotation procedure, and the comments were approved by knowledgeable neuro-radiologists. Three labels are used to label the photos with various tumor classes (1, 2, and 4). Label-1 detects necrotic and non-enhancing tumor cores (NCR/NET), Label-2 detects peritumoral edema, and Label-4 detects GD-enhancing tumors (ET).

2.2. Methods

A flowchart of the overall procedure created for picture segmentation and brain tumor identification is shown in Figure 1. Pre-procession is the first phase, then segmentation, morphological operation, and connected component analysis are the final two.

2.2.1. Pre-Processing

All BraTS 2017 & 2021 MRI scans undergo conversion to the NIfTI file format [36], co-registration to the same anatomical template (SRI24) [37], resampling to a consistent isotropic resolution (1 mm³), and finally, skull-stripping. In this study, we transformed 3D cube data into 2D images (slices) using ImageJ software to reduce the computational cost and ease the segmentation problem from 3D to 2D images. The central slices in the brain were used for further processing and tumor detection.

2.2.2. Segmentation Algorithm

Classical image processing techniques such as threshold-based, region-based, and clustering algorithms are frequently used in the segmentation of brain tumors. For segmentation, various techniques are employed in brain imaging, including spatial techniques, clustering-based techniques, histogram shape-based techniques, entropy-based techniques, and object attribute-based techniques. Techniques for threshold-based segmentation can be local or global. Although local is based on the histogram of a particular area of the image, global is based on the histogram of the entire image. There are two categories for threshold method types: bi-level threshold and multilayer threshold. Based on the intensity values and/or statistical properties (variance, entropy, objective function) of the histogram, multilevel can construct various thresholds, such as tri-level or quad-level, separating the pixels into multiple classes.

Threshold-based segmentation

An old and straightforward technique called threshold-based segmentation analyzes the brightness of pixels with a given grey level to identify different parts of the brain. The two types of thresholding techniques are global and local thresholding procedures. The Otsu technique, for instance, divides image pixels into two groups: the background and the foreground (objects) [8]. On the other hand, local thresholding only affects a limited portion of the image. Moreover, prior information about the contents of the image is necessary in order to choose the optimum threshold value. For instance, the intensity histogram and other statistical data could be utilized to determine the threshold value for various brain regions. The approaches of Otsu [8] and Kapur [9] are well recognized for achieving a good threshold; each of them suggests a different objective function that needs to be maximized in order to find the ideal threshold values. Both methods make use of the histogram to calculate the mean, variance, and related objective functions. The number of levels or classes is set by the operator. The method used by Otsu is a nonparametric thresholding technique that uses the largest variance value among the several classes as a segmentation criterion for images. According to [8,9], the intensity levels from a grayscale image are used to determine the probability distribution of the intensity values from the image’s histogram. In the case of RGB images, it is necessary to apply separation into single-component images. The Otsu’s objective function is:

$$\mathrm{J}\left(\mathrm{th}\right)=\max ({\delta }^2 (\mathrm{th}\left)\right) \mathrm{with} 0 < \mathrm{th} < \mathrm{L}-1.$$

(1)

The variance of the class is represented by ${\delta }^2$, where th stands for the threshold value. In our example, L stands for the image’s highest possible gray level, which is 255.

The initial algorithm divides images into two categories. For multi-level segmentation, it was expanded to calculate multiple thresholds [15]. Another nonparametric method for determining suitable threshold values is Kapur’s method [9]. It is based on the entropy and probability distribution of the image histogram. The approach’s objective is to identify the appropriate threshold for raising overall entropy. The degree of separation between classes in an image is indicated by its entropy. In this regard, the optimal threshold value distinguishes the classes properly when the entropy reaches its maximum value.

Multilevel threshold based on harmonic search optimization (HSO)

The problem with both bi-level and multi-level classification is determining the threshold values that correctly identify the classes. Bi-level divides the image into two classes, whereas multi-level is used to segment complex images and can produce multiple thresholds, such as tri-level or quad-level, splitting the pixels into multiple classes based on the intensity values and/or statistical parameters (like mean and variance).

The Harmony Search Algorithm (HSA) [30,38]. Because the original algorithm was designed for photonic color images, this algorithm was modified in our application to brain segmentation, as represented in Figure 2, in order to work optimally on MRI grey images. An n-dimensional real vector is used to represent each resolution, or “harmony,” in the fundamental HSA. A randomly generated starting population of harmony vectors is stored in harmony memory (HM). Thus, a new candidate harmony is made from the elements in the HM using a memory consideration method, either by a random re-initialization or a pitch adjustment operation. The HM is eventually updated by comparing the new candidate harmony to the HM’s worst harmony vector. The poorest harmony vector is replaced when the new candidate vector offers a more advantageous solution in the HM. The aforementioned procedure is repeated until a particular termination condition is met. The initialization of the HM, the creation of new harmony vectors, and the updating of the HM are the three main phases of the basic HS algorithm. The optimization technique known as harmony search optimization (HSO) uses k various factors as decision variables. The recommended segmentation method and two different objective functions, the Otsu and Kapur functions, as discussed in the preceding section, were combined to generate two distinct segmentation algorithms. Ref. [30] provides instructions on how to use both strategies. The algorithm and its parameters have been modified in order to optimize the segmentation of MRI brain tumor pictures in this study.

The following developed algorithm is presented as a flowchart in Figure 2.

The performance of the HSO method is significantly influenced by the values given to parameters. As these parameters interact in a highly nonlinear manner and there are currently no mathematical models that capture this interaction, determining the most suitable parameter values for each specific circumstance is a challenging process. Finding the ideal set of parameter values frequently involves fixing each parameter value to a random number within the parameter limits, followed by running HSO. If the result is not what is desired, new parameter values are defined, and the evolutionary algorithm is rerun. This process takes a while because it could take multiple experiments to arrive at a set of acceptable parameter values. The user’s chosen set of values may also not be the best possible but rather the best among an arbitrarily large number of trials. In order to reduce the number of experiments, the factorial design method first described in [38] was employed in this research to systematically identify the best HSO settings.

2.2.3. Morphological Operations

Morphological operations are mathematical procedures for extracting picture elements such as regions and boundaries. The retrieved zones produced using simple threshold or clustering algorithms are defective in the great majority of cases. As a result, morphological techniques can be employed as post-processing techniques to reduce noise or refine the structure of a complicated item [10]. In addition, one of the most common applications of morphological processes is image segmentation based on object form. Morphological methods link a picture to a limited number of points called structuring components. The structural element can be any shape or size, with the output image being the same size as the input [39].

In order to improve the object structure, morphological techniques such as thinning, thickening, and object marking can be used. Morphological image processing techniques are based on some operations called dilation, erosion, opening, and closing. Dilation is a basic morphological action that causes a foreground object to increase in size by expanding the outer layer of the object, adding pixels to the contour of the object. The erosion process, on the other hand, reduces the size of the object item. Our method involves performing morphological operations on binary pictures obtained after applying a multilevel threshold, with white pixels representing the tumor and black pixels representing the background. After multiple tests, the morphological operation chosen was three iterations of “erosion” followed by three iterations of “dilation” to preserve the original size of the tumor in the brain.

2.2.4. Connected Component Analysis

After morphologically analyzing the image, we found that there were a few little circular objects there that had nothing to do with the tumor. In order to deal with these artifacts, we employed a MatLab function called “connected component,” which automatically deletes objects that are smaller than a user-specified area size. The user can optionally supply the desired tiny area size to be deleted from the application. It was utilized to get rid of the noise pixels after the morphological process. The only thing visible in the photograph following this procedure is the brain tumor.

2.2.5. Performance Evaluation Metrics

Some of the evaluation metrics used to evaluate the brain tumor segmentation model’s performance include accuracy, sensitivity, specificity, and Dice Coefficient [40,41]. These measures are based on the values of the following four factors (True Positive, True Negative, False Positive, and False Negative).

3. Results

Figure 3 shows our multi-level thresholding segmentation method on a database of T2-weighted normal brain MR images.

The photos in Figure 3 were examined using the automatic multilayer threshold and show that the tumor is well separated from the other objects in the image. The color representation of the objects shows that in the first column of images Figure 3A, the red is primarily representing the tumor. In the second column of images, yellow is primarily representing the tumor; in the third column of images, the orange color represents the segmented tumor. Finally, in the fourth column of images, the orange represents the suspected tumor in Figure 3D.

Figure 4 shows a color-segmented image in image Figure 4C with three levels and four colors. The HSO thresholds method is used for the segmentation of the original image. The image has four colors: dark blue for the background, sky blue for brain tissue, yellow for the tumor’s surrounding area, and red for the tumor’s core.

The tumor is subsequently extracted using the component connect approach after the second step of morphological operations.

The four performance indicators were calculated (accuracy, precision, sensitivity, dice) to compare the result to the grand truth (GT), two of which are displayed in Figure 5 below.

The green-colored pixels in Figure 5 are those chosen by the proposed approach and not found in the GT. The purple pixels represent missing areas that the proposed approach did not detect.

4. Discussions

The multi-level threshold classifier’s test performance was assessed using the statistical metrics shown in Figure 6, such as accuracy, sensitivity, precision, and Dice Index. Additionally, higher accuracy and sensitivity ratings suggest better performance. Images that have been segmented (10 is represented out of 40 images). The accuracy of the entire set of photos is 99.5 percent on average, with a standard deviation of 0.07 percent. It demonstrates that the segmentation is extremely accurate. With a standard deviation of roughly 3.7 percent, the average precision obtained is around 97.5 percent. With a standard variation of roughly 3.6 percent, the average Dice result is around 87 percent. With a standard deviation of roughly 5%, the average sensitivity is around 78 percent. The global assessment of the detection of the tumor combining all four performance evaluation metrics on average is about 90.5%. Figure 6 illustrates the promising performance of our segmentation technique. For a radiologist or clinical practitioner, even a small increase in the sensitivity parameter is crucial for surgical planning. To compare our results to the state-of-the-art results. The selection of those methods was driven by the fact that they took part in the international “BraTS-2017-2021” challenge and utilized datasets from that competition. These datasets were manually segmented by subject-matter experts, and the best segmentation results were then made public. Other methods were excluded because they use different datasets and/or do not calculate the Dice Index. Different methods, including neural network and deep learning methods (Single path MLDeepMedic, U-Net, Rescue Net, and Cascaded Anistropic CNN) and a classical segmentation method were used in this comparison. The comparison is presented in

Table 1. Performance comparison between our proposed method and different approaches on the BraTS dataset [32].

Methods	Average Performance of Dice Index	Average Execution Time
Single path MLDeepMedic	79%	~6 h
U-Net	80%	~6 h
Rescue Net	95%	~6 h
Cascaded Anistropic CNN	87%	~5 h
K_Mean and FCM	57%	~45 s
Proposed method “Multilevel HSO”	87%	~2 min

below. The average performance of the Dice index and average time of execution are shown. The dice index was chosen because of its widespread use in the literature and the availability of Dice results for all compared methods. Although all of the compared methods use the same datasets, the number of photos in our experimental data differs from the number of images in the other approaches. The time for CNN and DLA was estimated using about 370 images for training on HP, core I7 computer, and using 40 iterations. The average dice index of 40 photographs is compared with others which are statistically representative of the entire collection of data assuming a Gaussian distribution of dice index among datasets.

Table 1 shows that our suggested method outperforms the unsupervised methods “K mean and FCM.” It is comparable to the “Cascaded Anistropic CNN,” and is somewhat better than “Single path ML DeepMedic” and “UNet.” It shows less performance than the Rescue Net approach, which is a supervised deep learning method that requires several hours of time and a huge number of images for the learning process as well as a lot of computer capacity. Our approach is also capable of accurately segmenting entire tumor regions, as seen in Figure 3 and Figure 4.

5. Conclusions

The results of the suggested approach can be acquired in about two minutes, and it has been demonstrated that it outperforms Fuzzy clustering unsupervised segmentation and is on par with other neural networks methods such as U-Net and Cascaded Anisotropic CNN. It has slightly poorer performance than the Rescue Net approach; all deep learning methods require a lot of learning which takes several hours of processing time as well as a large dataset for learning. It also requires a lot of CPU power and memory. The power of the proposed method is to provide real-time results comparable to CNN and deep learning, which makes the proposed method promising in terms of assisting clinicians in fast and accurate diagnosis, the location and size of MRI brain tumors in two minutes instead of waiting for hours. Furthermore, it is quasi-automatic and does not require training. The only parameter fixed is the number of levels required for the segmentation. The comparison of the results with other methods published recently (2021) in the literature makes the proposed method promising as an AI application for brain tumor detection.

The future direction is to explore and add methods based on each pixel in the region of interest as a refinement of the segmentation in order to improve the accuracy and the Dice index of brain detection for more accurate diagnosis.