*Article* **A Survey of Brain Tumor Segmentation and Classification Algorithms**

**Erena Siyoum Biratu 1, Friedhelm Schwenker 2,\*, Yehualashet Megersa Ayano 3, Taye Girma Debelee 1,3**


**Abstract:** A brain Magnetic resonance imaging (MRI) scan of a single individual consists of several slices across the 3D anatomical view. Therefore, manual segmentation of brain tumors from magnetic resonance (MR) images is a challenging and time-consuming task. In addition, an automated brain tumor classification from an MRI scan is non-invasive so that it avoids biopsy and make the diagnosis process safer. Since the beginning of this millennia and late nineties, the effort of the research community to come-up with automatic brain tumor segmentation and classification method has been tremendous. As a result, there are ample literature on the area focusing on segmentation using region growing, traditional machine learning and deep learning methods. Similarly, a number of tasks have been performed in the area of brain tumor classification into their respective histological type, and an impressive performance results have been obtained. Considering state of-the-art methods and their performance, the purpose of this paper is to provide a comprehensive survey of three, recently proposed, major brain tumor segmentation and classification model techniques, namely, region growing, shallow machine learning and deep learning. The established works included in this survey also covers technical aspects such as the strengths and weaknesses of different approaches, pre- and post-processing techniques, feature extraction, datasets, and models' performance evaluation metrics.

**Keywords:** brain tumor; classification; segmentation; region growing; shallow machine learning; deep learning

## **1. Introduction**

Machine learning has been applied in different sectors, the majority of the studies indicate that it was applied in agriculture [1], and health sectors [2,3] for disease detection, prediction, and classifications. In health sectors the most researched areas are breast cancer segmentation and classification [4–7], brain tumor detection and segmentation [8], and lung and colon cancer segmentation and classification [3].

The gold standard in brain tumor diagnosis is biopsy which includes resection and pathological examination using various cellular (histologic) examination techniques. However, the diagnosis using biopsy is invasive that may result in bleeding and even injury that results in functional loss [9]. As a result, non-invasive brain tumor diagnosis using magnetic resonance imaging is the mainstay of modern neuroimaging that enables physician to characterize structural, cellular, metabolic, and functional properties of brain tumor [9,10].

In a conventional structural MRI scan, a healthy brain contains white mater (WM), gray matter (GM), cerebrospinal fluid (CSF) [11]. The main variation of these tissues in a structural MRI scan depends on their water content. The white matter (WM), which is 70% water, is a myelinated axon that connects the cerebral cortex with other brain regions. Furthermore, it carries information between neurons and connects the right and left hemispheres of the brain. The gray matter, which is 80% water, contains neuronal and glial cells that control brain activity, and the basal nuclei which are located deep within

**Citation:** Biratu, E.S.; Schwenker, F.; Ayano, Y.M.; Debelee, T.G. A Survey of Brain Tumor Segmentation and Classification Algorithms. *J. Imaging* **2021**, *7*, 179. https://doi.org/ 10.3390/jimaging7090179

Academic Editors: Leonardo Rundo, Carmelo Militello, Vincenzo Conti, Fulvio Zaccagna and Changhee Han

Received: 29 June 2021 Accepted: 28 August 2021 Published: 6 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the white matter. Whereas, the cerebrospinal fluid is almost 100% water, and fills the space between the infoldings of the brain, between the brain and skull, and between the ventricular system in the brain[11,12].

Clinically, due to the variability in size, locality, rate of growth, and pathology, it is difficult to understand the manifestation of a brain tumor. However, a brain tumor is an abnormal mass of tissue, in which some cells grow and multiply uncontrollably. This uncontrollable growth takes up space within the skull and interferes with normal brain activity and damages the brain cells. The damage may be caused through increasing pressure in the brain, by shifting the brain or pushing against the skull, and by invading nerves and healthy brain tissues [13,14]. Different criteria can be used to classify brain tumor. A layered based tumor classification schema that has been proposed by WHO provides a detailed classification techniques that is more pertinent to radiological use. In this schema the hierarchy from top to bottom four layers, that are, final integrated diagnosis, histologic classification, WHO grade, molecular information [15]. However, brain tumors can be more generally grouped into primary and secondary (metastatic) tumors depending on their place of origin [16]. Primary brain tumors originates in the brain itself and are named for the cell types from which they originated. These primary tumors can be benign (non-cancerous) and malignant (cancerous). Benign tumors grow slowly and do not spread elsewhere or invade the surrounding tissues. However, they can put pressure on the brain and compromise its function. On the contrary, the malignant tumors grow rapidly and spread to surrounding tissues. On the other hand, secondary brain tumors originate from another part of the body. These tumors mainly occur due to cancer cells from somewhere else in the patient's body that spread to the brain. The most common causes of secondary brain tumors are lung cancer, breast cancer, melanoma, kidney cancer, bladder cancer, certain sarcomas, and testicular and germ cell tumors [13,16,17]. Each of these tumors has unique clinical, radiographic, and biological characteristics [13].

In MRI scanning, brain examination can be normal or abnormal. The normal brain tissues in MRI are characterized by gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) tissues. Apart from the normal tissues listed earlier the tumorous brain scan often contains core tumor, necrosis, and edema. Necrosis is a dead cell located inside a core tumor, while edema is located near active tumor borders. Edema is a swelling that exists due to trapped fluids around a tumor. It can be vasogenic in non-infiltrative extra-axial tumors, such as meningioma, or it can be infiltrative that invades WM tracts of a brain in tumors, such as glioma [10,18]. Furthermore, these tissues often have indistinguishable intensity features in structural MRI sequences, such as T1-w, T2-w, FLAIR. For instance, the difficulty in differentiating between the core tumor and associated inflammation was discussed [19]. In addition to that, Alves et al. [19] demonstrated the difficulty in differentiating tumors using signal intensities alone. They demonstrated using a case where two patients were diagnosed with two different brain tumor types due to both tumors have similar intensity features and both are surrounded by extensive edema.

## *1.1. Brain Tumor Imaging Modalities*

There are a variety of imaging techniques used to study brain tumors, such as magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and single-photon emission computed tomography (SPECT) imaging. However, CT and MR imaging are the most widely used techniques, because of their widespread availability and their ability to produce high-resolution images of normal anatomic structures and pathologies [20].

## 1.1.1. Magnetic Resource Imaging

Magnetic resonance imaging (MRI) of a brain generates several 3-dimensional image data that comprise the three anatomical views of a brain (axial, sagittal, and coronal) at different depths of a brain. Depending on the strength of the magnetic field and the sampling protocols, the image quality, slice thickness, and inter-slice gap vary [21,22]. During MR imaging, a patient lay in a strong magnetic field, almost 10,000 times stronger than the earth's magnetic field, that forces the protons in the water molecule of the body to align in either a parallel (low energy) or anti-parallel (high energy) orientation with the magnetic field. Then, a radiofrequency pulse is introduced that forces the spinning protons to move out of the equilibrium state. When a radiofrequency pulse pauses, the protons return to an equilibrium state and produce a sinusoidal signal at a frequency dependent on the local magnetic field. Finally, a radio antenna within the scanner detects the sinusoidal signal and creates the image [22,23]. The amount of signal produced by specific tissue types is determined by their number of mobile hydrogen protons, the speed at which they are moving, the time needed for the protons within the tissue to return to their original state of magnetization (T1), and the time required for the protons perturbed into coherent oscillation by the radiofrequency pulse to lose their coherence (T2) relaxation times. As T1 (spin-lattice, also known as longitudinal relaxation) and T2 (spin-spin, also known as traversal relaxation) times are time-dependent, the timing of the radio frequency pulse and the reading of the radiated RF energy change the appearance of the image. In addition, the repetition time (TR) describes the time between successive applications of RF pulse sequences, and the echo time (TE) tells the delay before the RF energy radiated by the tissue in question is measured. The variation of T1 and T2 relaxation times between tissues gives image contrast on T1- and T2-weighted (T1-w and T2-w) images. The T1-w sequence is characterized by short TR and short TE while the T2-w sequence is characterized by long TR and short TE. Tissues with shorter T1 (for example, white matter) appear brighter when compared to tissues with a longer T1 (for example, gray matter) in magnetic resonance images. The other intermediate sequence that adopts long TR from T2-w and short TE from T1-w is a proton density-weighted (PD-w). In PD-w, the number of protons per unit volume in tissues is the main factor in determining the formation of image [23,24].

In the current neuroimaging techniques different MRI brain scan procedures can be performed, these include, the conventional structural MRI, functional MRI, diffusionweighted imaging (DWI), and diffusion tensor imaging (DTI) [10]. In structural MRI procedure which mainly differentiates healthy and abnormal brain tissues based on their water molecule content is the most commonly employed standard imaging technique. This procedure helps to visualize healthy brain tissues and to map gross brain anatomy, tumoral vascularity, calcification, and radiation-induced micro hemorrhage [10,11]. The structural sequences include T1-w, T2-w, FLAIR, and contrast-enhanced T1-w [10]. The functional MRI (fMRI) on the other hand is used to capture the neural activity inside a brain through the ratio of oxygenated to the deoxygenated level of blood in the neighboring vasculature while performing a cognitive or motor task. The fMRI is used to localize eloquent cortex and differentiate between tumor grades [10]. The DWI captures the random motion of water molecules in a brain and it is used to characterize a tumor through identification of its cellularity and hypoxia, peritumoral edema, the integrity of WM tracts, and to differentiate between posterior fossa tumors [10,25]. Whereas, diffusion tensor imaging (DTI) is used to analyze the 3D diffusion direction, also known as diffusion tensor, of the water molecule. The DTI helps to determine local effects of the tumor on white matter tract integrity including tract displacement, the existence of vasogenic edema, tumor infiltration, and tract destruction [26].

## 1.1.2. Computed Tomography Imaging

A computed tomography (CT) scan was used in neuroimaging to help understand the functional and structural status of clinically significant signs of diseases. However, it provides less information than an MRI in brain tumor diagnosis. For instance, CT is inferior to MRI in the characterization of soft tissues like a brain and its use of ionizing radiation. However, a computed tomography (CT) scan can provide more detailed images of the bone structures near a brain tumor, such as the skull or spine. A CT scan may also be used to diagnose a brain tumor if the patient has implants like a pacemaker and when an MRI is not available. Currently, a CT is commonly used in the diagnosis of diseases like

acute hemorrhage Parkinson's, head trauma, and in determining age [27,28]. Therefore, in this survey work, brain tumor segmentation and classification techniques that use the brain scan image of MRI are only explored.

The remaining part of the paper is organized as follows, Section 2 illustrates related works to this survey work and shows their strengths and limitations. In Section 3, the literature search strategy, including the chronological span, journal databases, the keywords used for search, and the inclusion and exclusion criteria, is presented. In Section 4, the commonly used model performance metrics in evaluating the performance of brain tumor segmentation and classification algorithms are highlighted. In Section 5, different region growing, conventional shallow supervised machine learning, and deep learningbased brain tumor segmentation techniques are discussed. Furthermore, the reported performances are presented. The techniques used in conventional machine learning-based brain tumor classification and their classification performance are elaborated in Section 6. In addition, different deep learning models based brain tumor classification techniques with their reported performance are presented. Finally, the paper presents a discussion on Section 7 and a conclusion in Section 8.

## **2. Related Works**

The quest to find a better autonomous brain tumor segmentation and classification technique that can aid physicians in brain tumor diagnosis have been an active research area. As a result, several survey works have been completed to foster the research in the field and recap techniques used in brain tumor segmentation and classification. In Table 1, only some of the recent pieces of literature that are related to our survey work are listed. Furthermore, their strengths and limitations are clearly discussed.


**Table 1.** Survey literature on brain tumor segmentation and classification techniques.


**Table 1.** *Cont.*

Our work is tailored to provide a comprehensive survey of recently proposed different brain tumor segmentation and classification techniques, including region growing, shallow machine learning, and deep learning. The established work in this survey also covers technical aspects, such as the strengths and weaknesses of different approaches, together with their performance.

## **3. Method**

In this survey work, peer reviewed research papers from 2015 to 2021 that were published on Scopus and Web of Science indexed journals are surveyed to investigate the region growing, deep learning based brain tumor segmentation techniques, and machine learning and deep learning based brain tumor classification techniques. The databases that are extensively searched for this survey work were: (1) IEEE Xplore Digital Library, (2) Science Direct, (3) PubMed, (4) Google Scholar, and (5) MDPI. The search criterion includes ("Brain Tumor") AND ("Region Growing") AND ("Segmentation") AND ("Deep Learning") AND ("Machine Learning") AND ("Classification"). The methodology used for selecting literature is clearly shown in Algorithm 1. In addition, the paper inclusion criteria (IC) and exclusion criteria (EC) is indicated on Table 2.


**Table 2.** Inclusion and exclusion criteria for paper selection.


## **4. Performance Measuring Metrics**

Evaluating the segmentation and classification performance of a machine learning algorithm is an essential part of a research project. A machine learning model may give a satisfying result when evaluated using a metric, for instance, accuracy score but may give poor results when evaluated against other metrics such as precision or any other metric. Therefore, most of the time various evaluation metrics are applied to measure and compare the model performance.

In a segmentation task, true positive (TP) represents a pixel that is correctly predicted to belong to the given class according to the ground truth, whereas a true negative (TN) represents a pixel that is correctly identified as not belonging to the given class. On the other hand, a false positive (FP) is an outcome where the model incorrectly predicts a pixel not belonging to a given class. A false negative (FN) is an outcome where the model incorrectly predicts the pixel belonging to a given class. Similarly, for tumor classification task, TP represents a tumor class that is correctly predicted to belong to the given class according to the ground truth whereas a TN represents a tumor class that is correctly identified as not belonging to the given class. By the same token, false positive (FP) is an outcome where the model incorrectly predicts a tumor class not belonging to a given class. A false negative (FN) is an outcome where the model incorrectly predicts the class belonging to a given class. Therefore, keeping different performance metrics used in brain tumor segmentation and classification literature are listed as follows.

Accuracy (ACC) measures the ability of a model in correctly identifying all class or pixels, no matter if it is positive or negative.

$$\text{ACC} = \frac{TP + TN}{TP + TN + FP + FN} \tag{1}$$

Sensitivity (SEN) indicates the frequency of correctly predicted positive samples/pixels among all real positive/samples. It measures the models ability in identifying positive samples/pixels.

$$SEN = \frac{TP}{TP + FN} \tag{2}$$

Specificity (SPE) is the proportion of actual negatives, which was predicted as the negative (or true negative). It tells the percentage of classes/pixels could not correctly identified.

$$SPE = \frac{TN}{TN + FP} \tag{3}$$

Recall (RE) describes the completeness of the machine learning model's positive predictions relative to the ground truth. It tells the percentage of classes/pixels annotated in our ground truth, are also included in model's prediction.

$$RE = \frac{TN}{TP + FN} \tag{4}$$

Precision (PR) also known as positive predictive value (PPV) describes how often the model predicting correct class/pixel. It tells the the correct proportion of models predicted positives.

$$PR = \frac{TP}{TP + FP} \tag{5}$$

F1-Score is the most popular metric that combines both precision and recall. It represents harmonic mean of the two.

$$F1score = 2\frac{PR \ast RE}{(PR + RE)}\tag{6}$$

Intersection over union (IoU) also known as Jaccard index (JI) measures the percent overlap between the annotated ground truth mask and the model's prediction output.

$$IoI = \frac{TP}{TP + FP + FN} \tag{7}$$

Dice similarity coefficient (DSC) measures the spatial overlap between the ground truth tumor region and the model segmented region. A zero DSC value indicates no spatial overlap between the ground truth tumor region and model annotated result whereas a value indicates a indicating complete overlap between the two.

$$DSC = \frac{TP}{\frac{1}{2}(2TP + FP + FN)}\tag{8}$$

Area under the curve (AUC) measure of the ability of a classifier to distinguish between classes and is used as a summary of the receiver characteristics curve and it is an area under true positive rate vs. false positive rate.

Similarity index (SI) refers to the similarity between the expert annotated ground truth and the model's segmentation. It describes the similar identity between the input image and the detected tumor region.

$$SI = \frac{2TP}{2TP + FP + FN} \tag{9}$$

## **5. Brain Tumor Segmentation Methods**

Brain tumor imaging using techniques, such as MRI and CT, generate a significantly large number of images. Brain MRI scan of a single individual consists of several slices across the 3D anatomical view. Therefore, manual segmentation of brain tumors from magnetic resonance (MR) images is a challenging and time-consuming task. In addition, the artifacts introduced in the imaging process results in low-quality images that make the interpretation difficult. As a result, the manual brain MRI segment is susceptible for inter and intra observable variability. To alleviate these challenges and help radiologist, different automatic brain tumor segmentation techniques have been proposed in literature.

On these literature, authors have proposed an automated system for brain tumor segmentation techniques that provides objective, reproducible segmentation that are close to the manual results. These automated brain tumor segmentation can help to alleviate the difficulties associated with manually analyzing brain tumors. This will speed-up the brain image analysis process, improve diagnosis outcome, and make easy the follow-up of the disease through evaluating tumor progression [34].

In this section, among the proposed brain tumor segmentation techniques in the literature; region growing, machine learning, and deep learning based techniques will be surveyed to identify the experimental dataset, pre-processing, feature extraction, segmentation algorithm, and the reported performance.

## *5.1. Region-Based and Shallow Unsupervised Machine Learning Approach*

One of the most commonly used segmentation techniques in automated image processing applications is region-based segmentation. Regions in an image are a group of connected pixels that satisfy certain homogeneity criteria, such as pixel intensity values, shape, and texture [35]. In a region-based segmentation the image is partitioned into dissimilar regions so that the desired region is located precisely [36]. The region-based segmentation takes into account the pixel values, such as gray level difference and variance, and spatial proximity of pixels, such as Euclidean distance and region compactness in grouping pixels together. In brain tumor segmentation, region growing, and clustering algorithms are the most commonly used region based segmentation technique.

Clustering-based segmentation is one of the powerful region based segmentation techniques where an image is partitioned into a number of disjoint groups. In clustering based segmentation pixels with high similarity categorized in a given region whereas dissimilar pixels categorized into different regions [37]. Clustering techniques, which are an unsupervised learning method, have been widely investigated in medical image segmentation. However, in this survey work some of the most popular clustering methods, such as k-means and its varieties [38–44], fuzzy c-means [38,39,41,45], subtractive clustering (SC), and hybrid techniques [46–48].

K-means clustering is an unsupervised machine learning algorithm and it is commonly used to segment a region of interest from the remaining part of an image. K-means has been extensively tested in brain tumor segmentation and has shown acceptable accuracy [48]. The minimal computational requirement [37,48], simplicity to implement on large dataset [49], adaptation to new examples, and guaranteed convergence are some of the advantages that makes K-means popular segmentation algorithm. However, k-means suffers with incomplete delineation of the tumor region [49], selection of the initial centroid is not optimum [37,43], and it is sensitive to outliers [48,50]. Due to these limitations a number of solutions have been proposed, including, evenly spreading the initial cluster centers (k-means++), hybridizing k-means with other clustering techniques [49], adaptively initializing cluster centers, such as adaptive k-means [43], modified adaptive k-means (MAKM), and histogram based k-means.

Fuzzy c-means works by assigning membership values to each of the pixels in an image corresponding to the centers of the clusters depending on a certain similarity criteria [51]. In fuzzy c-means (FCM) clustering objects can belong to more than one cluster based on its degree of membership. Therefore, in such a type of soft clustering technique, image pixels can occupy multiple clusters. As a result, compared to hard-clustering techniques such as k-means, FCM performs better on relatively noise free images. However, in medical images such as brain MRI that can be easily affected by unknown noises, the FCM performance is severely affected [52]. A number of researches have been performed to improve the limitation of FCM [53–56].

In region growing brain tumor segmentation, tissues including tumorous regions are partitioned based on certain similarity criterion, such as homogeneity, texture, sharpness, and gray levels. The technique starts by selecting an initial seed based on predefined methods. Then, the neighboring pixels are added progressively to the seed pixel [57]. The region growing based segmentation can properly segment regions with similar properties and spatially separated regions. However, it is sensitive to noise and influenced by the similarity criterion [57]. Therefore, it may end up with disconnected regions and results in a hole in the segmented region. Furthermore, finding a good initial seed is not an easy task [57]. Region growing and conventional unsupervised machine learning based brain tumor segmentation techniques proposed in literature are summarized in Table 3. The table indicates the brain MRI dataset used in the experiment, the centroid initialization techniques, the objective function, and the segmentation performance.

## *5.2. Supervised Shallow Machine Learning Based Approach*

Supervised machine learning-based brain tumor segmentation approaches transformed the image segmentation problem into a tumorous pixel classification problem. The input vector for these supervised learning models consisted of different extracted features, and the output is a vector of desired classes for segmentation. In brain tumor segmentation, where tumor regions are often scattered all over the image, pixel classification rather than classical segmentation methods are often preferable [65]. Therefore, the traditional supervised machine learning algorithms have been used in the segmentation of a brain tumor from a head MRI scan [66–76].


**Table 3.** Region growing and shallow unsupervised machine learning based brain tumor segmentation.


**Table 3.** *Cont.*

<sup>1</sup> Darwinian Particle Swarm Optimization, <sup>2</sup> Discrete Wavelet Transform.

In this section, as shown in Table 4, most relevant literature on brain tumor segmentation using traditional machine learning algorithms, such as support vector machine (SVM), artificial neural network (ANN), random forest (RF) are surveyed to identify data used, the pre-processing, feature extraction techniques, the classifier model, and whether or not post-processing is implemented.

## *5.3. Deep Learning-Based Approach*

Deep learning methodologies produce automatic features that avoid or minimize the need for handcrafted features. In the deep learning-based brain tumor segmentation approach, the general strategy is to pass an image through the pipeline of deep learning building blocks and input image segmentation is performed depending on the deep features. In literature, there are a variety of deep learning techniques proposed for segmenting brain tumors. Some of such blocks contain deep convolutional neural networks (DCNNs), convolutional neural network (CNN), recurrent neural networks (RNNs), long short-term memory (LSTM), deep neural networks (DNNs), deep autoencoders (AEs), and generative adversarial networks (GANs). In this section, literature in terms of these building blocks, the dataset used, and the reported performance are presented as shown in Table 5.

**Table 4.** Summary of a shallow machine learning based segmentation.



<sup>1</sup> Simple Linear Iterative Clustering, <sup>2</sup> Concatenated and Connected Random Forest, <sup>3</sup> Multiscale Patch Driven Active Contour, <sup>4</sup> Random Decision Forest.




**Table 5.** *Cont.*

<sup>1</sup> Heterogeneous CNN + Conditional Random Fields-Recurrent Regression based Neural Network, <sup>2</sup> Deep Residual Dilate Network with Middle Supervision, <sup>3</sup> Fully Convolutional Neural Network, <sup>4</sup> OCcipito Module, <sup>5</sup> Atrous-Convolution Feature Pyramid.

## **6. Brain Tumor Classification Methods**

Based on the WHO's classification of central nervous system (CNS) tumors, there are more than 150 types of CNS tumors that are mainly categorized into primary and metastatic (secondary) tumors [99]. The primary tumors originate from the brain or the immediate surrounding tissues. Whereas, metastatic tumors arise from other body parts and migrate to the brain through the bloodstream. Metastatic tumors are considered cancerous or malignant, while primary tumors can be benign or malignant.

A biopsy is the existing gold standard procedure in brain tumor classification. However, it usually requires definitive brain surgery to take a sample [100,101]. On the other

hand, an automated brain tumor classification from an MRI is non-invasive so that it avoids tumor sample taking procedure and it is safer. In addition, the machine learning-based brain tumor classification from an MRI scan can improve the diagnosis and treatment planning [101]. As a result, an automatic brain tumor classification from MRI images using machine or deep learning techniques is an active research area, and promising results have been achieved [100,102–106].

## *6.1. Conventional Machine Learning Based Approach*

Machine learning is a paradigm where a machine is given a task where its performance improves with experience. Machine learning techniques are commonly grouped into three major types: supervised, unsupervised, and reinforcement learning [107]. Supervised learning is based on training a data sample from the data source with correct classification already assigned by domain experts, whereas, in unsupervised learning, the algorithm finds hidden patterns from the unlabeled data. On the other hand, reinforcement learning is carried out by making a sequence of decisions using reward signals. Therefore, the algorithm learns through receiving either rewards or penalties for the actions it performs [107]. Machine learning has been used in the classification of brain tumors from MRI images, and promising classification performance has been reported [108–115].

The traditional machine learning-based brain tumor classification techniques often consist of preprocessing, segmentation, feature extraction, and classification stages.

## 6.1.1. Pre-processing

Brain MRI scans are significantly affected by different types of noises, including salt and pepper, Gaussian, Rician, and speckle noise [116–118]. These noises impose challenges in machine learning-based applications [117,119]. Therefore, obtaining high-quality image denoising is one of the important tasks in the pre-processing stage. Each method used in MRI denoising has its advantages and disadvantages. Several methods have been developed for reducing noises based on statistical property and frequency spectrum distribution [119]. In addition to denoising, tasks such as removing tags, smoothing the foreground region, intensity inhomogeneity correction, maintaining relevant edges, resizing, cropping, and skull stripping are part of pre-processing [110–112].

## 6.1.2. Region of Interest (ROI) Detection

In an MRI brain scan, the segmentation task labels each voxel in an MRI image to specify its tissue type and anatomical structure [119]. The objective of ROI detection in tumor classification is to locate the tumor region from an MRI scan, improve the visualization, and allow quantitative measurements of image structures in the feature extraction stage [108,112]. Brain tumor segmentation can be performed in three different ways, namely, manual segmentation, semi-automatic segmentation, and fully automatic segmentation [119]. The autonomous brain segmentation techniques have been briefly discussed in Section 5.

## 6.1.3. Feature Extraction

The feature extraction techniques are mathematical models based on various image properties. The different types of features include texture, brightness, contrast, shape, Gabor transforms, gray-level co-occurrence matrix (GLCM), and wavelet-based features [115,120], histogram of local binary patterns (LBP) [121]. On the other hand, recently, deep features that are obtained from deep neural networks such as CNN have been used as input to SVM classifier to classify brain tumors [122]. In brain tumor classification, it is customary to fuse several features from different extraction models to improve the discrimination power of the machine learning model [123]. Furthermore, feature selection is applied for dimensionality reduction.

## 6.1.4. Classification

Different classification techniques have been proposed by many authors for identifying tumor types from brain images. Different authors have classified tumor into a variety of ways, for instance meningioma, glioma, and pituitary [109,121,122,124,125]; astrocytoma, glioblastoma, and oligodendrogliamo [112]; glioma tumor grades (I–IV) [113]; benign and malignant stages(I–IV) [126–129]; diffuse midline glioma, medulloblastoma, pilocytic astrocytoma, and ependymoma [102]; multifocal, multicentric, and gliomatosis [130]; ependymoma and pilocytic astrocytoma [120].

In brain tumor classification, the most commonly used classifiers are neural network [108–111,131], support vector machines (SVM) [108,115,124,127–130,132,133], Knearest neighbor (KNN) [112,121,130,134], Adaboost [126], and hybrid models [113,135,136]. The neural network was implemented using different architectures, such as feedforward neural network [110,125], multilayer perceptron neural network [109,137], and probabilistic neural network (PNN) [111,131]. Support vector machine (SVM) was commonly implemented using three kernels, linear, homogeneous polynomial, and Gaussian radial basis function (RBF) [108,115]. In the KNNclassifier, the testing feature vector is classified by finding the k-nearest training neighbor, that is, the classifier does not use any model to match and is only based on memory. However, KNN uses different measurements such as euclidean distance, city block, cosine, and correlation to find the nearest distance between the testing and training class feature vectors [134].

A summary of recent shallow machine learning-based brain tumor classification techniques is given on Table 6.

## *6.2. Deep Learning Approach*

Even though promising progress has been made in classifying brain tumors into their respective types from an MRI brain scan using shallow supervised machine learning algorithms, there are still challenges in classifying brain tumors from an MRI scan. These challenges are mainly due to the ROI detection, and extracting descriptive information using traditionally handcrafted feature extraction techniques is not efficient [122]. This inefficiency mainly arises due to the complex structure of brain anatomy and the highdensity nature of the brain.

Unlike shallow machine learning algorithms, deep learning is based on learning data representations and hierarchical feature learning. In deep learning-based brain tumor classification, the deep learning models discover the descriptive information that optimally represents different brain tumors. This nature of deep learning transforms the brain tumor classification from handcrafted feature-driven into data-driven problem [103]. Among the deep learning models, a convolutional neural network (CNN) is widely used in brain tumor classification tasks, and a substantial result has been achieved [100].

In the reviewed literature, there are differences in the techniques used for the classification of brain tumors. The difference encompasses: (i) the dataset used for classification including tumor types, (ii) the implemented pre-processing and data augmentation techniques, (iii) whether or not the ROI segmentation was used as a prior step in the classification, (iv) whether a pre-trained or custom-designed deep learning model is used.

*J. Imaging* **2021**, *7*, 179





1 Principal Component Analysis, 2 Particle Swarm Optimization, 3 Regularized Extreme Learning Machine.

For instance, Badža and Barjaktarovi´c [100] used publicly available contrast-enhanced T1-weighted brain tumor MRI scans [138]. The dataset contains meningioma, glioma, and pituitary brain tumor types scanned along with the three anatomical views, i.e., axial, sagittal, and coronal. The images were preprocessed using techniques, such as normalization and resizing. In addition, images in the dataset are augmented with 90*<sup>o</sup>* rotation and vertical flipping to increase the training dataset. Furthermore, they used a custom-designed CNN model trained with Adam optimizer with a mini-batch size of 16 and tested with 10—fold cross-validation. The weights of the convolution layers are initialized using a Glorot initializer. The model performance was measure using sensitivity, specificity, accuracy, precision, recall, and F1-score. The sensitivity for meningioma, glioma, and pituitary is 89.8%, 96.2%, and 98.4%, respectively. The specificity of the model for meningioma, glioma, and pituitary is 90.2%, 95.5%, and 97.7%, respectively. Furthermore, the models' overall accuracy, average precision, average recall, and F1-score are 95.4%, 94.81%, 95.07%, and 94.94%, respectively. The summary of this and other literature is presented on Table 7.

**Table 7.** Summary of deep learning based brain tumor classification techniques.



**Table 7.** *Cont.*


<sup>1</sup> Convolutional Neural Network based on Complex Networks, <sup>2</sup> Confidence Interval, <sup>3</sup> Automated Machine Learning, <sup>4</sup> Matthew's Correlation Coefficient, <sup>5</sup> Visual Geometry Group, <sup>6</sup> Deep Dense Inception Residual Network, <sup>7</sup> High Grade Glioma, <sup>8</sup> Low Grade Glioma.

## **7. Discussion**

This paper presented a thorough survey of techniques used in brain tumor segmentation and classification. The survey encompasses several traditional machine learning and deep learning-based methods with their quantitative performance. The conventional image segmentation techniques, that is, region growing and unsupervised machine learning used in brain tumor segmentation are presented in Table 3. The region growing with all other conventional image processing segmentation techniques is the earliest approach applied in brain tumor segmentation [161]. It is mainly affected by noises, poor image quality, and initial seed point. To overcome these challenges, an automatic seed point selection by optimization techniques and artificial intelligence-based seed point selection has been proposed [162]. In addition, it has a limitation in segmenting tumors that appear scattered across the brain. In the second generation segmentation techniques which are based on shallow unsupervised machine learning, such as fuzzy c-means and k-means grouping of pixels into more than one class has been achieved. However, these methods are also highly sensitive to noise. Therefore, through incorporating additional information and adaptively selecting the centroid, the segmentation performance of medical images can be improved [6]. In addition, the inherent ambiguous boundaries between normal tissues and brain tumors pose a significant challenge for conventional and clustering segmentation techniques. Therefore, to address this challenge, pixel-level classification-based segmentation techniques using traditional supervised machine learning have been proposed [70]. These methods are often accompanied by feature engineering, where the tumor descriptive pieces of information are extracted to train the model. Furthermore, the supervised machine learning segmentation output is further improved through post-processing [71,76].

Nowadays, conventional image processing and shallow machine learning-based brain tumor segmentation techniques are becoming obsolete due to the advent of deep learningbased techniques. The deep learning-based approach performs an end-to-end tumor segmentation by passing an MRI image through the pipeline of its building blocks. These models often extract tumor descriptive information automatically and avoid the need for handcrafted features. However, the need for a large dataset to train the models and the difficulty in interpreting the models hinders their usage in medical fields [163]. In terms of segmentation performance, it is evident from Tables 4 and 5 that the deep learning-based

**Table 7.** *Cont.*

and supervised shallow machine learning-based with post-processing has comparable performances. Asummary of the number of brain tumor segmentation techniques surveyed in this is given on Figure 1.

Aside from segmentation of brain tumor region from head MRI scan, classification of tumor into their respective histological type has great importance in diagnosis and treatment planning which actually requires biopsy procedure in today's medical practice [158]. Several methods which encompass shallow machine learning and deep learning have been proposed for brain tumor classification. The conventional shallow machine learning algorithms often consist of preprocessing, ROI detection, and feature extraction. However, due to the inherent noise sensitivity of MRI image acquisition, variations in the shape, size, location, and contrast of tumor tissue cells, extracting descriptive information is a challenging task. Therefore, nowadays, deep learning techniques are becoming the state-of-the-art approach to classify different types of brain tumors, such as astrocytoma, glioma, meningioma, and pituitary. Several brain tumor classifications have been discussed in this survey, and a summary of the number of brain tumor classification techniques surveyed in this paper are given on Figure 2.

Several brain tumor datasets that are collected by researchers datasets and those that are available on repositories were used in the training and testing of brain tumor classification models. The publicly available dataset provided by J. Cheng et al. [138], which contains meningioma, glioma, and pituitary tumor in T1-WC MRI-images is one of the most commonly used datasets in the training and testing classifier models. Using this dataset, Gumaei, A. et al. [125] has achieved a classification accuracy of 94.23% using a regularized extreme learning machine, while the Kokkalla, S. et al. [153] have reported a classification accuracy of 99.69% using custom modified deep-dense inception residual

network (DDIRNet). These results indicate that the deep learning-based model outweighs the shallow machine learning-based techniques for this particular dataset.

**Figure 2.** Number of brain tumor classification methods.

## *Challenges in Automatic Brain Tumor Segmentation and Classification*

The development of autonomous brain tumor segmentation and classification models using MRI images is still a challenging task. The challenges are due to several constraints including the effect of different types of noises embedded in the brain MRI images [116–118], motion and metal artifacts during image acquisition [164], low-resolution MRI images [165], and lack of deep learning models interpretability and transparency [166,167].

One of the most common challenges in machine learning-based brain tumor segmentation and classification is the noisiness of an MRI image. Therefore, noise estimation and denoising MRI images is a crucial pre-processing task for improving the accuracy of brain tumor segmentation and classification models. Therefore, several techniques have been proposed for denoising MRI images, such as modified iterative grouping median filter [118], Wiener filter and wavelet transform [168], non-local means [169], and deep learning-based approaches [170,171]. However, a robust denoising technique for MRI images is still challenging and the pursuit to obtain an efficient denoising technique has been an active research area [170]. Similarly, motion, metal, and other artifacts are also a source of challenge to the robustness of machine learning-based brain tumor segmentation and classification. Recently, deep learning-based solutions for minimizing the effects of these artifacts have been proposed [164,172]. MRI provides a high fidelity brain scan image compared to other imaging techniques. However, post-acquisition image processing techniques, including deep learning-based methods have been used to increase the resolution of MR images so that the efficiency of autonomous brain tumor segmentation and classification models improved[165,173]. The other major challenge is the lack of deep models' interpretability, and often they are perceived as black-box. As a result, attaining any evidence regarding the process they perform is difficult. However, the transparency and interpretability of deep learning techniques are crucial for the complete integration into medical diagnosis [166].

## **8. Conclusions**

Automating the brain tumor segmentation and classification task has tremendous benefits in improving the diagnosis, treatment planning, and follow-up of patients. Through applying various techniques, including conventional image processing, shallow machine learning, and deep learning techniques, undeniable progress have been achieved in automating brain tumor segmentation and classification tasks. However, building a fully autonomous system that can be used on clinical floors is still a challenging task.

Compared to region-growing and shallow machine learning algorithms, automating the brain tumor segmentation and classification using deep learning techniques have huge benefits. This is mainly due to the powerful feature learning ability of deep learning techniques. In addition, as can be shown in Figures 1 and 2, deep learning-based brain tumor segmentation and classification techniques are becoming the most active research area. In this paper, a comprehensive survey on region growing, shallow machine learning, and deep learning-based brain tumor segmentation and classification methods are presented. These methods are structurally categorized and summarized to give an insight to the reader of the dataset used, pre-processing, feature extraction, segmentation, classification, post-processing, and the reported model performances in the literature. Furthermore, the pros and cons of the methods and the model evaluation metrics have been discussed.

**Author Contributions:** Conceptualization, E.S.B.; Methodology, E.S.B.; Validation, Y.M.A., F.S., T.G.D.; Writing—original draft preparation, E.S.B.; Writing—review and editing, E.S.B., Y.M.A., F.S., T.G.D. All authors have read and agreed to the published version of the manuscript

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**

	- [CrossRef]
