This research presents a comprehensive overview of the application of Convolutional Neural Networks (CNNs) in brain tumor detection. It encompasses background information on meningiomas, gliomas, and pituitary tumors, and conventional, machine learning and deep learning approaches. The research explores the different brain tumor types, including gliomas, meningiomas, and pituitary tumors, highlighting their distinct characteristics and challenges associated with their detection. It further delves into traditional techniques employed in brain tumor diagnosis, such as manual segmentation and feature extraction. This is followed by examining machine learning approaches that utilize handcrafted features for classification tasks. Recently, deep learning methods like CNNs have shown remarkable performance in several medical image analysis tasks. A case study of the VGG-16 model is presented to exemplify the application of CNNs in brain tumor detection. Furthermore, the paper discusses how the lack of appropriate databases and ethical considerations hinder the widespread adoption and commercial utilization of CNNs in the medical field.CNNs have demonstrated impressive performance in various medical image analysis task. This paper highlights the need for further research to address the challenges associated with the integration of CNNs into clinical practice, aiming to enhance their reliability, interpretability, and regulatory compliance. The key takeaways from this work are CNNs’ potential to revolutionize brain tumor detection and the need for further research to integrate CNNs into clinical practice. This work sets itself apart by combining an in-depth analysis of brain tumors with an explanation of CNN applications.
Keywords: Convolutional Neural Networks (CNNs), brain tumor, detection, classification
This paper attempts to address all facets of the growing field of deep learning methods being used in medical imaging. Brain tumors present a serious medical issue, and it is essential to a patient’s long-term outcome that they are detected early, as conventional methods relying on manual segmentation and feature extraction are time-consuming and subjective. This research aims to explore the potential of CNNs to automatically extract meaningful features and learn complex patterns from medical images, thereby enhancing the accuracy and efficiency of brain tumor detection.
Deep learning methods, specifically CNNs, have demonstrated impressive performance in various medical image analysis tasks. To illustrate their potential in brain tumor detection, the paper presents a case study of the VGG-16 model, showcasing its application and emphasizing its effectiveness in automatically extracting relevant features from medical images for the purpose of diagnosis. Nevertheless, the widespread implementation and commercial utilization of CNNs in the medical field face several challenges. These challenges include the limited interpretability of deep learning models, the necessity for large, annotated datasets, and ethical considerations. Addressing these challenges is critical for the use of CNN-based diagnosis systems in clinical practice.
Brain tumours are a heterogeneous and complex group of neoplasms that pose significant diagnostic and therapeutic challenges. MRI (Magnetic Resonance Imaging) is a commonly used modality for detecting and characterising brain tumours. However, manual comprehension of MRI scans can be time-consuming and subject to observer variability. CNNs are a promising tool for automating the detection, segmentation, and classification of brain tumours in MRI scans accurately and quickly.
Amin, et al., provides a complete survey of machine learning techniques for brain tumor detection and classification1. Machine learning methods like feature extraction, feature selection, dimensionality reduction, and classification algorithms are covered, as well as traditional approaches like support vector machines (SVM), random forests, and neural networks. Deep learning and CNNs are also covered. The authors highlight the importance of pre-processing brain tumor images using image segmentation and normalization, as well as the challenges posed by class imbalance, limited datasets, and how machine learning models are interpreted. The publication also notes the gaps in this field, crucially the need for more standardized datasets that encompass a wide range of brain tumor types, sizes, and locations.
This survey acts as a foundational resource that encapsulates the evolution of machine learning in brain tumor detection. It provides a backdrop against which this research can be understood in a broader context.
Sharma, et al., presents a CNN model for the detection of brain tumors from MRIs aiming to overcome the limitations of manual tumor segmentation2. The researchers used a dataset obtained from an internet database, consisting of 253 MRI images. The authors used a CNN-based approach for feature extraction and segmentation, achieving high accuracy. The research reports an accuracy rate of 97.79% on the dataset.
Unfortunately, the reasearch does not provide detailed information about the specific CNN architecture and parameters used, limiting its reproducibility.The authors could have also discussed the limitations of the dataset they chose, such as the potential biases or variations present. While Amin et al., gives a broad overview of the field, Sharma et al., shows the practical results of a specific use of CNNs.
Background on Brain Tumors
Brain tumors are imaged using MRIs. It is a medical imaging technique that incorporates orating radio waves and magnets to construct images of the internal structures of the body. MRIs are useful in diagnosing and monitoring brain tumors because they produce high-resolution pictures of the brain. A patient is placed inside a large, cylindrical machine which generates a very strong magnetic field. This field causes the hydrogen atoms in the body to align with it. Radio waves are used to disturb this alignment, and when the radio waves are turned off, the hydrogen atoms emit signals that are detected by the machine. These signals, once processed, form cross-sectional images of the brain. The following figure shows MRI scans of meningiomas, gliomas, and pituitary tumors in various areas of the brain3.
Meningiomas are slow-growing tumors that form on meninges; membranes that enclose the brain and spinal cord. These types of tumors can compress adjacent regions of the brain like tissue, nerves, or blood vessels and are therefore categorized as brain tumors. : Meningiomas account for 40% of primary brain tumors in the United States. In 2023, an estimated 42,260 people will be diagnosed with meningioma. Most cases are diagnosed in adults age 65 and older, and the disease is rarely found in children.
Gliomas are tumors that begin in the glial cells of the central nervous system. Glial cells provide support to the nerve cells, called neurons. Gliomas constitute about 33% of all brain tumors. Astrocytomas, which develop from astrocytes, are the most common primary intra-axial brain tumor, accounting for nearly half of all primary brain tumors. Ependymomas and oligodendrogliomas are less common, accounting for 2-3% and 2-4% of primary brain tumors, respectively????. The diverse types of gliomas, ranging from astrocytomas to oligodendrogliomas, each with distinct characteristics and locations in the brain, necessitate varied detection strategies4.
Pituitary tumors develop in the pituitary gland, which controls hormone production and a range of bodily functions. Diagnosing these tumors generally requires imaging tests like MRIs or CT scans in tandem with a hormonal assessment. Treatment may involve medications to manage hormone levels, surgery to remove it, radiation therapy, or any combination thereof. In some cases, simple monitoring without active treatment may be recommended. Pituitary gland tumors make up about 17% of all primary brain tumors. Less than 0.2% of these tumors will be cancerous. The fact that a significant portion of these tumors is non-cancerous and their occurrence across various ages highlights the need for tailored detection strategies5.
Conventional Methods of Detection
Segmentation involves extracting the required region from the inputted images, and it is especially important to accurately identify the lesion regions. Manual segmentation is prone to errors, so semi- and fully automated-methods are used instead. Semi-automated methods are preferred over manual segmentation as they can achieve acceptable outcomes in segmenting tumour regions.
Conventional methods for brain tumour segmentation can be categorised as thresholding, region growing, and watershed methods.
Thresholding is a basic yet powerful technique for segmenting objectsThis method is used to choose threshold values based on image intensity. The optimal threshold value is determined using the Gaussian distribution method. Thresholding is typically used at the first stage of segmentation and can segment several regions within grey-scale images. However, selecting the right threshold is challenging in images with low contrast.
Region Growing Methods
Region growing (RG) methods segment the pixels of an image by analyzing neighbouring pixels with similar characteristics, based on predefined similarity criteria. However, the partial volume effect can limit the accuracy of these methods. The partial volume effect in medical imaging refers to the phenomenon where a single voxel (the smallest unit of a 3D image) contains multiple tissue types, leading to inaccurate or mixed intensity values in the image.
Watershed methods are used for the analysis of the intensity of MRIs as they have more proteinaceous fluid intensity. However, these methods may result in over-segmentation due to noise. To obtain accurate segmentation results, the watershed transform can be combined with statistical methods. Some common watershed algorithms include topological watershed, image foresting transform (IFT) watershed, and marker-based watershed.
Automated Detection Methods
Machine learning methods
Machine learning is a sub field of artificial intelligence while deep learning is a sub field of machine learning.
Data is collected first. This data, often vast and unstructured, undergoes a critical phase of preprocessing to refine and prepare it for further analysis. In this stage, irrelevant information is filtered out, ensuring that only the most pertinent features are retained in each image. Extracted features commonly are shape, size, texture, intensity, and edge information of the tumor. They are crucial as they help differentiate tumor tissue from normal brain tissue. Once the right model is selected it is meticulously trained; it learns and adapts to data patterns. This training is followed by rigorous testing and validation, when the model’s accuracy and reliability are determined in various scenarios.
Deep Learning Methods
Deep learning is a branch of machine learning centered around training deep convolutional neural networks (CNNs) to learn abstract representations from raw data. This makes them well-suited for sophisticated pattern recognition and working with large datasets, such as those used in medical imaging applications like brain tumor sorting.
Instead of manually extracting features from the data, this approach lets deep learning models process the raw medical images, like MRI scans, directly.
The CNN type of neural networks are specifically designed for analyzing visual informationand have been used widely in computer vision projects that involve detecting tumors in medical images.
Due to their feature extraction power, convolutional neural networks (CNNs) can be used to recognize various intricate patterns and features in medical scans. When trained, they can scan new images for potential tumor regions or accurately predict the type of tumor present, making them a valuable tool for radiologists.
Deep learning algorithms have already had success in improving brain tumor spot detection from MRI scans.
Here’s how it works within this context:
Before putting the MRI scans into the CNN, there are various steps taken to make sure the images are of suitable quality. This includes normalizing them, reducing noise, ensuring intensity is correct, and registering the images so they are all consistent and ready for the CNN model. Padding refers to the addition of extra pixels around the edge of the input image. It is done to allow the convolutional layer to apply the filter to the bordering elements of the input matrix.
Pooling, specifically max pooling, is used to reduce the spatial dimensions (height and width) of the input volume for the next convolutional layer. It helps in reducing the number of parameters and computation in the network, thereby also controlling overfitting.
Filters are used in convolutional layers to extract specific features from the input images. Each filter is a small matrix that is applied to the input data through a convolution operation, producing a feature map that highlights certain properties like edges, textures, or colors in the image.
Stride refers to the number of pixels by which we slide the filter across the input image. A larger stride results in smaller output dimensions, as the filter covers a larger area of the input image in each step6.
By using transfer learning, CNNs can take advantage of already established pre-trained models created using large-scale datasets (e.g., ImageNet). The first layers of the pre-trained model that have learned basic low-level features can be either frozen or fine-tuned, while the later layers are modified or removed to adapt to a specific tumor detection challenge.
In order to increase the quality of the training data and better equip the model with generalized abilities, data augmentation can be utilized. This involves applying transformations like rotation, translation, scaling and flipping to artificially diversify the original training dataset. In this way, Convolutional Neural Network models can learn features that are resistant and invariant amongst cancer cases.
In some cases, ensemble methods can be employed to enhance the tumor detection performance. Multiple CNN models can be trained independently, each with different initialization or hyperparameters, and if their predictions can be combined, a more robust and accurate decision can be reached. Additionally, ensemble methods can benefit from decreasing model distortion while increasing its capability to apply to other datasets.
Interpretability and Explainability
Model interpretability for medical applications includes techniques such as attention mechanisms, gradient-based visualization, and saliency mapping. Attention techniques enable the model to focus on the most relevant segments of an image, like specific regions of a brain MRI where tumors are most likely to be located. Gradient-based visualization or saliency mapping, on the other hand, generates visual explanations for the model’s decisions, highlighting the image areas most influential to the output. These methods not only enhance trust and reliability among clinicians but also aid radiologists in pinpointing critical areas in medical images, thereby potentially enhancing diagnostic accuracy.
The use of CNNs for tumor detection is constantly growing. Scientists are looking into more advanced structures, like 3D CNNs which can register three-dimensional data or pairing multiple types of scans (MRI and PET scans) to boost accuracy. Also, the supply of big annotated datasets and ongoing cooperation between researchers have enabled the production of even more powerful networks for cancer identification6.
There are several tools and techniques related to CNNs that have been developed for brain tumor detection. The following are some notable ones:
As an alternative to manual feature engineering, this CNN-based tool uses a 3D architecture to classify every voxel in an MRI scan as either cancerous or non-cancerous. It’s consistently demonstrated high performance in the BraTS dataset.
Although it’s commonly used for medical image segmentation tasks, including brain tumor detection, U-Net is still seen as a machine learning approach.
Caffe, TensorFlow, and PyTorch
These powerful frameworks are employed by researchers to undertake necessary tasks such as preprocessing, training models and validating them.
It should be noted that large datasets with annotations need to be created and validated properly for CNNs to be utilized successfully and safely in clinical settings, and so data scientists must collaborate closely with medical experts during development.
The principal divergence between machine learning and deep learning approaches lies in the degree of manual feature engineering required; while machine learning methods necessitate human input for selecting suitable features, deep learning methods possess the capability of automatically discovering relevant features from raw data inputs.
A study by Mubaraq Sanusi proposes the usage of a deep convolutional neural network (DCNN) to detect brain tumors from MRI images, building off of a pre-trained VGG-16 classification model5. The VGG-16 model was originally created by K. Simonyan and A. Zisserman from the University of Oxford in the paper “Very Deep Convolutional Networks for Large-Scale Image Recognition”, and has achieved 92.7% accuracy on ImageNet which contains over 14 million images from 1000 different classes.
This study7 employed a two-class dataset composed of MRI scans to detect brain tumors: positive class (tumor) and negative class (non-tumor). Results showed that the DCNN model proposed outperformed traditional methods in this regard.
This study used a collection of 253 brain MRI scans with various sizes. The images were sourced from Kaggle’s Brain MRI Images datasets, and are stored in JPG format. Within this dataset, there are 155 images depicting brain tumors, while the remaining 98 images represent normal brain scans.
The MRI images being used are altered to a particular size in order to be compatible with the VGG16 program. To ensure that all of the images have a comparable brightness, the intensity levels are modified. The values can be increased or decreased so that they are within a specified range, like between 0 and 1, or even standardized through z-score normalization.
To make better use of the dataset, the ImageDataGenerator function of the Keras library is used to augment the images. These changes include rotating the image 15 degrees clockwise, moving it up or down 10% of its height and width, randomly brightening or darkening 0-50%, shearing at an angle in a counterclockwise direction, and flipping the image horizontally or vertically.
Their proposed methodology utilizes DCNN architecture for brain tumor detection in brain MRI images.
The method comprises multiple sequential steps. Initially, the input image consists of the brain MRI scan. After the preprocessing of the images is performed, a pre-trained CNN, specifically the VGG-16 model, is employed for the classification of the images into two distinct classes: “YES” indicating the presence of a tumor and “NO” indicating the absence of a tumor.
The VGG-16 model consists of a total of 41 layers, out of which 16 layers have learnable parameters. It includes 13 convolutional layers and 3 fully connected layers.
The model utilizes rectified linear unit (ReLU) as the activation function in the hidden layers.
Mathematically, ReLU is expressed as f(x) = max(0,x)
The convolution layers use 3×3 filters (small square patterns) with a stride of 1 and padding of 1 pixel. Max-pooling layers follow each convolution layer to reduce the spatial dimensions.
The initial two convolution layers consist of 64 filters, each with a size of 3×3 pixels. These filters slide over the input image with a stride of 1 and padding of 1 pixel. The resulting feature map has a dimension of 224x224x64.
After each convolution layer, a max-pooling layer is applied. Max-pooling reduces the spatial dimension of the feature map by half. In this case, it uses 2×2 kernels with a stride of 2 pixels. The output of this operation is a feature map with dimensions 112x112x128.
The third and fourth convolutional layers contain 124 filters of size 3×3 pixels. They process the feature map from the previous layer and produce an output with dimensions 112x112x124. Another max-pooling layer is applied to the output of the previous convolution layers. It uses 2×2 kernels and a stride of 2 pixels, resulting in a feature map with dimensions 56x56x256. 5th, 6th, and 7th Convolution Layers: These layers consist of 256 filters with a size of 3×3 pixels. They process the feature map from the previous layer and produce an output with dimensions 56x56x256.
Third Max-Pooling: Another max-pooling layer is applied, reducing the spatial dimension of the feature map to 28x28x512.
8th to 13th Convolution Layers: This set of convolution layers also uses 3×3 filters and has 512 feature maps. They further process the feature map from the previous layer.
Final Fully-Connected (FC) Layers: The last part of the network consists of 3 fully connected layers. These layers contain filters of size 3×3 pixels and have ReLU-activated units. The first two FC layers have 4096 units each, and the last FC layer has 1000 units. The Softmax layer of 1000 units receives feature vectors with the help of the fully-connected layers. The Softmax activation function:
Overall, the VGG-16 model uses a series of convolutional and max-pooling layers to extract features from input images. These features are then passed through fully connected layers for classification or further processing.
This DCNN approach was very effective in identifying brain tumor from MR scans, achieving a higher classification accuracy (96%), high F1- Score (0.97), and improved precision recall value (0.93).
To evaluate the model’s performance more critically, the DCNN classifier was compared to traditional methods. We have included a table from the study5 , which shows that the new model outperforms the other approaches:
|Amin et al.8
|7 layered 2D CNN
|Reza et al.9
|MFDFA + random Forest
|Hemanth et al.10
|Mohsen et al.11
|SMO + SVM
The proposed architecture of the network operates exceptionally well in detecting tumors as compared to conventional methods. This model could be a revolutionary tool for diagnose brain tumors.
Commercially Available Tools
The Perelman School of Medicine at the University of Pennsylvania is collaborating with Intel and various research centers to use a secure machine learning system for recognizing brain tumors. It’s known as federated learning, and it eliminates the need to share patient information between institutions collaborating on deep learning projects. With help from Intel’s federated learning tools, researchers from the United States, Canada, the UK, Germany, Switzerland, and India are joining forces to construct a cutting-edge machine learning model. This model will be trained with an unprecedented amount of brain tumor data12.
Practicing in the Operating Room
This article outlines a two-pronged approach for bettering diagnostic accuracy and speed during brain tumor surgeries4. To enhance accuracy and speed during brain tumor surgeries, researchers have developed a novel approach combining AI with advanced imaging technology. Traditionally, surgeons had to wait for neuropathological analysis of tissue samples to determine tumor types during surgery, causing delays. To address this, a CNN was trained using over 2.5 million de-identified images from 415 patients, achieving a 94.6% accuracy rate. This CNN is utilized alongside stimulated Raman histology (SRH) microscopy, a recent technological advancement in imaging specimens rapidly and effectively. Combining the CNN’s precise analysis with SRH microscopy and traditional histology methods, this approach improves surgical decision-making and reduces delays particularly in challenging brain cancer cases.
Using CNNs for brain tumor detection can be a tricky task, and it is imperative to consider certain aspects to make sure the models are reliable, generalizable, and applicable in diagnosis and treatment.
Firstly, obtaining annotated databases necessary for training the CNN models can be challenging due to the infrequency of brain tumors. Also, getting access to labelled data requires specialists like radiologists to manually assign the correct labels for tumor regions on medical images; this process can be lengthy and costly. The lack of annotated data as well as its cost can impede the development of precise CNN models. An initiative that attempted to address this issue published 637 high-resolution imaging studies of 75 patients harboring 260 brain metastasis lesions13. Secondly, medical professionals and data scientists must work together closely to ensure the models are addressing the clinical needs, providing actionable results that fit into existing workflow models, and that their collaboration is effective despite any language or expectation differences.
Finally, ethical considerations such as patient data privacy and preventing model bias must be considered when creating and using a large-scale CNN model for this purpose. The increasing role of private entities in AI development raises significant privacy concerns. AI-driven methods may compromise the ability to keep patient data anonymous. AI can also be prone to errors and biases and may not be easily supervised by human medical professionals due to the “black box” problem, where the reasoning behind AI decisions can be opaque. This emphasizes the need for interpretable AI forms that can be integrated into medical care more effectively14??.
In conclusion, this research paper showcases the intersection of biology and technology in the field of bioinformatics, focusing on Convolutional Neural Networks (CNNs) for brain tumor detection. Deep learning techniques offer incredible possibilities to mechanize feature extraction from medical images.It stresses the interdisciplinary nature of bioinformatics.
By bridging the gap between biology and data science, this research is contributing to understanding the capabilities of CNNs in changing brain tumor detection. A case study of VGG-16 model was included to stress the practical usage of CNNs in medical imaging analysis.
Bioinformatics is powered by collaborationwhich combines expertise from biological and computational sciences. This research emphasizes this collaboration by utilizing medical imaging, machine learning, and clinical knowledge to improve diagnostic skills. The use of CNNs in brain tumor detection is motivated by the transformative capability of deep learning in bioinformatics.
The motivation behind this research is driven by recognizing how deep learning can revolutionize the area of bioinformatics intellectually, particularly with regard to brain tumor detection. By automating feature extraction from medical images, this research aspires to progress our comprehension and diagnosis of brain tumors thus ultimately enhancing patient outcomes. In summary, this research aims to contribute to the technical understanding of CNNs in medical imaging and discuss challenges that may arise in the implementation of CNNs.
- J. Amin, M. Sharif, A. Haldorai, M. Yasmin, and R. S. Nayak, “Brain tumor detection and classification using machine learning: a comprehensive survey,” Complex & Intelligent Systems, Nov. 2021, doi: https://doi.org/10.1007/s40747-021-00563-y. [↩]
- Sharma, Manav & Sharma, Pramanshu & Mittal, Ritik & Gupta, Kamakshi. Brain Tumour Detection Using Machine Learning. Journal of Electronics and Informatics. 2021, (PDF) Brain Tumour Detection Using Machine Learning [↩]
- S. Saeedi, S. Rezayi, H. Keshavarz, and S. R. Niakan Kalhori, “MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques,” BMC Medical Informatics and Decision Making, vol. 23, no. 1, Jan. 2023, doi: https://doi.org/10.1186/s12911-023-02114-6. [↩]
- “Gliomas,” www.hopkinsmedicine.org. https://www.hopkinsmedicine.org/health/conditions-and-diseases/gliomas#:~:text=Glioma%20is%20a%20common%20type [↩] [↩]
- M. Abu et al., “Deep Convolutional Neural Networks Model-based Brain Tumor Detection in Brain MRI Images.” Available: https://arxiv.org/pdf/2010.11978.pdf [↩] [↩] [↩]
- I. Goodfellow, Y. Bengio, and A. Courville, “Deep Learning,” Deeplearningbook.org, 2016. https://www.deeplearningbook.org/ [↩] [↩]
- M. Sanusi, “VGG-16 pretrained model,” Medium, Apr. 03, 2019. https://mubaraqsanusi.medium.com/vgg-16-pretrained-model-9cd600fd75e2 [↩]
- J. Amin, M. Sharif, M. Yasmin, and S. L. Fernandes, “Big data analysis for brain tumor detection: Deep convolutional neural networks,” Futur. Gener. Comput. Syst., vol. 87, pp. 290–297, 2018. [↩]
- S. M. S. Reza, R. Mays, and K. M. Iftekharuddin, “Multi-fractal detrended texture feature for brain tumor classification,” in Medical Imaging 2015: Computer-Aided Diagnosis, 2015, vol. 9414, p. 941410. [↩]
- G. Hemanth, M. Janardhan, and L. Sujihelen, “Design and Implementing Brain Tumor Detection Using Machine Learning Approach,” in 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), 2019, pp. 1289–1294. [↩]
- H. Mohsen, E. A. El-Dahshan, E. M. El-Horbaty, and A. M. Salem, “Brain tumor type classification based on support vector machine in magnetic resonance images,” Ann. “Dunarea Jos” Univ. Galati, Math. Physics, Theor. Mech. Fascicle II, Year IX, no. 1, 2017. [↩]
- HealthITAnalytics, “Deep Learning May Improve Identification of Brain Tumors,” HealthITAnalytics, Aug. 17, 2022. https://healthitanalytics.com/news/deep-learning-may-improve-identification-of-brain-tumors [↩]
- B. Ocaña-Tienda et al., “A comprehensive dataset of annotated brain metastasis MR images with clinical and radiomic data,” Scientific Data, vol. 10, no. 1, p. 208, Apr. 2023, doi: https://doi.org/10.1038/s41597-023-02123-0. [↩]
- B. Murdoch, “Privacy and artificial intelligence: challenges for protecting health information in a new era,” BMC Medical Ethics, vol. 22, no. 1, Sep. 2021, doi:https://doi.org/10.1186/s12910-021-00687-3. [↩]