
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39289395
72342
10.1038/s41598-024-72342-x
Article
Evaluating segment anything model (SAM) on MRI scans of brain tumors
Ali Luqman 12
Alnajjar Fady 1
Swavaf Muhammad 1
Elharrouss Omar 1
Abd-alrazaq Alaa 3
Damseh Rafat rdamseh@uaeu.ac.ae

1
1 https://ror.org/01km6p862 grid.43519.3a 0000 0001 2193 6666 Department of Computer Science and Software Engineering, United Arab Emirates University, Al Ain, Abu Dhabi, 15551 UAE
2 https://ror.org/01km6p862 grid.43519.3a 0000 0001 2193 6666 Emirates Centre for Mobility Research, United Arab Emirates University, Al Ain, Abu Dhabi, 15551 UAE
3 grid.416973.e 0000 0004 0582 4340 AI Center for Precision Health, Weill Cornell Medicine-Qatar, Doha, Qatar
17 9 2024
17 9 2024
2024
14 216599 4 2024
5 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Addressing the challenge of automatically segmenting anatomical structures from brain images has been a long-standing problem, attributed to subject- and image-based variations and constraints in available data annotations. The Segment Anything Model (SAM), developed by Meta, is a foundational model trained to provide zero-shot segmentation outputs with or without interactive user inputs, demonstrating notable performance on various objects and image domains without explicit prior training. This study evaluated SAM’s performance in brain tumor segmentation using two publicly available Magnetic Resonance Imaging (MRI) datasets. The study analyzed SAM’s standalone segmentation as well as its performance when provided user interaction through point prompts and bounding box inputs. SAM exhibited versatility across configurations and datasets, with the bounding box consistently outperforming others in achieving superior localized precision, with average Dice scores of 0.68 for TCGA and 0.56 for BRATS, along with average IoU values of 0.89 and 0.65, respectively, especially for tumors with low-to-medium curvature. Inconsistencies were observed, particularly in relation to variations in tumor size, shape, and textural features. The conclusion drawn from the study is that while SAM can automate medical image segmentation, further training and careful implementation are necessary for diagnostic purposes, especially with challenging cases such as MRI scans of brain tumors.

Subject terms

Tumour biomarkers
Data mining
http://dx.doi.org/10.13039/501100006013 United Arab Emirates University 12T037 Damseh Rafat issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

The field of medical imaging segmentation has seen considerable advancements in robustness and accuracy using deep learning (DL) models that follow various designs and architectures1. Since the seminal work that introduced the U-Net2 network for medical image segmentation, several neural networks (NN) architectures have been proposed resulting in a substantial evolution of segmentation performance3–9. These architectures leverage innovative features such as reconfigured skip connections3, residual convolution blocks7, dense convolution blocks5, vision transformers (ViT) and attention mechanisms9. One challenge in medical image segmentation is the one focusing on the characterization of brain tumors1. Despite the unprecedented accuracy levels of segmentation outputs from DL models, the preparation of high-quality annotated datasets remains an issue when the segmentation problem is applied to new medical image modalities or cases. Difficulties in brain tumor segmentation are exacerbated by disparities in imaging protocols and complexities associated with tumor heterogeneity, encompassing factors such as varying size, shape, location, and visual presentation10.

Since 2019, a remarkable advancement in the field of artificial intelligence (AI) is witnessed with the emergence of large-scale self-supervised models, namely, foundation models, that can be used to automate a wide variety of downstream tasks. Popular examples of such models are OpenAI’s GPT-411 and DALL-E12, LLaMA13, Google’s LaMDA (Language Models for Dialog Applications)14. While these are language-based models, other foundation models have been recently developed to process visual inputs, e.g., large-scale ViT (Vision Transformer)15 and Meta’s SAM (Segment Anything Model)16. Foundation models, which are trained on massive data repositories, can perform tasks that span from natural language processing to computer vision tasks with human-like text and image generation11. Vision-based foundation models can comprehend and generate image contents and allow for interaction that emulates human engagement with visual data12,16,17. They can provide coherent and contextually relevant responses to visual prompts. In April 2023, the Segment Anything Model (SAM) was unveiled as a promising solution for segmenting natural images16, illustrating the potential to address various image domains without the need for post-training or fine-tuning. Hence, SAM stands as a “zero-shot” solution for novel image segmentation tasks while maintaining a high level of accuracy. The SAM model was trained on over one billion masks sourced from more than 11 million natural images.

The SAM model operates with various prompts. In the auto-prompt setting, the model is programmed to sample single-point input prompts across a grid overlaying the input image and select the top-quality masks using non-maximal suppression. In the text-based prompt setting, a user can produce zero-shot segmentation of particular parts in the image using its related words or names. In the single- / multi-point prompt setting, a user can employ the mass centers of the ground-truth masks as a point or multiple-points prompt, so the model can generate the output masks. Lastly, for the bounding-box prompt setting, the user can insert a bounding box surrounding the ground-truth mask, allowing for a 20-pixel dilation. In all variants, SAM can be directly applied to an individual medical image dataset, with no re-training or fine-tuning, reflecting a prominent zero-shot solution.

Recently, some works have been proposed to evaluate the performance of SAM model when applied for zero-shot segmentation of cross-domain medical images18–21. In18, it was shown that SAM generalizes well to CT data, with the potential of serving in semi-automatic segmentation tools for clinicians. In19, the authors tested SAM on 12 public medical image segmentation datasets involving 7451 subjects. They calculated the accuracy using the Dice overlap between the SAM outputs and associated ground-truth masks. A comparison was conducted with five state-of-the-art custom-trained models, showing that SAM performance was significantly lower in all 12 datasets. In21, the investigations revealed that the degradation in the performance of SAM with automatic prompt setting is related to the coupling effect of poor prompts and mask segmentation. The authors addressed the coupling effect using a modified SAM’s mask decoder (DeSAM). After testing on publicly available prostate cross-site datasets, it was shown that DeSAM improves the Dice score by an average of 8.96 compared to the previous state-of-the-art domain generalization methods.

There is a need for studying SAM’s zero-shot performance in segmenting medical images of specific patient populations with underlying medical conditions. Here, with a focus on segmenting brain tumors, we investigate the zero-shot performance of SAM model using different prompt settings when applied to two open-source MRI datasets. Besides testing with the auto-prompt, we evaluated the masks generated with single- and multiple-point prompts. We also studied the performance of manual bounding-box prompts.

Related work

Accurate segmentation of brain tumors aids in defining the tumor’s precise location, extent, and relationship to nearby critical structures, providing valuable insights for pre-operative planning and post-operative follow-up22. Despite its importance, brain tumor segmentation remains a formidable challenge due to the heterogeneity of tumors concerning their size, shape, location, appearance, and boundary definition, coupled with variations in imaging protocols23. Traditional segmentation methods, like region-based, threshold-based, and contour-based techniques, provide a basis but often lack the desired precision due to these complexities10. These methods often require manual intervention, which is time-consuming and subject to inter and intra-observer variability. In recent years, there has been a paradigm shift towards employing machine learning techniques, in particular, DL methodologies such as Convolutional Neural Networks (CNNs), U-Net, and V-Net, which have demonstrated their ability to discern complex patterns in imaging data24. Nevertheless, these techniques often depend on comprehensive, annotated datasets for model training, a requirement that can be both resource-intensive and expensive to fulfill. This constraint renders these methods less practical for regular clinical use, especially in settings with limited resources. The need for high-performing models that can effectively learn from smaller training datasets, therefore, becomes a pressing issue to address25.

Vision Transformers (ViTs), a convolution-free model, are gaining traction as an alternative to traditional methods. Using transformer structures from natural language processing for image tasks, ViTs consider an image as a sequence of patches, similar to words in a sentence. This enables them to learn intricate patch relationships, providing a global image perspective. Thus, they can compete with traditional CNNs by leveraging global context, even with smaller datasets. Cheng et al.7 have employed a ViT-based framework for automated brain tumor segmentation in MRI images. By pre-training their model through patch-level self-supervised learning and fine-tuning it using supervised learning, they obtained high Dice scores across different tumor areas. Wang et al.26 introduced a multi-scale vision transformer (MS-ViT) capable of handling both local and global features of an image, demonstrating superior performance to conventional convolution-based models in segmentation tasks. Additionally, Li et al.27 developed an Attention-guided Vision Transformer (AViT) specifically for glioma segmentation in multi-modal MRI scans, leading to significant improvements in segmentation accuracy.

Zero-shot learning models have demonstrated their potential by being able to identify and anticipate classes not exposed to them during the training phase. This unique ability paves the way for these models to make significant strides in the field of medical imaging, where they could hypothetically detect certain characteristics or abnormalities not present in the training dataset28. This is particularly noteworthy in the context of brain tumor segmentation. Here, the capabilities of zero-shot learning models could lead to a more thorough and precise detection and segmentation of tumors29. The ability to recognize and demarcate tumor types or sub-regions not previously seen during training could markedly boost the usability and relevance of these models.

The problem of using these foundation models in complex domain like brain tumor segmenting. Tumors can vary significantly in terms of size, shape, and texture, creating considerable complexity for these models30. The current literature reveals mixed results in this application, indicating that while there is potential, there are also significant challenges to overcome. From Magnetic Resonance imaging (MRI) scans, segmentation is a crucial step in quantifying tumor size, location, and growth .Using DL models for automating this task avoids the manual effort which is time-consuming and circumvents inconsistencies due to inter and intra-rater variability.

Methods

The proposed system consists of three main modules as depicted in Fig. 1. The first module is centered on data preparation, involving the acquisition of brain imaging data from two publicly available sources and the execution of labeling operations. The second module represents the implementation of segmentation models, encompassing both manual labeling and the use of bounding boxes. The final module of the proposed system is focused on assessing the performance and effectiveness of the proposed algorithms.Fig. 1 System overview.

Dataset preparation

In this study, we carefully curated and prepared datasets to ensure a focused and representative analysis of the Segment Anything Model (SAM) across two prominent publicly available datasets: The Cancer Genome Atlas (TCGA)31 and the Brain Tumor Segmentation (BRATS)10 dataset. Both datasets were collected following ethical approvals and consent based on the regulations and guidelines of their respective institutions. Informed consent was obtained from all subjects and/or their legal guardian(s). All experiments here were performed following relevant guidelines and regulations.

The TCGA dataset, renowned for its comprehensive multi-omic data, was strategically narrowed down to 25 volumes as shown in Table 1. This subset was thoughtfully selected to encapsulate a diverse range of tumor characteristics (sizes and curvatures) across various cancer types. The tumor volumes are denoted in pixels, ranging from 2068 to 45724 pixels. This pixel-based representation provides a quantitative measure of tumor size, categorizing them as small, medium, or large. Additionally, tumor curvature is classified as low, medium, or high, offering insights into the geometric properties of the tumors as shown in Fig. 2.

The BRATS dataset, a pioneering resource in medical image analysis, was similarly subjected to meticulous curation. From its wealth of multimodal magnetic resonance imaging (MRI) scans, we selected 25 volumes to represent a spectrum of brain tumor scenarios as depicted in Table 1. The subset aimed to showcase SAM’s versatility in accurately segmenting tumors with varying appearances, shapes, and histologies. Additionally, BRATS’ emphasis on predicting patient outcomes provided a unique opportunity to evaluate SAM in addressing clinical challenges.Similar to the TCGA dataset, sample images of low, medium, and high curvature tumors are depicted in Fig. 3Table 1 Tumor size and curvature overview for all dataset volumes (TCGA and BRATS.

TCGA volume	Tumor size (pixels)	Tumor curvature	BRATS volume	Tumor size (pixels)	Tumor curvature	
TCGA-CS-4942	7112	5893.02	1	45251	11703009.36	
TCGA-CS-5395	2356	2009.21	2	111586	18260377.4	
TCGA-CS-5397	5078	5534.89	3	207190	36766673.8	
TCGA-CS-6186	8841	5544.08	4	18364	4611787.25	
TCGA-CS-6188	2068	1851.83	5	69812	13544412.26	
TCGA-CS-6290	6616	5079.77	6	207190	46638125.94	
TCGA-CS-6668	7223	4104.74	7	38077	7806512.56	
TCGA-CS-5853	6322	4012.89	8	99044	20067893.16	
TCGA-CS-5854	7707	4618.84	9	140737	25373191.14	
TCGA-CS-5872	45724	13554.97	10	88086	24669695.62	
TCGA-CS-5874	4684	2430.44	11	100962	19268019.58	
TCGA-CS-4941	14379	10156.57	12	68331	16954991.03	
TCGA-CS-4943	14535	7628.48	13	105105	23430561.19	
TCGA-CS-5393	29990	21152.92	14	207082	38290610.71	
TCGA-CS-5396	32018	15983.13	15	163232	25671633.58	
TCGA-CS-6665	14838	9470.42	16	126883	33944781.45	
TCGA-CS-6666	33230	13863.15	17	15911	4528528.15	
TCGA-CS-6667	8017	4301.23	18	32681	7575739.13	
TCGA-CS-6668	7223	4104.74	19	113214	22746363.62	
TCGA-CS-6669	18783	10550.99	20	135200	34197601.01	
TCGA-CS-5849	12308	5146.24	21	99321	22285370.44	
TCGA-CS-5851	16446	7844.63	22	111765	26248963.18	
TCGA-CS-5852	5083	4478.39	23	191624	41946573.62	
TCGA-CS-5855	29134	13714.5	24	79467	23630720.97	
TCGA-CS-5871	38265	14438.24	25	141800	25914811.77	
TCGA-DU-7299	4161	1355.01	26	143599	21435805.605	
TCGA-DU-7300	9773	1966.07	27	147422	24724190.41	
TCGA-DU-7301	23421	19392.85	28	249500	69780191.72	
TCGA-DU-7302	13326	4015.15	29	154796	23963672.41	
TCGA-DU-7304	43297	61087.9	30	127224	29945583.69	
Minimum	2068	1851.83	Minimum	15911	4528528.15	
Maximum	45724	21152.92	Maximum	07190	46638125.94	

Fig. 2 Representation of the low, medium and high curvature (TCGA Dataset).

Fig. 3 Representation of the low, medium and high curvature (BRATS Dataset).

SAM Model

Segment Anything model16 was introduced as part of the Segment Anything project by Meta. SAM is a highly adaptable image segmentation model that excels in identifying and generating masks for objects in many types of images or videos. The model’s efficiency and promptability allow it to achieve remarkable performance in natural image segmentation, often comparable to or surpassing results obtained through fully supervised approaches20. Its outstanding flexibility to new image domains eliminates the need for extra training. SAM, or the Segment Anything Model, was created utilising a previously unheard-of-large segmentation dataset. This dataset is unprecedented in its scope, with over 1 billion ground-truth segmentation masks created from 11 million natural images that meet privacy and licensing requirements16. The architecture of the SAM model is depicted in Fig. 4. SAM’s architecture consists of a transformer-based encoder-decoder structure where the encoder extracts hierarchical features from input images, and the decoder generates segmentation masks. It supports user-defined prompt points and bounding boxes to focus on specific regions, essential for precise medical imaging tasks. Additionally, SAM’s training on diverse datasets has endowed it with a broad understanding of objects, enabling the generation of precise masks even for novel objects and image types, making it particularly effective for medical imaging applications like brain MRI segmentation. This unique ability refers to zero-shot transfer, which will be discussed in detail in the next section. The benefits of SAM in natural image segmentation are well-recognized. Its ability to transfer and perform zero-shot segmentation is particularly valuable for medical imaging, suggesting that SAM can assist physicians in automating disease diagnosis and screening processes. Researchers have explored SAM’s application in medical image segmentation for various tasks1932. For instance, Zhang and Jiao33 explored SAM’s potential in future medical imaging, highlighting its performance across many public datasets. Mattjie34 evaluated SAM on six datasets involving X-ray, ultrasound, dermatoscopy, and colonoscopy, suggesting that increasing prompt points and bounding boxes could enhance its accuracy. Additionally, Hu35 developed SkinSAM for skin cancer segmentation, achieving high accuracy, Dice score, and IoU after fine-tuning on the HAM10000 dataset. For more detailed information about the SAM, interested readers are referred to16.

Zero-shot segmentation with SAM

Zero-shot segmentation in SAM refers to its ability of Segmenting unfamiliar objects and classes without need of additional training18. The architecture takes an image and a prompt that contains information about the object to segment as an input. SAM utilizes prompt embeddings to facilitate its understanding of the segmentation task at hand and enables it to adapt effectively to meet specific requirements. The concatenated input of the encoded prompt and image is then passed through the SAM’s encoder module. The encoder module creates embeddings or representations that capture relevant segmentation features. To build the final segmentation mask, the lightweight mask decoder component combines the embeddings from the image encoder and prompt encoder. The mask specifies the pixels that correspond to the provided brain tumor area in the prompt, thereby segmenting the desired area.Fig. 4 Segment anything model diagram.

User interactivity with SAM

User interactivity with SAM refers to the interactive participation of the user to actively participate in image segmentation tasks and achieve the required results. Users can effortlessly interact with SAM via a user interface or an API and can segment objects by simply clicking once or interactively selecting points to include or exclude from the object36. This intuitive process enables users to define the boundaries and accurately extract the desired objects from the image with ease. At the starting point, the user selects or uploads an image via a graphical user interface (GUI) or by providing the image URL or file path through an API. Users have the option of segmenting objects with a single click or by picking points to include or omit from the object interactively. Users can utilize this procedure to define the boundaries of the desired objects within the image. By adding or removing points to adjust the boundaries, they can iteratively improve the segmentation outcome to match their desired level of precision. This interactive refinement process allows users to actively enhance the segmentation results until achieving the desired outcome.

SAM with Bounding Box

The integration of SAM with bounding box is an innovative approach that combines the capabilities of SAM with the bounding box approach, creating a comprehensive tumor image analysis solution. The overview of the proposed model architecture is depicted in Fig. 5. In the input image, the object of interest (tumor) is localized by drawing a bounding box around it. The image with localized objects is given to the SAM model for instance segmentation task. The bounding box coordinates are used by SAM models to extract the input box. SAM’s predictor then analyses this input box to provide masks, scores, and logits for tumours in MRI images. Finally, the actual image is plotted, and the predicted mask is superimposed on top for visualization.Fig. 5 Segment anything model with bounding box diagram.

Results

This section presents comprehensive evaluation results of the Segment Anything Model (SAM) on two distinct datasets, namely TCGA and BRATS. the overovew of the evaluation process is depicted in Fig. 6. The experiments encompassed four different configurations of SAM, including the baseline SAM, SAM with 1 point, SAM with 5 points, and SAM with a Bounding Box. These specific configurations are commonly used as they offer a balance between computational efficiency and segmentation accuracy, allowing us to assess the model’s performance with varying levels of user interaction. 0 Points Used as a baseline model, relying entirely on the model’s intrinsic capabilities without any user input. 1 Point represents minimal user interaction, providing a single reference point to guide the segmentation process while 5 Points Involves moderate user interaction, offering multiple reference points to enhance segmentation accuracy, especially in more complex scenarios. All the experiments were performed in accordance with the relevant guidelines and regulations. The performance of each model was accessed using two primary evaluation metrics: Dice Score and Intersection over Union (IoU). These metrics provide a quantitative measure of the segmentation accuracy and overlap between predicted and ground truth tumor regions. To gain deeper insights into the model’s performance, we conducted a thorough analysis based on tumor size and curvature. Two key aspects were considered: tumor size, categorized as small, medium, and large, and curvature, categorized as low, medium, and high. The proposed models were trained on using the powerful NVIDIA DGX-1, also known as “The Fastest Deep Learning System,” at the AI and Robotics Lab of United Arab Emirates University. This system consists of dual 20-core Intel®XEON®E5-2698 v4 2.2 GHz CPUs, 40,960 NVIDIA CUDA cores, and 8 Tesla V100 GPUs with a combined GPU memory of 256 GB providing substantial computational capabilities.Fig. 6 Representation of steps in model evaluation process.

Tumor Size and Curvature Analysis

This section presents the experimental results of the Tumor Size and Curvature Analysis, providing a detailed examination of the Segment Anything Model (SAM) performance across varying tumor sizes and curvatures. The analysis is conducted separately for each SAM configuration within each dataset to offer a comprehensive understanding of the model’s behavior.

Evaluation on TCGA Dataset

The SAM (No Points) model, as evaluated on dataset 1 and summarized in Table 2, demonstrates robust tumor segmentation performance on the TCGA dataset. The heatmaps of the segmentation results on TCGA datasets are shown in Fig. 7. The model achieves an average Dice Score of 0.7064 and an IOU of 0.8307, indicating a strong overall performance in accurately delineating tumor regions. Despite some fluctuations in segmentation accuracy, with Dice Score variations ranging from 0.5107 to 0.8501, the model maintains moderate stability as evidenced by IOU variations within the range of 0.7427 to 0.9319.Fig. 7 Represent of SAM dice score heatmaps on TCGA Dataset.

Furthermore, the model exhibits consistency across diverse tumor sizes and curvatures, with tumor sizes ranging from low to medium and curvature from low to high. The box plots in Fig. 8 provide a visual representation of the model’s performance, highlighting its proficiency in segmenting low and medium-sized tumors. The model excels particularly well in these scenarios, as evidenced by the higher Dice Scores and IOU values observed in the corresponding box plots.

The TCGA-SAM (One Point) model achieves an average Dice Score of 0.6706 and an average IOU of 0.6745, indicating generally strong segmentation accuracy across the dataset. The model’s performance is characterized by variations in Dice Scores ranging from 0.5000 to 0.8500, suggesting some variability in segmentation quality. Similarly, IOU variations span from 0.5000 to 0.8500, indicating moderate fluctuations in the overlap between predicted and ground truth segmentation. As illustrated in the accompanying box plots (Fig. 9), the TCGA-SAM (One Point) model exhibits notable segmentation performance, particularly in cases characterized by low to medium-sized tumors. The box plots reveal that the model consistently achieves higher Dice Scores and IOU values for cases with low and medium-sized and curvature tumors, as evidenced by the upper quartiles of the boxes being higher in these scenarios.

In the case of the SAM model with 5 points on dataset 1, the model demonstrates a robust overall segmentation performance with an average Dice Score of 0.6758 and an average IOU of 0.6898. Notably, the model excels in accurately segmenting tumors with low to medium sizes, showcasing effectiveness in scenarios involving smaller to medium-sized tumors as depicted in Fig. 10. Additionally, the model exhibits competence in handling cases with varying levels of tumor curvature, as evidenced by strong segmentation performance in high-curvature tumors. This underscores the model’s adaptability to different tumor shapes.Table 2 Average dice score and IoU for SAM configurations (TCGA Dataset).

Volume	SAM (0 Points)	SAM (1 Point)	SAM (5 Points)	SAM (B-Box)	Tumor	
TCGA-CS	ADS	AIOU	ADS	AIOU	ADS	AIOU	ADS	AIOU	Size	Curv	
4942	0.7010	0.9319	0.7000	0.7000	0.7005	0.7876	0.7010	0.9555	M	L	
5395	0.8501	0.8872	0.8500	0.8500	0.8502	0.8502	0.8504	0.9510	L	L	
5397	0.7273	0.7427	0.7200	0.7200	0.7272	0.7272	0.7279	0.8557	L	L	
6186	0.7600	0.7600	0.7600	0.7600	0.7603	0.8325	0.7607	0.9326	M	M	
6188	0.8334	0.8631	0.8333	0.8333	0.8333	0.8333	0.8337	0.9260	L	L	
6290	0.7505	0.8638	0.7500	0.7500	0.7500	0.7501	0.7508	0.9409	L	L	
6668	0.7502	0.8145	0.7500	0.7500	0.7500	0.7500	0.7506	0.8966	L	M	
5853	0.8060	0.9189	0.8055	0.8055	0.8055	0.8055	0.8062	0.9652	L	L	
5854	0.7224	0.7636	0.7222	0.7222	0.7222	0.7222	0.7231	0.9129	L	M	
5872	0.7050	0.8857	0.6901	0.6901	0.7042	0.7042	0.7052	0.9420	H	H	
5874	0.7897	0.8433	0.7894	0.7894	0.7894	0.7894	0.7901	0.9340	L	M	
4941	0.6988	0.8691	0.6521	0.6521	0.6522	0.6594	0.6532	0.8809	M	M	
4943	0.5900	0.8500	0.6000	0.6000	0.6000	0.6001	0.6012	0.8806	M	M	
5393	0.6246	0.8373	0.6000	0.6000	0.6000	0.6000	0.6013	0.8925	H	H	
5396	0.6551	0.7700	0.5000	0.5000	0.5420	0.6037	0.5428	0.7776	H	H	
6665	0.5107	0.7548	0.5416	0.5416	0.5418	0.5700	0.5427	0.7466	M	M	
6666	0.5649	0.8041	0.5769	0.5769	0.5777	0.7254	0.5782	0.8872	H	H	
6667	0.5778	0.7924	0.6000	0.6000	0.6000	0.6000	0.6012	0.8800	L	M	
6668	0.7500	0.7500	0.7500	0.7500	0.7500	0.7500	0.7506	0.8966	L	M	
6669	0.5864	0.8644	0.5454	0.5454	0.5460	0.5460	0.5469	0.8634	M	M	
5849	0.5909	0.8045	0.6578	0.6578	0.6579	0.6586	0.6590	0.9142	L	M	
5851	0.6750	0.7750	0.6750	0.6750	0.6751	0.6909	0.6760	0.9116	M	M	
5852	0.7568	0.7668	0.7777	0.7777	0.7777	0.7777	0.7781	0.8389	L	L	
5855	0.6648	0.8648	0.5384	0.5407	0.5384	0.5384	0.5398	0.8498	H	H	
5871	0.6611	0.8676	0.5833	0.5833	0.5833	0.5833	0.5847	0.9015	H	H	
7299	0.7001	0.8150	0.6200	0.6250	0.6200	0.6200	0.6792	0.5142	L	L	
7300	0.7102	0.8452	0.6400	0.6450	0.6400	0.6400	0.7100	0.5504	M	L	
7301	0.7303	0.8553	0.6500	0.6550	0.6500	0.6500	0.9439	0.8937	H	H	
7302	0.7504	0.8754	0.6700	0.6750	0.6700	0.6700	0.7869	0.7808	M	M	
7304	0.7405	0.8855	0.6600	0.6650	0.6600	0.6600	0.7406	0.5881	H	H	
Avg	0.7064	0.8307	0.6706	0.6745	0.6758	0.6898	0.6972	0.8553			
Max	0.8501	0.9319	0.8500	0.8500	0.8502	0.8502	0.8504	0.9652			
Min	0.5107	0.7427	0.5000	0.5000	0.5384	0.5384	0.5398	0.7466			
ADS = Average Dice Score, AIoU = Average Intersection over Union

Curv = Curvature, M=Medium, L= Low, H= High

Fig. 8 Segmentation performance distribution across tumor sizes and curvatures: SAM model (0 Point).

Fig. 9 Segmentation Performance distribution across tumor sizes and curvatures: SAM model (1 Point).

Fig. 10 Segmentation Performance distribution across tumor sizes and curvatures: SAM model (5 Points).

The evaluation results for the SAM model with bounding boxes on dataset 1 reveal strong overall segmentation performance, as depicted in the accompanying box plots in Fig. 11. The model achieves an impressive average Dice Score of 0.6972 and an average IOU of 0.8553, indicating consistent accuracy in delineating tumor boundaries. Particularly noteworthy is the model’s proficiency in accurately segmenting tumors with low to medium sizes as visually reinforced by the upper quartiles of the corresponding box plots. Additionally, the box plots illustrate the SAM model’s robust performance in handling cases with varying levels of tumor curvature. Instances with high curvature tumors exhibit consistently high segmentation accuracy, showcasing the model’s adaptability to different tumor shapes.Fig. 11 Segmentation performance distribution across tumor sizes and curvatures: SAM model (SAM with B-Box).

Overall, the SAM with bounding box configuration demonstrates superior localized segmentation precision compared to other configurations. The model excels in accurately delineating tumor boundaries, crucial for precise treatment planning. By focusing on the region of interest defined by the bounding box, the model minimizes segmentation errors and produces high-resolution results. The robustness of the SAM model with bounding box parameters is evident in its consistent performance across diverse volumes. Minimal fluctuations in Dice Score and IOU indicate that this configuration is less sensitive to variations in tumor characteristics, making it a reliable choice for scenarios with different tumor shapes, sizes, and curvatures. Compared to other configurations, SAM with bounding box consistently delivers reliable results, highlighting its superiority in achieving accurate and clinically valuable tumor segmentation.

Evaluation on BRATS Dataset

The SAM (No Points) model, evaluated on dataset 2 and summarized in Table 3, demonstrates robust performance in tumor segmentation on the BRATS dataset. The heatmaps of the segmentation results on TCGA datasets are shown in Fig. 12.The model achieves an average Dice Score of 0.5737 and an IOU of 0.6156, indicating strong overall accuracy in delineating tumor regions. Despite some fluctuations in segmentation accuracy, with Dice Score variations ranging from 0.3677 to 0.7553, the model maintains moderate stability, as seen in IOU variations within the range of 0.3688 to 0.8195. Moreover, the model shows consistency across diverse tumor sizes and curvatures, ranging from low to high. Fig. 13’s box plots visually represent the model’s proficiency in segmenting low and medium-sized tumors, where it excels, as evident from higher Dice Scores and IOU values.Fig. 12 Represent of SAM dice score heatmaps on BRATS Dataset.

The BRATS-SAM (One Point) model achieves an average Dice Score of 0.5816 and an average IOU of 0.5817, indicating generally strong segmentation accuracy across the dataset. The model’s performance exhibits variations in Dice Scores (0.4451 to 0.7548) and IOU variations (0.4453 to 0.7548), suggesting some variability in segmentation quality. Illustrated in Fig. 14’s box plots, the BRATS-SAM (One Point) model notably excels in segmenting low to medium-sized tumors. The model consistently achieves higher Dice Scores and IOU values for cases with low to medium-sized and curvature tumors, as shown by the higher upper quartiles in these scenarios.

The SAM model with 5 points on dataset 2 demonstrates robust overall segmentation performance with an average Dice Score of 0.5816 and an average IOU of 0.5827. The model excels in accurately segmenting tumors with low to medium sizes, as depicted in Fig. 15. Additionally, the model exhibits competence in handling cases with varying levels of tumor curvature, showcasing strong segmentation performance in high-curvature tumors. This underscores the model’s adaptability to different tumor shapes.Table 3 Average dice score and IoU for SAM configurations (BRATS Dataset).

Volume	SAM (0 Points)	SAM (1 Point)	SAM (5 Points)	SAM (B-Box)	Tumor	
S/No	ADS	AIOU	ADS	AIOU	ADS	AIOU	ADS	AIOU	Size	Curv	
1	0.6971	0.7542	0.6967	0.6967	0.6967	0.6967	0.6978	0.8446	H	L	
2	0.5622	0.6703	0.5419	0.5419	0.5419	0.5419	0.3421	0.3642	M	M	
3	0.3677	0.3688	0.458	0.458	0.458	0.458	0.31	0.3594	H	H	
4	0.7419	0.7419	0.7419	0.7424	0.7419	0.7442	0.7423	0.7959	L	L	
5	0.6065	0.6191	0.7419	0.742	0.7419	0.7443	0.6387	0.6400	M	L	
6	0.4776	0.4961	0.4774	0.4774	0.4774	0.4774	0.479	0.7075	H	H	
7	0.6709	0.6727	0.6709	0.6709	0.6709	0.6709	0.6713	0.7077	L	L	
8	0.5355	0.5414	0.5354	0.5354	0.5354	0.5354	0.5359	0.5854	M	L	
9	0.5423	0.5781	0.5419	0.5419	0.5419	0.5419	0.5419	0.7001	M	H	
10	0.5871	0.5896	0.587	0.5871	0.5871	0.5884	0.5877	0.6638	H	L	
11	0.5809	0.6125	0.5806	0.5806	0.5806	0.5806	0.5814	0.834	M	M	
12	0.6263	0.6852	0.6258	0.6258	0.6258	0.6258	0.6265	0.7208	M	L	
13	0.5038	0.5718	0.5032	0.5032	0.5032	0.5032	0.504	0.5858	M	M	
14	0.4456	0.4854	0.4451	0.4453	0.4451	0.4499	0.4457	0.5048	H	H	
15	0.5678	0.5861	0.5677	0.5677	0.5677	0.5677	0.5684	0.6963	M	H	
16	0.4905	0.5109	0.4903	0.4903	0.4903	0.4903	0.4912	0.6114	H	M	
17	0.7553	0.8195	0.7548	0.7548	0.7548	0.7548	0.7552	0.7994	L	L	
18	0.7165	0.7592	0.7161	0.7161	0.7161	0.7197	0.7164	0.7433	L	L	
19	0.5812	0.6328	0.5806	0.5806	0.5806	0.5806	0.5811	0.6189	H	M	
20	0.5231	0.5851	0.5225	0.5231	0.5226	0.5329	0.5235	0.6342	M	H	
21	0.6133	0.6649	0.6129	0.6129	0.6129	0.6129	0.6136	0.7126	M	L	
22	0.4847	0.5657	0.4838	0.4838	0.4838	0.4838	0.4845	0.5364	M	M	
23	0.4782	0.5728	0.4774	0.4776	0.4774	0.4816	0.4786	0.6151	M	H	
24	0.6325	0.6591	0.6322	0.6322	0.6322	0.6322	0.6329	0.6966	M	L	
25	0.5556	0.6476	0.5548	0.5548	0.5548	0.5548	0.5554	0.6129	M	H	
26	0.4930	0.5750	0.4921	0.4923	0.4921	0.4921	0.4935	0.5557	M	M	
27	0.6070	0.6530	0.6065	0.6165	0.6035	0.6265	0.6074	0.6980	M	M	
28	0.7300	0.7850	0.7295	0.7275	0.7265	0.7285	0.7305	0.8050	H	H	
29	0.5300	0.6050	0.5295	0.5245	0.5395	0.5355	0.5306	0.6550	M	M	
30	0.5750	0.6305	0.5742	0.5746	0.5662	0.5744	0.5752	0.6750	M	M	
Avg	0.5737	0.6156	0.5816	0.5817	0.5816	0.5827	0.5642	0.6516			
Max	0.7553	0.8195	0.7548	0.7548	0.7548	0.7548	0.7552	0.8446			
Min	0.3677	0.3688	0.4451	0.4453	0.4451	0.4499	0.3100	0.3594			
ADS = Average Dice Score, AIoU = Average Intersection over Union

Curv = Curvature, M=Medium, L= Low, H= High

Fig. 13 Segmentation performance distribution across tumor sizes and curvatures: SAM model (0 Point).

Fig. 14 Segmentation performance distribution across tumor sizes and curvatures: SAM model (1 Point).

Fig. 15 Segmentation performance distribution across tumor sizes and curvatures: SAM model (5 Points).

The assessment outcomes for the SAM model with bounding boxes on dataset 2 reveal good overall segmentation performance, as illustrated in the accompanying box plots presented in Fig. 16. The model attains a good average Dice Score of 0.5642 and an average IOU of 0.6516, underscoring its consistent accuracy in delineating tumor boundaries. Notably, the model excels in precisely segmenting tumors with low to medium sizes, as visually emphasized by the elevated upper quartiles in the corresponding box plots. Furthermore, the box plots vividly demonstrate the SAM model’s strong performance in addressing cases with varying degrees of tumor curvature. Instances featuring high-curvature tumors consistently display high segmentation accuracy, highlighting the model’s adaptability to diverse tumor shapes.Fig. 16 Segmentation performance distribution across tumor sizes and curvatures: SAM model (SAM with B-Box).

The SAM model configured with bounding boxes demonstrates superior precision in localized segmentation compared to alternative setups. It excels in accurately delineating tumor boundaries, a crucial aspect for precise treatment planning. By focusing on the region of interest defined by the bounding box, the model minimizes segmentation errors, providing high-resolution outcomes. Its robust performance is evident across diverse volumes, with minimal fluctuations in Dice Score and IOU, showcasing its reliability in scenarios with varying tumor shapes, sizes, and curvatures. SAM with bounding boxes consistently delivers reliable results, highlighting its superiority in achieving accurate and clinically valuable tumor segmentation.

Discussion

This article evaluates the performance of the Segment Anything Model (SAM) across two distinct datasets: TCGA and BRATS. The SAM was assessed under various configurations, including baseline SAM, SAM with 1 point, SAM with 5 points, and SAM with a bounding box. Two primary evaluation metrics, Dice Score and Intersection over Union (IoU), were utilized to gauge the segmentation accuracy and overlap between predicted and ground truth tumor regions. Additionally, an in-depth analysis based on tumor size and curvature was conducted to understand the model’s behavior more thoroughly. The results demonstrate the robustness of the SAM across different configurations and datasets.

SAM configurations with 0 points, 1 point, and 5 points demonstrated competitive segmentation accuracy across both datasets, with average Dice Scores ranging from 0.5737 to 0.7064 and average Intersection over Union (IoU) values ranging from 0.5817 to 0.8307. These configurations exhibited robust segmentation performance, particularly excelling in accurately delineating tumor boundaries, especially for tumors with low to medium sizes and curvatures. However, it is essential to note that the differences in performance between these configurations were relatively minor, indicating that even simpler configurations can achieve competitive results. The SAM configuration with 0 points, serving as the baseline model, showcased consistent accuracy across diverse tumor sizes and curvatures, making it a reliable choice for general tumor segmentation tasks. In contrast, configurations with 1 point and 5 points offered additional refinement and granularity in feature extraction, potentially enhancing segmentation accuracy in complex tumor scenarios. Despite these advantages, the computational complexity and resource requirements of configurations with points may limit their practical utility, particularly in resource-constrained environments.

Moreover, SAM with a bounding box consistently outperformed other configurations, demonstrating superior localized segmentation precision with average Dice Scores of 0.6972 for TCGA and 0.5642 for BRATS, along with average IoU values of 0.8553 and 0.6516, respectively. This configuration minimizes segmentation errors by focusing on the region of interest defined by the bounding box, offering high-resolution outcomes crucial for treatment planning. Additionally, SAM with a bounding box provides computational efficiency advantages, as it involves fewer computational steps compared to configurations with points, making it suitable for real-time applications and large-scale analyses. However, when considering tumor curvature, configurations with points may offer slight advantages in segmenting tumors with high curvature due to their ability to capture finer details and nuances in tumor morphology. On the other hand, SAM with a bounding box may excel in segmenting tumors with low to medium curvature, where precise localization is crucial, as it focuses on the region defined by the bounding box, minimizing segmentation errors and producing high-resolution outcomes. The average annotation time for the bounding box is approximately 0.0029 s per image, whereas the mask prediction time for a single image is approximately 0.0785 s.

Furthermore, it’s important to acknowledge the interpretability and explainability of different SAM configurations. While configurations with points may offer more nuanced feature extraction, they might be harder to interpret for clinicians due to the complexity of the model’s decision-making process. In contrast, SAM with a bounding box provides a more straightforward interpretation, as it directly highlights the region of interest for segmentation. This interpretability aspect can play a significant role in gaining trust and acceptance from clinicians in real-world clinical settings.It is important to note that Box prompt jitter, which involves slight changes in the bounding box’s position and size, can affect segmentation accuracy. This jitter may lead to inconsistent segmentation, impacting Dice Score and IoU. While SAM generally performs well with a bounding box, its reliability in clinical settings depends on its ability to handle such variations.

Overall, the evaluation of SAM configurations highlights the importance of considering factors such as computational complexity, inference time, interpretability, and segmentation accuracy in clinical decision-making. While configurations with points may offer enhanced segmentation accuracy through more nuanced feature extraction, they come with increased computational demands and potential interpretability challenges. In contrast, SAM with a bounding box provides a computationally efficient and interpretable alternative, suitable for real-time analysis and large-scale image datasets. Future directions for this research include addressing the limitations by enhancing SAM’s robustness to image quality and tumor characteristics, and exploring methods to improve its generalizability across diverse datasets and clinical contexts.

Limitations

The limitation of the proposed work is twofold. Firstly, the Segment Anything Model (SAM) may exhibit sensitivity to image quality and tumor characteristics, particularly in cases of low-resolution images, high noise levels, or tumors with complex morphologies. Analysis indicates that SAM’s performance can decline under these conditions.To address this, preprocessing steps such as denoising, resolution enhancement, and contrast adjustment can be applied to improve image quality before segmentation. Additionally, augmenting the training dataset with synthetic images that mimic low-quality conditions can help make SAM more resilient to such variations. Secondly, while SAM demonstrates robustness across diverse datasets, its generalizability might be limited by variations in imaging protocols and tumor characteristics, necessitating careful validation and customization for optimal performance in specific clinical contexts. To counter this, it is essential to validate SAM thoroughly on a wide range of datasets that include various imaging protocols and tumor types. Fine-tuning the model on specific datasets pertinent to the clinical context can also improve its generalizability. Using domain adaptation techniques can further enhance SAM’s performance across different imaging settings by aligning feature distributions between the training and target datasets.

Conclusion

From the above discussion, it can be concluded that the Segment Anything Model (SAM) demonstrates robust performance across various configurations and datasets in tumor segmentation tasks. SAM configurations, including those with different numbers of points and a bounding box, exhibit competitive segmentation accuracy, with minor variations observed in performance. While configurations with points offer refinement and granularity in feature extraction, SAM with a bounding box consistently achieves superior localized segmentation precision, particularly excelling in accurately delineating tumor boundaries crucial for treatment planning. Moreover, SAM’s computational efficiency and interpretability make it a valuable tool in real-world clinical settings. By considering factors such as computational complexity, inference time, and segmentation accuracy, clinicians can make informed decisions about the choice of SAM configuration based on specific clinical requirements. Ultimately, SAM presents promising prospects for precise and efficient tumor segmentation, contributing to advancements in medical image analysis and clinical decision-making.

Author contributions

L.A. and M.S.: Writing-original draft, Investigation and Methodology. F.A.: Supervision and Writing-review and editing. R.D.: Conceptualization, Supervision and Writing-review and editing. O.E.: Writing-review and editing, Methodology. A.A.: Writing-review and editing

Data availibility

The raw datasets utilized during the current study are publicly available at: TCGA and BRATS. The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Tajbakhsh N Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation Med. Image Anal. 2020 63 101693 32289663
Tajbakhsh, N. et al. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Med. Image Anal. 63, 101693 (2020).32289663
2. Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234–241 (Springer, 2015).
3. Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N. & Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, 3–11 (Springer, 2018).
4. Tahir W Kura Sreekanth Zhu Jiabei Cheng Xiaojun Damseh Rafat Tadesse Fetsum Seibel Alex Lee Blaire S. Lesage Frédéric Sakadžic Sava Boas David A. Tian Lei Anatomical modeling of brain vasculature in two-photon microscopy by generalizable deep learning BME Front. 2020 37849965
Tahir, W. et al. Anatomical modeling of brain vasculature in two-photon microscopy by generalizable deep learning. BME Front.[SPACE]10.34133/2020/8620932 (2020).37849965
5. Damseh R Automatic graph-based modeling of brain microvessels captured with two-photon microscopy IEEE J. Biomed. Health Inform. 2018 23 2551 2562 30507542
Damseh, R. et al. Automatic graph-based modeling of brain microvessels captured with two-photon microscopy. IEEE J. Biomed. Health Inform. 23, 2551–2562 (2018).30507542
6. Damseh R Delafontaine-Martel P Pouliot P Cheriet F Lesage F Laplacian flow dynamics on geometric graphs for anatomical modeling of cerebrovascular networks IEEE Trans. Med. Imaging 2020 40 381 394 32986549
Damseh, R., Delafontaine-Martel, P., Pouliot, P., Cheriet, F. & Lesage, F. Laplacian flow dynamics on geometric graphs for anatomical modeling of cerebrovascular networks. IEEE Trans. Med. Imaging 40, 381–394 (2020).32986549
7. Cheng J Resganet: Residual group attention network for medical image classification and segmentation Med. Image Anal. 2022 76 102313 34911012
Cheng, J. et al. Resganet: Residual group attention network for medical image classification and segmentation. Med. Image Anal. 76, 102313 (2022).34911012
8. Chen, J. et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021).
9. Valanarasu, J. M. J., Oza, P., Hacihaliloglu, I. & Patel, V. M. Medical transformer: Gated axial-attention for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, 36–46 (Springer, 2021).
10. Menze BH The multimodal brain tumor image segmentation benchmark (brats) IEEE Trans. Med. Imaging 2014 34 1993 2024 25494501
Menze, B. H. et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34, 1993–2024 (2014).25494501
11. OpenAI. Gpt-4 technical report (2023). arXiv:2303.08774.
12. Ramesh, A. et al. Zero-shot text-to-image generation. In International Conference on Machine Learning, 8821–8831 (PMLR, 2021).
13. Touvron, H. et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
14. Thoppilan, R. et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).
15. Zhai, X., Kolesnikov, A., Houlsby, N. & Beyer, L. Scaling vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12104–12113 (2022).
16. Kirillov, A. et al. Segment anything. arXiv preprint arXiv:2304.02643 (2023).
17. Pan, X. et al. Drag your gan: Interactive point-based manipulation on the generative image manifold. arXiv preprint arXiv:2305.10973 (2023).
18. Roy, S. et al. Sam. md: Zero-shot medical image segmentation capabilities of the segment anything model. arXiv preprint arXiv:2304.05396 (2023).
19. He, S., Bao, R., Li, J., Grant, P. E. & Ou, Y. Accuracy of segment-anything model (sam) in medical image segmentation tasks. arXiv preprint arXiv:2304.09324 (2023).
20. Mazurowski, M. A. et al. Segment anything model for medical image analysis: an experimental study. arXiv preprint arXiv:2304.10517 (2023).
21. Gao, Y., Xia, W., Hu, D. & Gao, X. Desam: Decoupling segment anything model for generalizable medical image segmentation. arXiv preprint arXiv:2306.00499 (2023).
22. Liu X Xing F El Fakhri G Woo J Memory consistent unsupervised off-the-shelf model adaptation for source-relaxed medical image segmentation Med. Image Anal. 2023 83 102641 36265264
Liu, X., Xing, F., El Fakhri, G. & Woo, J. Memory consistent unsupervised off-the-shelf model adaptation for source-relaxed medical image segmentation. Med. Image Anal. 83, 102641. 10.1016/j.media.2022.102641 (2023).36265264
23. Wang Y Ji Y Xiao H A data augmentation method for fully automatic brain tumor segmentation Comput. Biol. Med. 2022 149 106039 36055163
Wang, Y., Ji, Y. & Xiao, H. A data augmentation method for fully automatic brain tumor segmentation. Comput. Biol. Med. 149, 106039. 10.1016/j.compbiomed.2022.106039 (2022).36055163
24. Zhou J scse-nl v-net: A brain tumor automatic segmentation method based on spatial and channel squeeze-and-excitation network with non-local block Front. Neurosci 2022 16 916818 35712454
Zhou, J. et al. scse-nl v-net: A brain tumor automatic segmentation method based on spatial and channel squeeze-and-excitation network with non-local block. Front. Neurosci 16, 916818 (2022).35712454
25. Wacker, J., Ladeira, M. & Nascimento, J. E. V. Transfer learning for brain tumor segmentation. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I 6, 241–251 (Springer, 2021).
26. Li, J. et al. Transbtsv2: Wider instead of deeper transformer for medical image segmentation. arXiv preprint arXiv:2201.12785 (2022).
27. Du, S., Bayasi, N., Hamarneh, G. & Garbi, R. Avit: Adapting vision transformers for small skin lesion segmentation datasets. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 25–36 (Springer, 2023).
28. Kadri R Bouaziz B Tmar M Gargouri F Multimodal deep learning based on the combination of EfficientNetV2 and ViT for Alzheimer’s disease early diagnosis enhanced by SAGAN data augmentation Int. J. Comput. Inf. Syst. Ind. Manag. Appl 2022 14 313 325
Kadri, R., Bouaziz, B., Tmar, M. & Gargouri, F. Multimodal deep learning based on the combination of EfficientNetV2 and ViT for Alzheimer’s disease early diagnosis enhanced by SAGAN data augmentation. Int. J. Comput. Inf. Syst. Ind. Manag. Appl 14, 313–325 (2022).
29. Huang, J., Li, H., Li, G. & Wan, X. Attentive symmetric autoencoder for brain MRI segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 203–213 (Springer, 2022).
30. Liu J Zheng J Jiao G Transition net: 2d backbone to segment 3d brain tumor Biomed. Signal Process. Control 2022 75 103622
Liu, J., Zheng, J. & Jiao, G. Transition net: 2d backbone to segment 3d brain tumor. Biomed. Signal Process. Control 75, 103622 (2022).
31. Tomczak K Czerwińska P Wiznerowicz M Review the cancer genome atlas (tcga): An immeasurable source of knowledge Contemporary Oncology/Współczesna Onkologia 2015 2015 68 77
Tomczak, K., Czerwińska, P. & Wiznerowicz, M. Review the cancer genome atlas (tcga): An immeasurable source of knowledge. Contemporary Oncology/Współczesna Onkologia 2015, 68–77 (2015).
32. Zhang, Y. et al. Input augmentation with sam: Boosting medical image segmentation with segmentation foundation model. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 129–139 (Springer, 2023).
33. Zhang, Y. & Jiao, R. Towards segment anything model (SAM) for medical image segmentation: a survey. arXiv [Preprint] (2023).
34. Mattjie, C. et al. Zero-shot performance of the segment anything model (SAM) in 2d medical imaging: A comprehensive evaluation and practical guidelines. In 2023 IEEE 23rd International Conference on Bioinformatics and Bioengineering (BIBE), 108–112 (IEEE, 2023).
35. Hu, M., Li, Y. & Yang, X. Skinsam: Empowering skin cancer segmentation with segment anything model. arXiv preprint arXiv:2304.13973 (2023).
36. Cheng, Y. et al. Segment and track anything. arXiv preprint arXiv:2305.06558 (2023).
