
==== Front
PLoS One
PLoS One
plos
PLOS ONE
1932-6203
Public Library of Science San Francisco, CA USA

10.1371/journal.pone.0308326
PONE-D-24-00318
Research Article
Research and Analysis Methods
Imaging Techniques
Research and Analysis Methods
Mathematical and Statistical Techniques
Mathematical Functions
Convolution
Medicine and Health Sciences
Oncology
Cancers and Neoplasms
Computer and Information Sciences
Neural Networks
Biology and Life Sciences
Neuroscience
Neural Networks
Biology and Life Sciences
Cell Biology
Cell Physiology
Cell Fusion
Biology and Life Sciences
Organisms
Eukaryota
Plants
Flowering Plants
Vanilla
Medicine and Health Sciences
Diagnostic Medicine
Computer and Information Sciences
Artificial Intelligence
Machine Learning
Deep Learning
Multifunctional aggregation network of cell nuclei segmentation aiming histopathological diagnosis assistance: A new MA-Net construction
Multifunctional aggregation network of cell nuclei segmentation
https://orcid.org/0000-0002-2106-237X
Pu Qiumei Conceptualization Funding acquisition Methodology Project administration Supervision Writing – review & editing 1 *
Tian Jinglong Formal analysis Software Writing – original draft Writing – review & editing 1
Wei Donghao Formal analysis Software Writing – original draft Writing – review & editing 1
Shu Qingming Writing – review & editing 2
Sun Minghui Writing – review & editing 3
https://orcid.org/0000-0002-9796-0221
Zhao Lina Conceptualization Funding acquisition Methodology Project administration Supervision Writing – review & editing 3 *
1 School of Information Engineering, Minzu University of China, Beijing, China
2 Department of Pathology, Third Medical Center of Chinese PLA General Hospital, Beijing, China
3 Key Laboratory for Biomedical Effects of Nanomaterials and Nanosafety, Institute of High Energy Physics, Chinese Academy of Sciences, Beijing, China
Li Xiaowei Editor
Sichuan University, CHINA
Competing Interests: The authors declare no competing interests.

* E-mail: linazhao@ihep.ac.cn (LZ); puqiumei@muc.edu.cn (QP)
6 9 2024
2024
19 9 e03083263 1 2024
22 7 2024
© 2024 Pu et al
2024
Pu et al
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Automated diagnostic systems can enhance the accuracy and efficiency of pathological diagnoses, nuclear segmentation plays a crucial role in computer-aided diagnosis systems for histopathology. However, achieving accurate nuclear segmentation is challenging due to the complex background tissue structures and significant variations in cell morphology and size in pathological images. In this study, we have proposed a U-Net based deep learning model, called MA-Net(Multifunctional Aggregation Network), to accurately segmenting nuclei from H&E stained images. In contrast to previous studies that focused on improving a single module of the network, we applied feature fusion modules, attention gate units, and atrous spatial pyramid pooling to the encoder and decoder, skip connections, and bottleneck of U-Net, respectively, to enhance the network’s performance in nuclear segmentation. The dice coefficient loss was used during model training to enhance the network’s ability to segment small objects. We applied the proposed MA-Net to multiple public datasets, and comprehensive results showed that this method outperforms the original U-Net method and other state-of-the-art methods in nuclei segmentation tasks. The source code of our work can be found in https://github.com/LinaZhaoAIGroup/MA-Net.

National Key Research and Development Program of China 2021YFA1200904 https://orcid.org/0000-0002-2106-237X
Pu Qiumei National Key Research and Development Program of China 2020YFA0710700 https://orcid.org/0000-0002-2106-237X
Pu Qiumei http://dx.doi.org/10.13039/501100001809 National Natural Science Foundation of China 31971311 https://orcid.org/0000-0002-2106-237X
Pu Qiumei http://dx.doi.org/10.13039/501100001809 National Natural Science Foundation of China 12375326 https://orcid.org/0000-0002-2106-237X
Pu Qiumei Innovation Program for IHEP E35457U210 https://orcid.org/0000-0002-9796-0221
Zhao Lina National Key Research and Development Program of China (2021YFA1200904 and 2020YFA0710700), the National Natural Science Foundation of China (31971311, 12375326), the Innovation Program for IHEP (E35457U210). Data AvailabilityThe data used in our work can be accessed through the following link: https://doi.org/10.6084/m9.figshare.26073790.v2.
Data Availability

The data used in our work can be accessed through the following link: https://doi.org/10.6084/m9.figshare.26073790.v2.
==== Body
pmcIntroduction

According to the report of cancer data, released by the International Agency for Research on Cancer (IARC) in January 2022, there were about 19.29 million new cancer cases increased in the world in 2021 [1]. When cancer is in its early stage, early treatment can greatly improve the survival rate of patients, while reducing the incidence rate of patients and the cost of cancer treatment [2]. Histopathological diagnosis of tissues is the most reliable means of cancer diagnosis and serves as the gold standard for cancer determination in modern medical practices [3]. Pathologists can determine the proportion of cancer cells and grade cancer by observing rich information such as cell location, density, and morphological features in digitized histopathological images from patient biopsies. This plays a crucial role in the diagnosis and treatment of cancer. However, analyzing digital pathology images demands extensive experience from pathologists, leading to subjectivity in manual assessments. Moreover, this task is labor-intensive, and when confronted with a large volume of digital pathology images, doctors might experience fatigue, leading to the possibility of misdiagnosis or overlooking crucial details, potentially significantly impacting subsequent patient treatment. With the assistance of computer-aided diagnostic systems, diagnostic efficiency can be significantly enhanced, yielding more precise and objective analytical outcomes. This aids pathologists in promptly devising follow-up treatment plans for patients. Image segmentation techniques can extract crucial foreground information from images while eliminating irrelevant background interference. Precise nuclei segmentation serves as the foundation for numerous subsequent pathological diagnostic tasks and is a vital component of pathology-assisted diagnostic systems.

Currently, nuclear segmentation methods can broadly be classified into two categories: traditional segmentation methods and deep learning-based segmentation methods. Traditional segmentation methods include threshold-based segmentation [4], watershed segmentation [5], morphological processing [6], etc. The threshold-based segmentation method is the simplest image segmentation technique, dividing image pixels into different categories based on predefined threshold parameters. This method disregards spatial information of pixels, making it susceptible to noise interference and resulting in less precise segmentation outcomes. The watershed segmentation method uses the difference of gray distribution and background color in the nuclear image to segment the nuclei. The morphological segmentation method is also vulnerable to the influence of background factors, resulting in fuzzy segmentation results. For example, Prakarsa et al. [7] proposed a medical blood cell image segmentation model based on the global adaptive threshold segmentation method, but some features are still lost. Gu et al. [8] proposed a marker-controlled watershed algorithm for nuclear clustering segmentation, and the segmentation effect is not obvious. Therefore, the traditional segmentation methods are still limited to segmenting the nuclei through prior knowledge, and it is difficult to obtain accurate results when segmenting the nuclear image with complex background.

Neural networks represent a statistically-driven methodology, adapting network weights through extensive data training. During the inference phase, these networks automatically extract abstract features from input images, enabling the completion of subsequent tasks like classification and segmentation. In comparison to conventional approaches, neural networks often produce segmentation outcomes characterized by higher resolution and accuracy. The Convolutional Neural Network, as known as CNN, has been widely used to deal with visual image problems, such as object detection [9,10], image segmentation [11,12], medical image processing [13,14], and so on. One of the key advantages of CNNs is their ability to generate feature maps with distinct receptive fields at different depths within the network. This enables the integration of these feature maps to gather both local and global information from images. The advantage of CNNs has sparked researchers’ interest in exploring how CNN-based methods can better achieve image segmentation, so far, numerous outstanding segmentation models have been proposed, such as FCN, SegNet, Deeplab, and the U-Net series. These models have been widely applied across various fields for image segmentation tasks.

While image segmentation based on deep learning has exhibited significant advancements over traditional methods, empirical evidence demonstrates that the segmentation performance is not satisfactory for images containing minuscule cell nuclei or exhibiting low contrast between nuclei and the background. The primary reasons behind these issues can be attributed to two factors: Firstly, the inadequate feature extraction capability of the network’s encoder section makes it challenging to effectively extract pertinent information from intricate cell staining images for distinguishing between cell nuclei and background tissues. Secondly, the insufficiency of spatial information in the images obtained by the decoder is due to the downsampling process. While downsampling can abstract more semantic features, it simultaneously leads to the loss of spatial information. Consequently, the decoder is unable to access sufficient information to generate accurate segmentation outcomes.

To address these challenges, we proposed a multifunctional aggregation network named MA-Net. The contributions of this paper are as follows:

In the encoder section of our model, we have integrated the Down-sampling Fusion module and the Context Extractor module to enhance the network’s feature extraction capability and address the issue of missing small cell structures.

During the decoding phase, we leverage Attention Gates and the Up-sampling Fusion module to process feature maps, furnishing the decoder with higher-quality features. This approach results in the generation of improved segmentation outcomes.

By employing a composite loss function based on Binary Cross Entropy Loss and Dice Loss during training, we further enhance the segmentation efficacy for small targets. The model’s performance is evaluated on two datasets, MoNuseg and TNBC. Experimental results substantiate that our proposed MA-Net outperforms other improved models in terms of cell nuclei segmentation tasks.

Related work

The U-Net network proposed by Ronneberger et al. [15] has gained widespread adoption in various medical image segmentation tasks due to its outstanding performance. It adopts the encoder-decoder structure and skillfully combines the deep-level and shallow-level information by skipped connection (concatenation). The deep abstract information of the decoding layer makes better use of the shallow information transmitted by the encoding layer, making the image segmentation effect better. Lu et al. [16] applied a U-Net-based network to segment the retinal layer of optical coherence tomography (OCT) images. Sarhan et al. [17] used improved U-Net for the optic disc segmentation in high-precision retinal fundus images. The U-Net is also applied to other aspects of segmentation, such as cartilage and meniscus from knee MRI data [18] and lung lesions from COVID-19 chest CT scan images [19].

Ozan et al. [20] proposed a novel U-Net-based model consisting of the attention gate (AG), which can automatically learn to focus on target structures of different shapes and sizes. The key of AG is that it can make the model suppress irrelevant regions of input images and highlight salient features useful for a specific task. Also, AG could be easily applied to U-Net or other CNN architectures with minimal computation and high prediction accuracy. Zhou et al. [21] proposed a novel and more powerful UNet++ architecture. They integrated U-Nets at the depth of 1 to 4 into one architecture. UNet++ could be pruned by a deep supervision technique. The advantage of pruning is that it could reduce the inference time but decrease a little accuracy performance. Gu et al. [22] present a context encoder network (CE-Net). Because of its two main parts, dense atrous convolution(DAC) block and residual multi-kernel pooling(RMP) block, CE-Net enables to capture more deep-level information and retains more spatial information for medical 2D image segmentation. Xiang et al. [23] present a Bi-directional O-shape network (BiO-Net). The model has a simple structure that recurrently reuses the building blocks by adding no more extra blocks and parameters.

Methodology

Our proposed MA-Net employs U-Net as the backbone, as shown in Fig 1, the network structure can be divided into four parts: the downsampling path, the bottleneck block(the bottom of the U-shaped structure), the upsampling path, and the skip connections. Previous work has typically focused on improving individual parts to enhance performance. However, we believe that improving a single part is insufficient, especially for complex scenes like nucleus segmentation. Our improvement strategy involves simultaneously enhancing all four parts. In the upsampling and downsampling paths, we optimize the feature fusion strategy to better integrate features with different receptive fields. In the bottleneck block, we further process the encoded high-level semantic features. In the skip connection structure, we use spatial attention mechanisms to better handle image details.

10.1371/journal.pone.0308326.g001 Fig 1 Overview of the proposed MA-Net architecture.

During the encoding process, we incorporate the Down-sampling Fusion module to integrate image spatial information (pink parts in Fig 1). The module directly performs multiple downsampling operations on the input image (via 2-stride convolution instead of max pooling) to form a structure similar to an image pyramid, thereby obtaining information at different scales of the original image. These feature maps are then connected to the U-Net encoder as additional inputs. It is mathematically shown in Eq 1. DFi(12H×12W×2C)=Convs2(DFi−1(H×W×C))i=1,2,3,4 (1)

Where function Convs2(•) is a stride 2 convolution operation, DFi(H×W×C) denote the feature map at the i-th layer of the U-Net encoder.

At the end of the downsampling, we utilized the Context Extractor Module composed of dense atrous convolution (DAC) block and residual multi-kernel pooling (RMP) block to extract context semantic information and generate more deep-level feature maps. DAC is composed of dense atrous (dilated) convolutions, which can achieve a larger receptive field with fewer parameters, overcoming the limitations of pooling layers that cause loss of image semantic information [24]. The DAC consists of four parallel atrous convolution branches with different dilated rates, allowing it to extract features from different sizes and scales. RMP uses four residual convolution kernels of different sizes to encode global context information. The kernels sizes are set to 2×2, 3×3, 4×4, and 6×6. It could detect objects of different sizes on account of these different sizes of kernels.

In the decoder part, in order to strengthen feature propagation and feature reuse, and alleviate the vanishing gradient problem, we added the Up-sampling Fusion Module. This module was inspired by Gao et al. [25] who proposed Dense-nets. The output of each upsampling is spliced with the feature map of subsequent upsampling. We employ transpose convolution with a stride of 2 in each dense connection path to ensure that the generated feature maps have appropriate dimensions and channel numbers. The calculation formula is given in Eq 2. DFi=Concat(U(4,i),U(3,i),⋯,U(i+1,i))i=1,2,3 (2)

where function U(i,j) is upsampling operation to upsample featuremap from layer j to layer i, function Concat(•) is concatenate operation, UFi denotes an output feature map of up-sampling fusion module.

Attention Gate Module (AG) is added to the skip-connection. AG introduces spatial attention to the feature maps by assigning pixel-level weights, enhancing the response of important regions while suppressing irrelevant signals. In this way, the network can focus more on the foreground of the cell nuclei, mitigating the adverse effects of complex irrelevant background information on the segmentation results to some extent.

At the last, We replaced the convolution blocks of the original U-Net with Residual Modules. It helps alleviate the issue of information loss in traditional convolutional networks to some extent and avoids the gradient disappearance and gradient explosion problems that may be caused by the deep convolution neural network, so as to enhance the robustness of the model.

Fig 2 illustrates the structural details of the aforementioned modules. In summary, the goal of MA-Net is to retain as much image detail information as possible while improving the quality of feature maps. This aims to enhance the network’s ability to recognize cell nuclei of different sizes and colors, thereby avoiding phenomena such as holes in segmented nuclei, nuclei adhesion, and the omission of small nuclei in the segmentation results.

10.1371/journal.pone.0308326.g002 Fig 2 Illustration details of the constructed MA-Net’s main modules.

Experiments and results

4.1 Dataset

In our experiments, we utilized two publicly available cell nucleus segmentation datasets, namely MoNuSeg [26] and TNBC [27]. The Indian Institute of Technology Guwahati shared their diligently prepared MoNuSeg dataset. It includes 44 H&E stained images sampled from 7 different organs, officially using 30 images for the training set and the remaining 14 images for the testing set. We divided each image of original size 10002 into 4 parts of the same size 5002 then resize them into 5122 as our method inputs, enlarging the dataset by 4 times.

The segmentation and stitching of images are commonly used data augmentation methods, such as CutMix [28] and CutOut [29]. The data augmentation method we employ is rational and effective, it serves the purpose of increasing the amount of data. Although the process of segmentation may result in some complete cells being divided into two parts, it does not have a detrimental impact on the segmentation results.

This is because both the MoNuSeg and TNBC datasets are obtained by splitting huge-sized whole-slide images (WSIs). Due to the excessively large size of WSIs, it is necessary to divide them into smaller-sized images before they can be directly input into the network, the division of cell nuclei into two halves is inevitably present in the original data of these two datasets. The image augmentation method we utilized only changes the size of the input image without altering the scaling level, thus it does not lead to any loss of information contained in the image. Consequently, this method does not have adverse effects on the segmentation results.

The TNBC dataset is comprised of 50 histopathology images at the size of 5122, divided into 11 groups. There are no official training sets or testing sets. So we chose 1 image from each group to make up our testing set, and the remaining 39 images for the training set. Both datasets include pixel-level annotated mask images for the cell nuclei semantic segmentation task and we set 30% of the training set as our experimental validation set.

4.2 Implementation details

Firstly, we conducted an ablation experiment on the MoNuSeg dataset to compare the impact of each module of MA-Net on the final segmentation results. Then we used MA-Net and other advanced methods to carry out comparative experiments respectively and obtained the experimental results.

Adam [30] optimizer would be the first choice for training most deep learning models. We used it and set its initial learning rate as 0.001 and decay rate as 0.1. The training set was not augmented, we set the batch size as 4 in both the training and the testing phases. The experimental models were trained by 100 epochs in all experiments without any pre-trained parameters and the convolution channels of the first convolution layer were set 64 to all models. The experiment was run on a single NVIDIA TESLA V100 GPU with Keras [31] and Tensorflow [32].

The loss function is of great significance to the convergence speed and final effect of the model. All models used in this experiment are end-to-end deep learning models, the segmentation results consist of two classes: foreground (cell nuclei) and background (non-cell nuclei). Essentially, it involves performing pixel-level binary classification on the image, Binary Cross Entropy(BCE) is the most common loss function for binary classification problem as shown in Eq 3. The BCE loss only focuses on the overall pixel classification accuracy and is independent of the foreground’s area ratio. However, the dataset contains a significant variation in cell nucleus sizes, and the BCE loss may not be sensitive enough to accurately classify small-size nuclei. We decided to combine Dice coefficient loss [33] in Eq 4 and BCE loss as loss function in Eq 5 to alleviate the problem of small-size targets detecting. LBCE=−1n∑i(gilnpi+(1−gi)ln(1−pi)) (3)

LDice=1−2∑ipigi∑ipi+∑igi (4)

LBCE−Dice=βLDice+(1−β)LBCE (5)

In Eqs 3 to 5, pi stands for pixel-level predicted probability. gi stands for pixel-level ground truth. β stands for the weight coefficient and we set it to 0.75 in experiments.

To evaluate the performance, we adopt a total of four evaluation indicators, which are Precision(Pre), Accuracy(Acc), Dice coefficient(Dice), and MIoU. They have been commonly and widely used to evaluate the effect of semantic segmentation. Pre=TPTP+FP (6)

Acc=TP+TNTP+FP+TN+FN (7)

Dice=2TP2TP+FP+FN (8)

MIoU=12(TPFN+FP+TP+TNFN+FP+TN) (9)

In Eqs 6 to 9, TP, TN, FP, FN represented the number of true positive, true negative, false positive, false negative pixels, respectively.

Fig 3 shows the training and inference scheme for nuclei segmentation using our constructed MA-Net. In the training stage, we cut the training images and their corresponding ground truth images into several patches and resize them to size 5122, then train the MA-Net model. In the inference stage, we used the same way above to get the testing patches, then fed them to the trained MA-Net and got the segmented result.

10.1371/journal.pone.0308326.g003 Fig 3 Proposed training and inference to segment nuclei using MA-Net.

4.3 Results and discussion

Our constructed MA-Net is composed of five extra modules based on U-Net. We conducted a simple Ablation Experiment on the MoNuSeg data set to verify the impact of these modules on the final segmentation results.

The Residual module is added to each convolution block, and its computational cost is relatively small compared with the other 4 modules, we call it the small module for the moment. The other four modules, the Attention Gate module, the Context Extractor module, the Down-sampling Fusion Module, and the Up-sampling Fusion Module, are added in fixed places with a relatively large cost of computation, which can be called big modules for the moment.

As Table 1 shows, after adding residual connections to the convolutional blocks of the original U-Net, the segmentation performance of the model has improved, on this basis, we sequentially incorporate other improvement modules into the network and evaluate the network performance. The experimental results demonstrate that the performance of the model improves regardless of which major module is individually incorporated, The improvement brought by the Attention Gate module is particularly evident. This phenomenon indicates the necessity of further processing the feature maps extracted by the encoder. By introducing spatial attention in the skip-connection structure, the response of relevant regions in the feature maps is enhanced, thereby restoring more image details during the decoding stage. Furthermore, by incorporating the upsampling fusion module and downsampling fusion module alongside the attention module, the segmentation performance is further improved, This aligns with our improvement strategy, which suggests that the performance of the original U-Net is hindered by the loss of information during downsampling. Introducing more image detail information during the decoding stage can enhance the segmentation results.

10.1371/journal.pone.0308326.t001 Table 1 Ablation experimental results.

Res for the Residual module, AG for the Attention Gate module, CE for the Context Extractor module, Down for the Down-sampling Fusion Module, and Up for the Up-sampling Fusion Module.

Model	Pre(↑)	Acc(↑)	Dice(↑)	MIoU(↑)	
1.U-Net	0.7013	0.8864	0.7393	0.7235	
2.U-Net+Res	0.7162	0.8915	0.7552	0.7467	
3.U-Net+Res+AG	0.7332	0.9056	0.7680	0.7602	
4.U-Net+Res+CE	0.7185	0.8976	0.7647	0.7439	
5.U-Net+Res+Down+Up	0.7218	0.9040	0.7664	0.7523	
6.U-Net+Res+AG+CE	0.7355	0.9085	0.7734	0.7635	
7.U-Net+Res+AG+Down+Up	0.7416	0.9086	0.7707	0.7637	
8.U-Net+Res+CE+Down+Up	0.7303	0.9019	0.7692	0.7606	
9.U-Net+ALL	0.7803	0.9204	0.7936	0.7826	

At the same time, we noticed that the DAC module does not exhibit significant improvements when used alone (combined with U-Net and residual modules as the baseline network). However, when used in conjunction with other modules, it can greatly enhance the performance of the network. The DAC module allows for the extraction of higher-level abstract features. However, the DAC module is located at the bottom of the U-shaped structure, specifically at the end of the encoder. The input feature maps of this module contain advanced semantic information but have lost a significant amount of spatial information. Using the DAC module alone cannot effectively address the issue of information loss and therefore cannot fully leverage its advantages. However, when used in conjunction with other modules designed to enhance detail information, it achieves significant segmentation results and receives the highest evaluation scores.

In our proposed MA-Net, the aggregation of multiple modules is not a simple additive relationship. Each module has its own strengths and weaknesses. The aggregation of these five modules compensates for the shortcomings of individual modules and highlights their respective advantages. As a result, it forms a new model that achieves stronger segmentation performance by leveraging the strengths of each module.

In order to more intuitively illustrate the advantages of MA-Net in nuclear segmentation, we randomly selected 3 images from the testing set of the MoNuSeg dataset and the TNBC dataset respectively, and displayed the segmentation results comparison between MA-Net and other methods in Fig 4. In these figures, the red box represents the locally enlarged image, and the orange box is the detail we focus on.

10.1371/journal.pone.0308326.g004 Fig 4 Comparison of the segmentation results obtained by 3 different models on the testing set of the MoNuSeg and the TNBC.

After being preprocessed, there are 120 images in the MoNuSeg dataset and 39 images in the TNBC dataset for model training. In supervised deep learning, the more training sets, the better results that model will produce to a certain degree. In Fig 4, it can be seen from the details of the display segmentation results figure that more training sets (the MoNuSeg training sets) can better enable the models to learn more feature information and produce better segmentation performance in the details.

While in the TNBC dataset, due to fewer training sets, most models could roughly predict the cell nuclei, it is not enough for the models to deal with the problems like multiple nuclear overlaps and unclear boundaries. The MA-Net we constructed still shows stable and accurate segmentation ability when there are few training sets, which proves that the model does have good robustness.

Compared with the detail in the orange boxes, we can see that in Fig 4A, Attention-UNet and CE-Net both have false recognition of smaller nuclei and a small number of intercellular adhesions exist. In Fig 4B, U-Net and UNet++ both have intercellular adhesions, while MA-Net shows a strong ability to deal with the problems of multiple nuclear overlaps, intercellular adhesions, unclear nuclear boundaries, and smaller nuclei misprediction.

In this cell nuclei segmentation task, the method we proposed is compared to the vanilla U-Net and other state-of-the-art methods. Our model is improved according to the ideas of several other models and harmoniously integrates the advantages of these models. Therefore, it can better extract the feature map of nuclei in the encoder path, and in the decoder path, the image details that may be lost during down-sampling are better restored with the help of concatenating connection and our proposed up-sampling fusion module. It did achieve a higher evaluation score than the other seven models. MA-Net achieves 0.7948 in Precision score, 0.9413 in Accuracy score, 0.8085 in Dice score, and 0.8047 in MIoU score on the MoNuSeg testing set and 0.7803 in Precision score, 0.9204 in Accuracy score, 0.7936 in Dice score, and 0.7826 in MIoU score on the TNBC testing set.

Table 2 shows the evaluation scores of eight cell nuclei segmentation models on the MoNuSeg dataset and TNBC dataset respectively. As one can see, the method we proposed achieves a remarkable improvement and the best evaluation results on almost every evaluation indicator on the MoNuSeg testing set. Specifically, our method achieves 0.0639 higher than SegNet [34] in MIoU score and 0.0790 higher than the vanilla U-Net in Precision score. Also, it achieves 0.0289 higher than CE-Net and 0.0135 higher than UNet++, the best-known U-Net variant in Dice score. On the TNBC testing set, our method still achieves the greatest score on almost every evaluation indicator, although the score gap with other methods is not so significant as the experiment results on the MoNuseg testing dataset. Specifically, MA-Net achieves 0.0515 points higher than the vanilla U-Net in MIoU score and about 0.02 higher than UNet++, CE-Net, and Attention U-Net in Accuracy score. Also, it achieves over 0.2 higher than SegNet and DU-Net [35] in Dice score. Although MA-Net did not get the best result in Precision score, it still ranked in second place. The TNBC dataset consists of 50 histopathology images, it is a relatively small dataset compared with the MoNuSeg dataset. The score of the evaluation result on the MoNuSeg is lower than that on TNBC on average. We could say that the MoNuSeg dataset is more challenging, while the TNBC dataset could test the robustness of the models according to the unstable evaluation result of SegNet and DU-Net. The segmentation task on the TNBC dataset is relatively easier for most model.

10.1371/journal.pone.0308326.t002 Table 2 Comparison of segmentation results of all the experimental methods on the MoNuSeg testing set and the TNBC testing set.

Model	Pre(↑)	Acc(↑)	Dice(↑)	MIoU(↑)	
MoNuSeg	TNBC	MoNuSeg	TNBC	MoNuSeg	TNBC	MoNuSeg	TNBC	
U-Net[15]	0.7013	0.7695	0.8864	0.9238	0.7393	0.7383	0.7235	0.7532	
SegNet[34]	0.6621	0.4799	0.8699	0.8479	0.7308	0.5208	0.7185	0.5945	
Attention-UNet[20]	0.7332	0.7966	0.9056	0.9226	0.7680	0.6902	0.7602	0.7320	
CE-Net[22]	0.7185	0.7910	0.8976	0.9271	0.7647	0.7397	0.7439	0.7557	
BiO-Net[23]	0.7728	0.7539	0.9142	0.9323	0.7769	0.7813	0.7682	0.7811	
UNet++[21]	0.7614	0.7919	0.9136	0.9272	0.7801	0.7384	0.7726	0.7573	
DU-Net[35]	0.7218	0.7628	0.9040	0.9022	0.7664	0.5813	0.7583	0.6618	
MA-Net(Ours)	0.7803	0.7948	0.9204	0.9413	0.7936	0.8085	0.7826	0.8047	

In addition to the above evaluation indicators, we also used ROC curves to evaluate all the models, which is shown in Fig 5 In the ROC coordinates, the closer the ROC curve to the upper left corner, the more accurate the corresponding model is. In another way, the accuracy of the ROC curve can be more clearly judged by the area under the ROC curve (AUC) score, which is also marked in the lower right corner of the figure. As one can see, the AUC scores of our MA-Net in both datasets are the highest among all the models, achieving 0.9667 and 0.9730 respectively.

10.1371/journal.pone.0308326.g005 Fig 5 ROC curves in different models.

(a) ROC curves on the testing set of the MoNuseg dataset; (b) ROC curves on the testing set of the TNBC dataset.

We also compared MA-Net with advanced models based on self-attention mechanisms such as TransUnet [36]. Among them, TransUnet utilizes the Transformer for feature extraction in the encoder part. Due to the relatively small size of the dataset we used, making it difficult to train from scratch. thus, we initialized it with imagenet21k pre-trained parameters. UCTransNet [37] solely employs the Transformer module to process skip connections and does not use it for feature extraction. Hence, we directly trained UCTransNet without using pre-trained parameters, and the same applies to DCA-UNet [38] and MaxVit-Unet [39].

As shown in Table 3, except for MaxVit-Unet achieving slightly higher precision than MA-Net on the TNBC dataset, all other metrics were lower than those of MA-Net. The fundamental module employed in MaxVit-Unet is based on multi-axis attention [40], a sparse attention mechanism characterized by linear computational complexity and theoretically possessing a global receptive field. MA-Net is entirely implemented based on convolutional neural networks (CNNs) has relatively lower requirements for data volume during the training process. Therefore, it has an advantage when working with small-scale datasets.

10.1371/journal.pone.0308326.t003 Table 3 Comparison of MA-Net with attention-based state-of-the-art methods.

Model	Pre(↑)	Acc(↑)	Dice(↑)	MIoU(↑)	
MoNuseg	TNBC	MoNuseg	TNBC	MoNuseg	TNBC	MoNuseg	TNBC	
TransUnet[36]	0.779	0.799	0.906	0.932	0.788	0.781	0.768	0.781	
UCTransNet[37]	0.752	0.791	0.903	0.924	0.787	0.732	0.766	0.755	
DCA-UNet[38]	0.716	0.759	0.877	0.896	0.732	0.765	0.718	0.748	
MaxVit-Unet[39]	0.756	0.836	0.906	0.940	0.791	0.766	0.772	0.781	
MA-Net(Ours)	0.780	0.795	0.920	0.941	0.794	0.809	0.783	0.805	

Additionally, we tested our method on more datasets, including CryoNuSeg [41], BBBC039 [42] and kaggle 2018 data science bowl [43], all of which are nuclei segmentation datasets. We specifically compared Attention-UNet, CE-Net, and MaxVit-Unet with our MA-Net. These three models correspond to the common improvement strategies for UNet, namely enhancing the skip connection structure, improving the bottleneck block, and refining both the upsampling and downsampling paths, respectively.

The comparison results are shown in Table 4. Despite all three datasets being nuclear segmentation datasets, they differ in staining styles, cell densities, and image resolutions. Our model consistently achieved the best performance across all datasets, with the most significant improvement observed on the CryoNuSeg dataset. This demonstrates that MA-Net possesses strong generalization capabilities and can effectively adapt to the task of nuclear segmentation.

10.1371/journal.pone.0308326.t004 Table 4 Comparison results on supplementary datasets.

Dataset	Model	Pre(↑)	Acc(↑)	Dice(↑)	MIoU(↑)	
CryoNuseg	Attention-UNet	0.748	0.892	0.754	0.734	
CE-Net	0.706	0.888	0.761	0.734	
MaxVit-Unet	0.781	0.888	0.737	0.722	
Ours	0.781	0.901	0.769	0.750	
BBBC039	Attention-UNet	0.969	0.987	0.971	0.963	
CE-Net	0.970	0.986	0.970	0.962	
MaxVit-Unet	0.976	0.988	0.972	0.965	
Ours	0.976	0.987	0.973	0.965	
Data Science Bowl-2018	Attention-UNet	0.854	0.958	0.833	0.849	
CE-Net	0.894	0.955	0.824	0.842	
MaxVit-Unet	0.894	0.962	0.850	0.867	
Ours	0.894	0.964	0.852	0.869	

Table 5 lists the training speed and the number of parameters for each model. Although MA-Net achieves the best segmentation performance among the models, it has a larger number of parameters and requires longer training times. Future work will focus on reducing the model parameters and increasing computational speed while maintaining segmentation performance. This can be achieved by further improving the network structure using techniques such as depthwise separable convolutions and group convolutions.

10.1371/journal.pone.0308326.t005 Table 5 Comparison of the specific parameters of all the experimental methods.

Model	MoNuSeg	TNBC	Params (M)	model size (MB)	
Time for training (s/epoch)	time for training (s/epoch)	
U-Net	13.98	4.56	34	131	
SegNet	10.55	3.05	29	112	
Attention-UNet	12.19	5.25	34	133	
CE-Net	4.49	2.37	40	154	
BiO-Net	20.20	4.87	59	228	
UNet++	22.56	8.28	36	138	
DU-Net	23.20	8.41	60	230	
TransUnet	25.27	11.46	101	404	
UCTransNet	24.00	8.91	64	257	
DCA-UNet	20.00	6.57	42	170	
MaxVit-Unet	18.37	4.48	24	97	
MA-Net(Ours)	24.84	10.36	132	505	

Conclusions

Computer-aided automatic Cell Nuclei Segmentation is significant in medical image recognition and histopathological diagnosis assistance. In this paper, we have proposed a novel improved U-Net method, named MA-Net, used for cell nuclei segmentation. Compared with the vanilla U-Net structure, the proposed MA-Net aggregates the advantages of the multi-modules including, strengthened the contracting path by the Down-sampling Fusion Module, the expansive path by the Up-sampling Fusion Module, the skip-connection part by the Attention Gate Module, the Context Extractor Module and the Residual Module. MA-Net can obtain more original information from inputs images, obtain more information in deep-level and retain spatial information and use the information for image segmentations more precisely. The results of our experiment on the MoNuSeg and TNBC datasets show that MA-Net is able to improve the cell nuclei segmentation effect, achieving 0.7826 and 0.8047 in MIoU score on the above datasets, which is 0.0591 and 0.0515 higher than vanilla U-Net. Also, it achieved the highest AUC score of 0.9667 and 0.9730 on the above typical datasets. It turns out that our method has reasonable segmentation performance and generalization ability. However, MA-Net also has its shortcoming that it spends more time in the training phase than the traditional one. MA-Net could be applied to a new semantic field as long as using the new training images and the corresponding labeled ground truth to train it. Our work demonstrates that, compared to improving a single part of the U-Net, simultaneously optimizing the structures of the encoder, decoder, skip connections, and bottleneck is more beneficial for feature extraction and fusion, thereby enhancing model performance. Future research will focus on model lightweighting, aiming to reduce the number of parameters while maintaining model performance.

10.1371/journal.pone.0308326.r001
Decision Letter 0
Li Xiaowei Academic Editor
© 2024 Xiaowei Li
2024
Xiaowei Li
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version0
8 Mar 2024

PONE-D-24-00318Multifunctional Aggregation Network of Cell Nuclei Segmentation Aiming Histopathological Diagnosis Assistance: A New MA-Net ConstructionPLOS ONE

Dear Dr. Zhao,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Apr 22 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Xiaowei Li

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure: "National Key Research and Development Program of China (2021YFA1200904 and 2020YFA0710700), the National Natural Science Foundation of China (31971311, 12375326), the Innovation Program for IHEP (E35457U210)."

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript: "This work was supported by the National Key Research and Development Program of China (2021YFA1200904 and 2020YFA0710700), the National Natural Science Foundation of China (31971311, 12375326), the Innovation Program for IHEP (E35457U210)."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "National Key Research and Development Program of China (2021YFA1200904 and 2020YFA0710700), the National Natural Science Foundation of China (31971311, 12375326), the Innovation Program for IHEP (E35457U210)."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper introduces a method U-Net based model, MA-Net(Multifunctional Aggregation Network), to accurately segmenting nuclei and the model utilized the subsampling fusion module and upsampling fusion module on the U-Net encoder and decoder respectivel. Overall, the manuscript has a good structure and better validation of results via ablation studies. However, I suggest the manuscript to undergo major revision and resubmit for another quick review.

The following are some major corrections that can be incorporated to improve quality:

1) More experiments on several other datasets such as Fluorescence Microscopy Image Dataset and any other medical image segmentation datasets has to be performed to effectively understand the contribution of the proposed approach and validation of methodology. I suggest authors to make more evaluation on similar datasets.

2) Similarly, a detailed comparison on several recent methods (papers published on or after 2020) such as TransNuSeg, CellViT, has to be performed. The Authors can refer to https://paperswithcode.com/sota/medical-image-segmentation-on-monuseg for more details on the state-of-the-art methods.

3) The Abstract is too long and can be condensed highlighting the novelty and best results on datasets.

4) The recommend authors to highlight the novelty in methodology. The manuscript is very similar to CE-Net (Context Encoder Network for 2D Medical Image Segmentation) thereby might be a further extension to cell nuclei segmentation application.

5) I suggest authors to add future scope for further improvements and research in this direction and add and explain a figure for some of the failure cases of using this methodology.

Minor corrections:

1) Extra full stop on the first line in Abstract. I recommend Authors to carefully read and review the manuscript for any typographical errors.

2) Equations can be enhanced in the methodology. I suggest authors to use LaTeX while formulation equations.

3) Hyperlinks has to be added to citations and references for helping readers to quickly refere the cited studies.

Reviewer #2: Dear Editor, I want to thank you very much for the invitation to review a manuscript entitled "Multifunctional Aggregation Network of Cell Nuclei Segmentation Aiming Histopathological Diagnosis Assistance: A New MA-Net Construction" for your journal. In fact, the authors raised an assessment about a U-Net based deep learning model, called MA-Net(Multifunctional Aggregation Network), to accurately segmenting nuclei from H&E stained images. The improved model utilized the subsampling fusion module and upsampling fusion module on the U-Net encoder and decoder respectively, to better restore the detailed information lost during the image encoding process and avoid blurry boundaries in segmentation result. however, I have some minor points:

comment 1: What about the accrual result of this technique with different types of cancers? is it accurate?

comment 2: "Experimental results substantiate that our proposed MA-Net outperforms other improved models in terms of cell nuclei segmentation tasks" I think this sentence is confusing, how can you support this information?

Comment 3: Some editing for English language is required throughout the manuscript due to too many mistakes.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Mohamed Hadi Mohamed Abdelhamid

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment Submitted filename: Review Comments_PONE-D-24-00318.docx

10.1371/journal.pone.0308326.r002
Author response to Decision Letter 0
Submission Version1
21 Jun 2024

Response to reviewers

Major Corrections

1.Comment: More experiments on several other datasets such as Fluorescence Microscopy Image Dataset and any other medical image segmentation datasets has to be performed to effectively understand the contribution of the proposed approach and validation of methodology. I suggest authors to make more evaluation on similar datasets.

Reply:

Thank you for your suggestions. We have conducted additional experiments using more datasets to further demonstrate the effectiveness of our proposed method. The results and analysis of these experiments have been added to Section 4.3 "Results and Discussion" in the main text.

2.Comment: Similarly, a detailed comparison on several recent methods (papers published on or after 2020) such as TransNuSeg, CellViT, has to be performed. The Authors can refer to https://paperswithcode.com/sota/medical-image-segmentation-on-monuseg for more details on the state-of-the-art methods.

Reply:

Thank you for your suggestion. To further validate the effectiveness of our proposed method, we conducted additional experiments using state-of-the-art Transformer-based approaches post-2023 on the MoNuSeg and TNBC datasets. The results and analysis of these experiments are presented in Section 4.3, Results and Discussion.

3.Comment: The Abstract is too long and can be condensed highlighting the novelty and best results on datasets.

Reply:

Thank you for reviewing our paper and providing your suggestions. We understand that the abstract is an essential part for readers to quickly grasp the research content and outcomes. We have made the necessary simplifications and optimizations to the abstract in the revised manuscript.

4.Comment: The recommend authors to highlight the novelty in methodology. The manuscript is very similar to CE-Net (Context Encoder Network for 2D Medical Image Segmentation) thereby might be a further extension to cell nuclei segmentation application.

Reply:

Thank you for reviewing our paper and for your valuable comments. While both our work and CE-Net focus on semantic image segmentation, and thus inevitably share some similarities, it is important to emphasize that our work is not an extension or continuation of CE-Net. Our design approach has several key differences:

Firstly, CE-Net's improvements mainly focus on the Bottleneck part of the U-Net network, whereas our work involves improvements to the encoder, decoder, skip connections, and Bottleneck of the U-Net. We conducted comparative experiments using CE-Net as a baseline model, and the results demonstrate the effectiveness of our proposed feature fusion strategy and spatial attention mechanism. Secondly, our work differs from CE-Net in terms of training setup. Considering the characteristics of cell nucleus medical imaging, we adopted a hybrid loss function different from CE-Net and employed different data augmentation methods. Lastly, in terms of application, CE-Net is applied to retinal vessel images, while our work primarily targets cell nucleus segmentation.

Regarding your concerns about the novelty of our method, we have revised the abstract and relevant sections of the main text to further emphasize the originality and novelty of our work.

5.Comment: I suggest authors to add future scope for further improvements and research in this direction and add and explain a figure for some of the failure cases of using this methodology.

Reply:

Thank you for your suggestions. We also recognize the importance of analyzing the current limitations and considering improvements for future work. We have added relevant content at the end of Section 4.3 "Results and Discussion" and in Section 5 "Conclusions".

Minor Corrections

1.Comment: Extra full stop on the first line in Abstract. I recommend Authors to carefully read and review the manuscript for any typographical errors.

Reply:

Thank you for pointing out the issues. We have carefully reviewed the manuscript of the article and made corrections to the formatting and textual errors.

2.Comment: Equations can be enhanced in the methodology. I suggest authors to use LaTeX while formulation equations.

Reply:

Thank you for your suggestions. We have carefully reviewed and verified the formulas in the text, and we have used MathType to rewrite them, correcting any formatting and layout errors.

3.Comment: Hyperlinks has to be added to citations and references for helping readers to quickly refere the cited studies.

Reply:

Thank you for your suggestion. We have added hyperlinks to all the references cited in the article. Readers can now click on the citation numbers in the text to navigate to the corresponding references.

Special thanks to you for your good comments.

Attachment Submitted filename: Response to Reviewer Comments.docx

10.1371/journal.pone.0308326.r003
Decision Letter 1
Li Xiaowei Academic Editor
© 2024 Xiaowei Li
2024
Xiaowei Li
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version1
23 Jul 2024

Multifunctional Aggregation Network of Cell Nuclei Segmentation Aiming Histopathological Diagnosis Assistance: A New MA-Net Construction

PONE-D-24-00318R1

Dear Dr. Zhao,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Xiaowei Li

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I thank the Authors for thoroughly and carefully revising the manuscript and it is now much better, and interesting for the readers.

Reviewer #2: Dear Editor and Authors,

Thank you for your work on this manuscript. We appreciate your efforts in addressing all the comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Mohamed Hadi Mohamed Abdelhamid

**********

10.1371/journal.pone.0308326.r004
Acceptance letter
Li Xiaowei Academic Editor
© 2024 Xiaowei Li
2024
Xiaowei Li
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
30 Jul 2024

PONE-D-24-00318R1

PLOS ONE

Dear Dr. Zhao,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Xiaowei Li

Academic Editor

PLOS ONE
==== Refs
References

1 Siegel R L , Miller K D , Fuchs H E , et al . Cancer statistics, 2022[J]. CA: A Cancer Journal for Clinicians, 2022, 72 (1 ). 10.3322/caac.21708.
2 Benson A B , D’Angelica M I , Abbott D E , et al . Hepatobiliary cancers, version 2.2021, NCCN clinical practice guidelines in oncology[J]. Journal of the National Comprehensive Cancer Network, 2021, 19 (5 ): 541–565. 10.6004/jnccn.2021.0022.34030131
3 Zhang Z , Chen P , McGough M , et al . Pathologist-level interpretable whole-slide cancer diagnosis with deep learning[J]. Nature Machine Intelligence, 2019, 1 (5 ): 236–245. 10.1038/s42256-019-0082-8.
4 Çayır S , Serteli E A , Ayalti S , et al . Segmentation of the main structures in Hematoxylin and Eosin images[C]//2018 26th Signal Processing and Communications Applications Conference (SIU). IEEE, 2018: 1–4. 10.1109/SIU.2018.8404814.
5 Shen T , Wang Y . Medical image segmentation based on improved watershed algorithm[C]//2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). IEEE, 2018: 1695–1698. 10.1109/IAEAC.2018.8577861.
6 Beucher S , Meyer F . The morphological approach to segmentation: the watershed transformation[M]// Mathematical morphology in image processing. CRC Press, 2018: 433–481.
7 Prakarsa Mandyartha E , Anggraeny F T , Muttaqin F , et al . Global and Adaptive Thresholding Technique for White Blood Cell Image Segmentation[C]// Journal of Physics Conference Series. 2020, 1569 (2 ): 022054. 10.1088/1742-6596/1569/2/022054.
8 Gu L , Zheng Y , Bise R , et al . Semi-supervised learning for biomedical image segmentation via forest oriented super pixels (voxels)[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2017: 702–710. 10.1007/978-3-319-66182-7_80.
9 Tan M , Pang R , Le Q V . Efficientdet: Scalable and efficient object detection[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 10781–10790. 10.1109/CVPR42600.2020.01079.
10 Sun P , Zhang R , Jiang Y , et al . Sparse r-cnn: End-to-end object detection with learnable proposals[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 14454–14463. 10.1109/CVPR46437.2021.01422.
11 Zhang Y , Liu H , Hu Q . Transfuse: Fusing transformers and cnns for medical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2021: 14–24. 10.1007/978-3-030-87193-2_2.
12 Liu C , Chen L C , Schroff F , et al . Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 82–92. 10.1109/CVPR.2019.00017.
13 Chen J , Frey E C . Medical Image Segmentation via Unsupervised Convolutional Neural Network[C]// Medical Imaging with Deep Learning (MIDL) 2020. 2020 . 10.48550/arXiv.2001.10155.
14 Tournier J D , Smith R , Raffelt D , et al . MRtrix3: A fast, flexible and open software framework for medical image processing and visualisation[J]. Neuroimage, 2019, 202 : 116137. 10.1016/j.neuroimage.2019.116137.31473352
15 Ronneberger O , Fischer P , Brox T . U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 234–241. Springer (2015) 10.1007/978-3-319-24574-4_28.
16 Ma D , Lu D , Heisler M , et al . Cascade Dual-Branch Deep Neural Networks for Retinal Layer and Fluid Segmentation of Optical Coherence Tomography Incorporating Relative Positional Map[C]// Medical Imaging with Deep Learning 2020. 2020.
17 Sarhan Abdullah , Ali Al-Khaz’Aly Adam Gorner , et al . Utilizing Transfer Learning and a Customized Loss Function for Optic Disc Segmentation from Retinal Images[C]// Proceedings of the Asian Conference on Computer Vision (ACCV), 2020.2020 .
18 Berk , Norman , Valentina , et al . Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry.[J]. Radiology, 2018. 10.1148/radiol.2018172322.
19 Mahmud T , Rahman A , Fattah S A , et al . CovSegNet: A Multi Encoder–Decoder Architecture for Improved Lesion Segmentation of COVID-19 Chest CT Scans[J]. IEEE, 2021(3). 10.1109/TAI.2021.3064913.
20 Oktay Ozan , et al . "Attention u-net: Learning where to look for the pancreas." arxiv preprint arxiv:1804.03999 (2018).
21 Zhou Z , Siddiquee M M R , Tajbakhsh N , et al . UNet++: A Nested U-Net Architecture for Medical Image Segmentation[C]// 4th Deep Learning in Medical Image Analysis (DLMIA). 2018. 10.1007/978-3-030-00889-5_1.
22 Gu Z , Cheng J , Fu H , et al . CE-Net: Context Encoder Network for 2D Medical Image Segmentation[J]. IEEE Transactions on Medical Imaging, 2019:1–1. 10.1109/TMI.2019.2903562.
23 Xiang T , Zhang C , Liu D , et al . BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 74–84. Springer(2020). 10.1007/978-3-030-59710-8_8.
24 Chen L C , Papandreou G , Kokkinos I , et al . Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40 (4 ): 834–848. 10.1109/TPAMI.2017.2699184.28463186
25 Huang G , Liu Z , Laurens V , et al . Densely Connected Convolutional Networks[J]. IEEE Computer Society, 2016. 10.1109/CVPR.2017.243.
26 Kumar N , Verma R , Sharma S , et al . A dataset and a technique for generalized nuclear segmentation for computational pathology[J]. IEEE Transactions on Medical Imaging (TMI) 36 (7 ), 1550–1560 (2017). 10.1109/TMI.2017.2677499.
27 Naylor P , Laé M , Reyal F , Walter T . Segmentation of nuclei in histopathology images by deep regression of the distance map[J]. IEEE Transactions on Medical Imaging (TMI) 38 (2 ), 448–459 (2018). 10.1109/TMI.2018.2865709.
28 Yun S , Han D , Oh S J , et al . Cutmix: Regularization strategy to train strong classifiers with localizable features[C]// Proceedings of the IEEE/CVF international conference on computer vision. 2019: 6023–6032.
29 DeVries T , Taylor G W . Improved regularization of convolutional neural networks with cutout[J]. arXiv preprint arXiv:1708.04552, 2017.
30 Kingma D.P. , Ba J. Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015).
31 Chollet F. and others, Keras . GitHub. [Online]. Available: https://github.com/keras-team/keras.
32 Abadi M. , Agarwal A. , Barham P. , Brevdo E. , Chen Z. , Citro C. , Corrado G. S. , Davis A. , Dean J. , Devinet al M. , “TensorFlow: Large Scale Machine Learning on Heterogeneous Distributed Systems,” arXiv preprint arXiv:1603.04467, 2015.
33 Crum W R , Camara O , Hill D L G . Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis[J]. IEEE Transactions on Medical Imaging, 2006, 25 :p.1451–1461. doi: 10.1109/TMI.2006.880587 17117774
34 Badrinarayanan V , Kendall A , Cipolla R . SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017:1–1. 10.17863/CAM.17966.
35 AlGhamdi M , Abdel-Mottaleb M , Collado-Mesa F . DU-Net: convolutional network for the detection of arterial calcifications in mammograms[J]. IEEE transactions on medical imaging, 2020, 39 (10 ): 3240–3249. 10.1109/TMI.2020.2989737.32324546
36 Chen J , Lu Y , Yu Q , et al . Transunet: Transformers make strong encoders for medical image segmentation[J]. arxiv preprint arxiv:2102.04306, 2021.
37 Wang H , Cao P , Wang J , et al . Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer[C]// Proceedings of the AAAI conference on artificial intelligence. 2022, 36 (3 ): 2441–2449.
38 Ates G C , Mohan P , Celik E . Dual cross-attention for medical image segmentation[J]. Engineering Applications of Artificial Intelligence, 2023, 126 : 107139.
39 Khan A R , Khan A . MaxViT-UNet: Multi-axis attention for medical image segmentation[J]. arxiv preprint arxiv:2305.08396, 2023.
40 Tu Z , Talebi H , Zhang H , et al . Maxvit: Multi-axis vision transformer[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2022: 459–479.
41 Mahbod A , Schaefer G , Bancher B , et al . CryoNuSeg: A dataset for nuclei instance segmentation of cryosectioned H&E-stained histological images[J]. Computers in biology and medicine, 2021, 132 : 104349.33774269
42 Ljosa V. , Sokolnicki K. & Carpenter A. Annotated high-throughput microscopy image sets for validation. Nat Methods 9 , 637 (2012). doi: 10.1038/nmeth.2083 22743765
43 Goodman Allen , Carpenter Anne , Park Elizabeth , jlefman-nvidia , Josette_BoozAllen , Kyle , Maggie , Nilofer , Peter Sedivec , Will Cukierski . (2018). 2018 Data Science Bowl. Kaggle. https://kaggle.com/competitions/data-science-bowl-2018.
