
==== Front
Ann Med
Ann Med
Annals of Medicine
0785-3890
1365-2060
Taylor & Francis

39258876
10.1080/07853890.2024.2399759
2399759
Version of Record
Research Article
Oncology
Predicting BRCA mutation and stratifying targeted therapy response using multimodal learning: a multicenter study
Y. LI et al.
Li Yi ab
Xiong Xiaomin ab
Liu Xiaohua c
Xu Mengke b
Yang Boping d
Li Xiaoju e
Li Yu e
Lin Bo b
https://orcid.org/0000-0001-8693-3060
Xu Bo ab
a School of Medicine, Chongqing University, Chongqing, China
b Chongqing Key Laboratory for Intelligent Oncology in Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
c Bioengineering College of Chongqing University, Chongqing, China
d Department of General Gynecology, Women and Children’s Hospital of Chongqing Medical University, Chongqing Health Center for Women and Children, Chongqing, China
e Department of Pathology, Chongqing University Cancer Hospital and School of Medicine, Chongqing University, Chongqing, China
Supplemental data for this article can be accessed online at https://doi.org/10.1080/07853890.2024.2399759.

CONTACT Bo Xu xubo731@cqu.edu.cn Chongqing Key Laboratory for Intelligent Oncology in Breast Cancer, Chongqing University Cancer Hospital and School of Medicine, Chongqing University, Chongqing 400030, China
Bo Lin linbo@cqu.edu.cn Chongqing Key Laboratory for Intelligent Oncology in Breast Cancer, Chongqing University Cancer Hospital, Chongqing 400030, China
11 9 2024
2024
11 9 2024
56 1 239975914 3 2024
29 7 2024
30 7 2024
KnowledgeWorks Global Ltd.11 9 2024
published online in a building issue11 9 2024
© 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
2024
The Author(s)
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.

Abstract

Background

The status of BRCA1/2 genes plays a crucial role in the treatment decision-making process for multiple cancer types. However, due to high costs and limited resources, a demand for BRCA1/2 genetic testing among patients is currently unmet. Notably, not all patients with BRCA1/2 mutations achieve favorable outcomes with poly (ADP-ribose) polymerase inhibitors (PARPi), indicating the necessity for risk stratification. In this study, we aimed to develop and validate a multimodal model for predicting BRCA1/2 gene status and prognosis with PARPi treatment.

Methods

We included 1695 slides from 1417 patients with ovarian, breast, prostate, and pancreatic cancers across three independent cohorts. Using a self-attention mechanism, we constructed a multi-instance attention model (MIAM) to detect BRCA1/2 gene status from hematoxylin and eosin (H&E) pathological images. We further combined tissue features from the MIAM model, cell features, and clinical factors (the MIAM-C model) to predict BRCA1/2 mutations and progression-free survival (PFS) with PARPi therapy. Model performance was evaluated using area under the curve (AUC) and Kaplan-Meier analysis. Morphological features contributing to MIAM-C were analyzed for interpretability.

Results

Across the four cancer types, MIAM-C outperformed the deep learning-based MIAM in identifying the BRCA1/2 genotype. Interpretability analysis revealed that high-attention regions included high-grade tumors and lymphocytic infiltration, which correlated with BRCA1/2 mutations. Notably, high lymphocyte ratios appeared characteristic of BRCA1/2 mutations. Furthermore, MIAM-C predicted PARPi therapy response (log-rank p < 0.05) and served as an independent prognostic factor for patients with BRCA1/2-mutant ovarian cancer (p < 0.05, hazard ratio:0.4, 95% confidence interval: 0.16–0.99).

Conclusions

The MIAM-C model accurately detected BRCA1/2 gene status and effectively stratified prognosis for patients with BRCA1/2 mutations.

Keywords

BRCA
targeted therapy
deep learning
cancer
multimodal
interpretability
National Natural Science Foundation of China 10.13039/501100001809 81974464 61906022 Chongqing Natural Science Foundation cstc2020jcyj-msxmX0482 Chongqing University Research Fund 2021CDJXKJC004 Chongqing Technology Innovation and Application Development Project CSTB2023TIAD-KPX0050 This work was supported by the National Natural Science Foundation of China [81974464, 61906022], Chongqing Natural Science Foundation [cstc2020jcyj-msxmX0482], Chongqing University Research Fund [2021CDJXKJC004], and Chongqing Technology Innovation and Application Development Project [CSTB2023TIAD-KPX0050].
==== Body
pmcIntroduction

Breast cancer susceptibility genes, BRCA1 and BRCA2, serve as essential tumor suppressor genes pivotal in the DNA damage response, repair of DNA double-strand breaks, and transcriptional regulation [1,2]. Mutations in BRCA1/2 genes lead to homologous recombination repair deficiency (HRD), causing genomic instability that can potentially culminate in cancer development [3,4]. The cumulative risk of developing breast cancer for BRCA1/2 mutation carriers is estimated to be 39%–65%, while the risk of ovarian cancer is 11%–39% [5]. Additionally, BRCA1/2 mutations were associated with prostate cancer [6], pancreatic cancer [7], and other cancer types [8]. Importantly, individuals with BRCA1/2 mutations show a high sensitivity to poly (ADP-ribose) polymerase inhibitors (PARPi), which have been shown to extend the progression-free survival (PFS) of patients with advanced ovarian cancer, especially those with germline and somatic BRCA1/2 mutations [9–11]. The status of BRCA1/2 is considered a predictive biomarker for PARPi therapy in ovarian, breast, prostate, and pancreatic cancers [12–15].

Depending on the type of cancer, different guidelines determine the priority of patients for BRCA1/2 genetic testing. The European Society for Medical Oncology (ESMO) [16] recommends that all non-mucinous ovarian cancer patients receive BRCA1/2 testing. The National Comprehensive Cancer Network (NCCN) [17] suggests that all pancreatic cancer patients undergo BRCA germline testing. As for breast cancer patients, the decision to test is based on specific criteria such as young age (≤50 years), a family history of cancer, and bilateral breast cancer [18]. However, in clinical practice, patients who meet these criteria often do not receive BRCA1/2 genetic testing due to the high costs, time consumption, and complexity of the testing process [19,20]. Furthermore, even among BRCA1/2-mutant ovarian cancer patients who receive PARPi treatment, 52% still experience disease progression after five years of follow-up [21]. This indicates a need for risk stratification for individuals with BRCA1/2 mutations based on their response to targeted therapy, which is crucial for guiding treatment decisions that cannot be determined through gene sequencing alone. Hence, there is an urgent need for the development of a new approach that can accurately detect the BRCA1/2 gene status and provide personalized stratification based on individual responses to targeted treatment.

Molecular alterations play a crucial role in cancer progression. These alterations lead to functional changes in tumor cells, affecting their morphology and microenvironment [22]. Hematoxylin and eosin (H&E)-stained pathological slides reveal these complex morphological changes, including cell and nucleus shapes and sizes, mitotic activity [23], and alterations in the extracellular matrix [24]. While subtle changes may be difficult to detect with the naked eye, artificial intelligence (AI) technology, particularly deep learning (DL) algorithms, enables the identification and quantification of these changes from digitized H&E slides [25]. Since 2018, several studies have demonstrated that DL can directly predict genetic mutations, such as EGFR, TP53, IDH, and microsatellite instability from digital whole-slide images (WSIs) [26–29]. However, previous studies that utilized DL to detect BRCA1/2 gene status from H&E images mainly focused on unimodal information, lacking the integration of multimodal data [30,31]. Both germline and somatic BRCA1/2 genetic mutations result in DNA damage repair defects, which can impact the morphology of tumor cells, specifically the size and shape of the nucleus [4]. Therefore, combining DL-extracted tissue features with cell features may yield promising results in predicting BRCA1/2 mutations. It is worth noting that previous studies only predicted BRCA1/2 gene status and did not effectively stratify patients with BRCA1/2 mutations based on their prognostic information.

In this study, we aimed to develop a multimodal framework that integrates tissue features, cell features, and clinical characteristics to detect BRCA1/2 gene status in ovarian, breast, prostate, and pancreatic cancers. In addition, we aimed to explore morphological features associated with BRCA1/2 gene status across these four cancer types and conduct quantitative analyses of the tumor microenvironments. To this end, we investigated the ability of the multimodal model to predict the PFS of ovarian cancer patients with BRCA1/2 mutations after receiving PARPi treatment.

Materials and methods

Study design and participants

The study design is illustrated in Figure 1. This study retrospectively collected data from 874 patients diagnosed with ovarian, breast, prostate, and pancreatic cancers at the Chongqing University Cancer Hospital (CUCH) between 1 January 2005 and 30 September 2023. And 1145  H&E-stained pathological slides were involved. The inclusion criteria were as follows: (1) Patients diagnosed with primary ovarian, breast, prostate, and pancreatic cancers; (2) Germline and/or somatic testing of BRCA1 and BRCA2 genes using next generation sequencing (NGS); (3) Availability of post-operative diagnostic pathology H&E images. The exclusion criteria were: (1) Patients diagnosed with metastatic ovarian, breast, prostate, and pancreatic cancers; (2) Patients who received neoadjuvant therapy before surgery; (3) Lack of post-operative pathology slides or poor quality of H&E pathology images. We collected clinical information, BRCA1/2 gene sequencing outcomes, and post-operative H&E pathology images for all patients. Clinical characteristics were collected, including age, stage, and grade. To determine the BRCA1/2 status, we conducted clinical significance analyses on all variants based on the classification criteria [32,33] for BRCA1 and BRCA2 genes. Variants identified as pathogenic or likely pathogenic were categorized into the BRCA1/2 mutation group, while variants deemed likely benign or benign were classified into the BRCA1/2 wild type group. Variants of uncertain significance, which have uncleared clinical significance in guiding treatment, were not included in the study. This study was approved by the Ethics Committee of Chongqing University Cancer Hospital (Ethics number: CZLS2023213-A), and informed consent was waived due to the study’s retrospective design. All procedures performed in this study adhered to the principles of the Declaration of Helsinki.

Figure 1. Workflow of the MIAM-C model and study design. Firstly, the WSIs were preprocessed for feature extraction. After feature evaluation and modeling, three sets of features were generated: tissue, cell, and clinical features. These features were then integrated to construct the MIAM-C model. Secondly, the performance of MIAM-C in detecting BRCA1/2 gene status was validated in the testing sets across four cancer types. Thirdly, attention mechanism-based visualization techniques were employed to produce high-resolution heatmaps. These heatmaps highlight areas of high attention, indicating the morphological features that contribute to the model’s prediction. Finally, the MIAM-C model was used to predict PFS in patients after receiving PARPi therapy.

WSI: whole slide image; MIAM-C: multi-instance attention model-C; PFS: progression-free survival; PARPi: poly (ADP-ribose) polymerase inhibitors; ROC: receiver operating characteristic; UV Cox: univariable Cox; MV Cox: multivariate Cox.

Futhermore, an external testing set 1 was included, consisting of 413 patients with ovarian, breast, prostate, and pancreatic cancers from The Cancer Genome Atlas (TCGA) database, along with 420 slides. Additionally, an external testing set 2 was created, comprising 130 ovarian cancer patients with 130 slides from the Chongqing Health Center for Women and Children (CQHCWC) between 1 June 2010 and 30 September 2023. In total, the study included 1417 eligible patients with 1695 slides (Supplementary Tables 1–4 and Supplementary Figure 1).

PARPi therapy dataset

Patients meeting the specified criteria were included in the PARPi therapy dataset. The inclusion criteria were as follows: (1) Histologically confirmed primary ovarian cancer; (2) Treatment with PARPi (such as olaparib, niraparib, talazoparib, and pamiparib) either alone or in combination with bevacizumab as maintenance therapy; (3) Long-term follow-up observations after receiving targeted therapy. The exclusion criteria were: (1) Additional treatments such as radiation, surgery, or traditional medicine during PARPi treatment; (2) Occurrence of recurrence or metastasis prior to the initiation of PARPi treatment; (3) Loss of follow-up before disease progression. Disease progression was evaluated according to the Solid Tumor Response Evaluation Criteria (RECIST version 1.1), with a follow-up period up to 30 December 2023. In total, 85 BRCA1/2-mutant ovarian cancer patients were enrolled in this dataset (Supplementary Table 5).

H&E slide processing and sample preparation

First, H&E-stained histopathological slides were digitized into whole slide images (WSIs) using the KFBio KF-PRO-005-HI digital slide scanner at a 40× magnification rate (0.5 microns/pixel). Pathologists then confirmed that all WSIs contained high-quality images of tumors for analysis. Two experienced pathologists used QuPath open-source software (version 0.3.2) to annotate tumor regions of interest (ROIs). Another pathologist reviewed these ROIs to ensure accuracy. To address the issue of oversized digital images, all WSIs were segmented into non-overlapping tiles (512 × 512 pixels) at a resolution of 1 µm/pixel, with a stride of 1.0. Tiles with more than 50% blank area were eliminated. Additionally, the tiles were color-normalized using the Structure-Preserving Color Normalization (SPCN) technique [34]. To improve the accuracy of the predictive model, various data augmentation techniques were employed. These techniques involved random vertical flips (0.5), random horizontal flips (0.5), and adjustments to brightness, contrast, saturation, and hue settings. WSIs were then divided into training and internal testing sets at the patient level in the CUCH cohort, using a random sampling method. The TCGA cohort was used as external testing set 1, and the CQHCWC cohort served as external testing set 2. WSIs from the same patient were always allocated to the same set for consistency and accuracy.

Development of the MIAM model

The architecture of the multi-instance attention model (MIAM) included a feature learning block based on a Convolutional Neural Network (CNN) with a residual connection structure, a salient risk learning block based on a multi-instance self-attention mechanism, and a classification mapping block based on fully connected networks and softmax layers. Using a pre-trained ResNet 34 model as the feature extractor, a WSI generated an M × 1000 feature matrix, where M represents the number of tumor tile instances. Next, we introduced a multi-instance self-attention mechanism. This mechanism treated the M tiles as an instance bag and calculated the gene mutation risk level through the self-attention correlation between the tiles. We then utilized the softmax function to normalize the attention distribution across the M dimension. This process transformed the M × 1000 feature vectors into 1 × M attention scores. The attention scores were further processed by matrix multiplication with the M × 1000 feature maps. Subsequently, they were fed into a linear transformation to map the weighted feature representations to the predictive probability space. From this process, we obtained N binary outputs, corresponding to BRCA1/2 mutations and wild type. N represents the number of total instance bags. We employed cross-entropy error to calculate the error between these N binary outputs and the instance labels. We used the Adam optimizer to train the model, achieving convergence through multiple iterations. Ultimately, we obtained the optimal predictive model, taking the average of each instance’s predictive scores in the same WSI as the final prediction probability for the WSIs. The MIAM model was implemented using Python with the PyTorch and Sklearn libraries (Python version 3.9.7, PyTorch version 1.11.0, Sklearn version 0.24.2).

In the training phase, patients were randomly assigned into non-overlapping training (80%) and testing (20%) sets five times. This process was repeated five times using five-fold cross-validation. During the training process, we used a batch size of 32, an initial learning rate of 0.001, and a total of 20 training epochs. To improve the model’s performance on the validation set, we introduced a dynamic learning rate adjustment strategy. If the loss of the validation set did not decrease within two epochs, we multiplied the learning rate by a factor of 0.1 to downscale.

Development of the MIAM-C model

Not only tissue features associated with BRCA1/2 mutations but also cell features [35] and clinical factors [36,37] have been reported to correlate with BRCA1/2 mutations. We aimed to construct a medical multimodal model (MIAM-C) to achieve a more comprehensive prediction of BRCA1/2 gene mutations by integrating pathological tissue features, cell features, and clinical characteristics. First, we extracted 1000 features from the MIAM model without a classification block. These features were obtained by aggregating features from each slide through statistical mean, median, and variance, resulting in a 3000-dimensional tissue feature vector. Next, we employed the StarDist method within QuPath for cell segmentation and feature extraction, setting the detection probability threshold above 0.5. We adopted a manually annotated tumor cell classifier for iterative training to distinguish tumor cells from other cells. We calculated the characteristics of cells and nuclei in tumor areas, including area, perimeter, roundness, and eosin staining. Features extracted from each slide were then aggregated through mean, median, and variance, resulting in a 147-dimensional cell feature vector. To reduce feature dimensions and identify those closely associated with BRCA1/2 mutations, we applied the Principal Component Analysis (PCA) algorithm. Subsequently, we integrated the tissue features, cell features, and clinical factors (age, stage, grade) using the random forest (RF) algorithm to construct the MIAM-C model. To evaluate the performance advantages, we compared the MIAM-C model with MIAM and assessed their differences in the external testing sets.

Visualization and interpretability

To explore the key features focused on by the MIAM-C model during decision-making, we employed the Shapley Additive exPlanations (SHAP) algorithm [38] to compute feature importance scores for their contribution to the model. To enhance the interpretability of MIAM-C, we combined descriptive and quantitative analyses to explore the morphological features of the most attended regions in a slide. Specifically, we extracted attention scores for all tiles, which were processed through a fully connected layer (FC) and converted to probability distributions via softmax. We used class activation mapping (CAM) to construct heatmaps for the top 10% of tiles, visually presenting the areas focused on by MIAM-C. These attention tiles were then analyzed with a pre-trained HoverNet model for simultaneous cell segmentation and classification [39]. Cells were classified into several categories: tumor cells (red), lymphocytes (green), connective tissue (blue), necrotic cells (yellow), and non-tumor epithelial cells (orange) [40]. For each cell type, we assessed the frequency of occurrence in the highest attention tiles and explored the distribution between the BRCA1/2 mutation and wild type groups.

PFS prediction using MIAM-C

We investigated the application of MIAM-C in predicting PFS for ovarian cancer patients with BRCA1/2 mutations receiving PARPi treatment. A risk score was generated for each patient based on the predictive score obtained from MIAM-C. To determine the optimal cut-off value, we utilized X-tile software (version 3.6.1) to stratify patients into high-risk and low-risk groups. Patients categorized as high-risk showed shorter PFS, indicating a poor prognosis. Conversely, patients in the low-risk group demonstrated longer PFS, suggesting a better prognosis.

Statistical analysis

To evaluate the performance of the MIAM and MIAM-C models, we used multiple metrics, including area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). To assess the significance of morphological features associated with cell types, we used Mann-Whitney U tests to evaluate the statistical differences in cell distributions within high-attention regions between BRCA1/2 mutation and wild type groups. We used Kaplan-Meier survival analysis and the log-rank test to assess significant differences in PFS between high-risk and low-risk groups stratified by MIAM-C. Prognostic factors were evaluated through both univariate and multivariate Cox regression analyses. A p-value of less than 0.05 was considered statistically significant. Statistical analyses were conducted using Python (version 3.8) and R (version 3.4.0) software.

Results

Clinical characteristics

A total of 1,417 eligible participants were enrolled across three dependent cohorts. The CUCH cohort consisted of 874 patients, with 508 having ovarian cancer, 194 with breast cancer, 107 with prostate cancer, and 65 with pancreatic cancer. The TCGA cohort included 413 patients, with 105 having ovarian cancer, 147 with breast cancer, 125 with prostate cancer, and 36 with pancreatic cancer. The CQHCWC cohort included 130 patients with ovarian cancer. Patients from the CUCH cohort were randomly assigned to training and internal testing sets, while patients from the other cohorts served as external testing sets. Detailed clinical information and BRCA1/2 genotype for the training, internal testing, and external testing sets are illustrated in Figure 2 and Supplementary Tables 1–4.

Figure 2. Heatmaps present clinical characteristics and molecular features at the per-patient level in the training, internal testing, and external testing sets.

Performance of MIAM-C

The performance of MIAM and MIAM-C in ovarian, breast, prostate, and pancreatic cancers was evaluated. In the training set, MIAM demonstrated an AUC ranging from 0.786 (0.741–0.831) to 0.873 (0.822–0.925), while in the testing sets, the AUC ranged from 0.686 (0.338–1.000) to 0.804 (0.641–0.967). Compared to MIAM, the AUC of MIAM-C significantly increased in the training set, ranging from 0.854 (0.733–0.975) to 0.990 (0.975–1.000), and in the testing sets, it increased to 0.785 (0.665–0.905) to 0.863 (0.592–1.000) (Figure 3). Additionally, the MIAM-C model showed high sensitivity, specificity, PPV, and NPV in the testing sets (Table 1). These results indicated that MIAM-C outperformed the tissue feature-based DL model MIAM, suggesting that MIAM-C was more accurate and stable in detecting BRCA1/2 mutations.

Figure 3. Performance comparison of MIAM-C and MIAM models in detecting BRCA1/2 gene status. (A-D) The AUC values of MIAM-C and MIAM in the training, internal testing, and external testing sets for ovarian, breast, prostate, and pancreatic cancers, respectively.

MIAM-C: multi-instance attention model-C.

Table 1. The performance of different models in predicting BRCA1/2 mutations in four cancer types.

 	 	AUC	Sensitivity	Specificity	PPV	NPV	
Ovarian cancer	 	 	 	 	 	 	
Internal testing	 	 	 	 	 	 	
 	MIAM-C	0.834 (0.753–0.915)	0.790 (0.701–0.879)	0.800 (0.712–0.887)	0.666 (0.564–0.769)	0.883 (0.813–0.952)	
 	MIAM	0.800 (0.713–0.887)	0.697 (0.597–0.797)	0.670 (0.568–0.772)	0.517 (0.410–0.623)	0.814 (0.729–0.899)	
External testing 1	 	 	 	 	 	 	
 	MIAM-C	0.785 (0.665–0.905)	0.772 (0.650–0.895)	0.726 (0.596–0.855)	0.425 (0.294–0.555)	0.924 (0.845–1.000)	
 	MIAM	0.760 (0.635–0.884)	0.772 (0.650−0.895)	0.666 (0.531−0.801)	0.377 (0.253−0.502)	0.918 (0.836−0.999)	
External testing 2	 	 	 	 	 	 	
 	MIAM-C	0.816 (0.722–0.910)	0.757 (0.654–0.861)	0.773 (0.671–0.874)	0.531 (0.416–0.647)	0.903 (0.831−0.975)	
 	MIAM	0.757 (0.653−0.860)	0.636 (0.522−0.750)	0.670 (0.557−0.782)	0.396 (0.288−0.503)	0.844 (0.755−0.932)	
Breast cancer	 	 	 	 	 	 	
Internal testing	 	 	 	 	 	 	
 	MIAM-C	0.859 (0.716–1.000)	0.666 (0.475–0.858)	0.896 (0.771–1.000)	0.727 (0.545–0.909)	0.866 (0.726–1.000)	
 	MIAM	0.804 (0.641–0.967)	0.750 (0.572–0.927)	0.862 (0.720–1.000)	0.692 (0.504–0.880)	0.892 (0.765–1.000)	
External testing	 	 	 	 	 	 	
 	MIAM-C	0.821 (0.748–0.894)	0.517 (0.421–0.612)	1.000 (1.000–1.000)	1.000 (1.000–1.000)	0.764 (0.683–0.846)	
 	MIAM	0.756 (0.674–0.839)	0.758 (0.676–0.840)	0.846 (0.777–0.914)	0.758 (0.676–0.840)	0.846 (0.777–0.914)	
Prostate cancer	 	 	 	 	 	 	
Internal testing	 	 	 	 	 	 	
 	MIAM-C	0.829 (0.629–1.000)	0.714 (0.475–0.952)	0.904 (0.747–1.000)	0.714 (0.475–0.952)	0.904 (0.747–1.000)	
 	MIAM	0.792 (0.576–1.000)	0.714 (0.475–0.952)	0.714 (0.475–0.952)	0.454 (0.208–0.700)	0.882 (0.709–1.000)	
External testing	 	 	 	 	 	 	
 	MIAM-C	0.819 (0.647–0.990)	0.666 (0.466–0.866)	0.915 (0.788–1.000)	0.375 (0.200–0.549)	0.972 (0.898–1.000)	
 	MIAM	0.742 (0.552–0.933)	0.666 (0.466–0.866)	0.677 (0.478–0.877)	0.136 (0.046–0.225)	0.963 (0.877–1.000)	
Pancreatic cancer	 	 	 	 	 	 	
External testing	 	 	 	 	 	 	
 	MIAM-C	0.863 (0.592–1.000)	0.666 (0.315–1.000)	0.909 (0.679–1.000)	0.400 (0.081–0.718)	0.967 (0.825–1.000)	
 	MIAM	0.686 (0.338–1.000)	0.333 (0.043–0.624)	0.909 (0.679–1.000)	0.250 (0.004–0.495)	0.937 (0.743–1.000)	
AUC: area under the curve; NPV: negative predictive value; PPV: positive predictive value; MIAM-C: multi-instance attention model-C.

BRCA1 and BRCA2 gene mutations significantly increase cancer risk, yet their pathogenicity and distribution of these mutations vary across different cancer types. To better understand this, a subgroup analysis was conducted using the MIAM-C model in groups with BRCA1 and BRCA2 mutation. However, because the number of patients with BRCA1/2 mutations in prostate and pancreatic cancers were limited, our analysis primarily focused on ovarian and breast cancer. In ovarian cancer, the MIAM-C model achieved AUC values of 0.757 (0.651–0.863) in the internal testing set, 0.794 (0.657–0.931) in the external testing set 1, and 0.707 (0.536–0.878) in the external testing set 2 within the BRCA1 mutation group, respectively. In the BRCA2 mutation group, the MIAM-C model achieved AUC values of 0.706 (0.576–0.835) in the internal testing set, 0.673 (0.429–0.916) in the external testing set 1, and 0.813 (0.698–0.929) in the external testing set 2, respectively. In breast cancer, MIAM-C achieved AUC values of 0.859 (0.677–1.000) in the internal testing set and 0.730 (0.622–0.838) in the external testing set within the BRCA1 mutation group, respectively. In the BRCA2 mutation group, the AUC values were 0.669 (0.415–0.922) in the internal testing set and 0.772 (0.666–0.878) in the external testing set 1 (Table 2).

Table 2. Performance of MIAM-C in the BRCA1 and BRCA2 mutation groups.

Cohorts	Groups	AUC	Sen	Spe	PPV	NPV	
Ovarian cancer	 	 	 	 	 	 	
Internal testing	 	 	 	 	 	 	
 	BRCA1 mutation group	0.757 (0.651–0.863)	0.774 (0.670–0.878)	0.721 (0.610–0.832)	0.470 (0.354–0.586)	0.909 (0.837–0.981)	
 	BRCA2 mutation group	0.706 (0.576–0.835)	0.681 (0.549–0.813)	0.660 (0.527–0.793)	0.294 (0.186–0.401)	0.909 (0.824–0.993)	
External testing 1	 	 	 	 	 	 	
 	BRCA1 mutation group	0.794 (0.657–0.931)	0.750 (0.604–0.895)	0.688 (0.536–0.841)	0.300 (0.175–0.424)	0.939 (0.856–1.000)	
 	BRCA2 mutation group	0.673 (0.429–0.916)	0.833 (0.629–1.000)	0.650 (0.404–0.895)	0.125 (0.024−0.225)	0.984 (0.915−1.000)	
External testing 2	 	 	 	 	 	 	
 	BRCA1 mutation group	0.707 (0.536–0.878)	0.666 (0.492–0.841)	0.669 (0.495–0.843)	0.170 (0.076–0.264)	0.951 (0.866–1.000)	
 	BRCA2 mutation group	0.813 (0.698–0.929)	0.809 (0.692–0.926)	0.724 (0.594–0.855)	0.361 (0.241–0.482)	0.951 (0.886–1.000)	
Breast cancer	 	 	 	 	 	 	
Internal testing	 	 	 	 	 	 	
 	BRCA1 mutation group	0.859 (0.677–1.000)	0.714 (0.484–0.944)	0.823 (0.624–1.000)	0.454 (0.222–0.686)	0.933 (0.801–1.000)	
 	BRCA2 mutation group	0.669 (0.415–0.922)	0.500 (0.246–0.753)	0.771 (0.539–1.000)	0.272 (0.078–0.466)	0.900 (0.729–1.000)	
External testing	 	 	 	 	 	 	
 	BRCA1 mutation group	0.730 (0.622–0.838)	0.483 (0.370–0.597)	0.872 (0.790–0.955)	0.500 (0.385–0.614)	0.865 (0.780–0.950)	
 	BRCA2 mutation group	0.772 (0.666–0.878)	0.551 (0.432–0.670)	0.883 (0.800–0.965)	0.533 (0.414–0.652)	0.890 (0.810–0.970)	
AUC: area under the curve; NPV: negative predictive value: PPV, positive predictive value; MIAM-C: multi-instance attention model-C.

Since the incidence of BRCA1/2 mutations varies across different stages of cancer, we examined the performance of the MIAM-C model in different stages. Due to the limited number of patients in various stages of pancreatic cancer, a subgroup analysis of ovarian, breast, and prostate cancers in different stages was carried out, which led to similar results (Supplementary Figure 2), indicating that the MIAM-C model exhibited the capability to predict BRCA1/2 gene status.

Interpretability of the MIAM-C model

To further elucidate our model, SHAP-styled attribution decision plots was utilized to interpret the contribution weights and directions of various features within the MIAM-C model. It was discovered that tissue features significantly contributed to the model in ovarian, breast, prostate, and pancreatic cancers (Supplementary Figure 3). Next, we evaluated which tissue morphological features were critical for the model in detecting BRCA1/2 mutation status. The top 10% of tiles in each slide based on attention scores were selected to display the characteristics of the most important tiles. Using the CAM algorithm, it was observed that BRCA1/2 mutations in the MIAM-C model displayed typical characteristics of high-grade tumors, such as densely cellular, high mitotic rate, and increased nuclear/cytoplasmic (N/C) ratio, and lymphocytic infiltration in four cancer types (Figures 4A and 5A, and Supplementary Figures 4A and 5A).

Figure 4. Visualization of the important regions and cell quantitative analysis of MIAM-C in ovarian cancer. (A) Visualization of the important local tumor areas highly associated with BRCA1/2 mutations. The first row displays the raw high-attention tiles. The second row shows the corresponding heatmap. According to the CAM algorithm, red areas indicate regions with greater contributions from the model. (B) Exemplar high-attention tiles from the BRCA1/2 mutation, wild type groups, and corresponding cell labels. (C) Quantification of cell types in the high-attention tiles for BRCA1/2 mutation and wild type groups. The differences between the groups were calculated through a two-sided wilcoxon rank-sum test (*p < 0.05). In a box plot, the middle line represents the median, the edges of the box correspond to the first and third quartiles, and the whiskers are the minimum and maximum values of the data.

MIAM-C: multi-instance attention model-C; CAM: class activation mapping.

Figure 5. Visualization of the important regions and cell quantitative analysis of MIAM-C in breast cancer. (A) Visualization of the important local tumor areas highly associated with BRCA1/2 mutations. The first row displays the raw high-attention tiles. The second row shows the corresponding heatmap. According to the CAM algorithm, red areas indicate regions with greater contributions from the model. (B) Exemplar high-attention tiles from the BRCA1/2 mutation, wild type groups, and corresponding cell labels. (C) Quantification of cell types in the high-attention tiles for BRCA1/2 mutation and wild type groups. The differences between the groups were calculated through a two-sided wilcoxon rank-sum test (*p < 0.05). In a box plot, the middle line represents the median, the edges of the box correspond to the first and third quartiles, and the whiskers are the minimum and maximum values of the data.

MIAM-C: multi-instance attention model-C; CAM: class activation mapping.

In addition to analyzing the attention heatmap, cell segmentation and classification were also conducted in the high-attention regions of both the BRCA1/2 mutation and wild type groups (Figures 4B and 5B, and Supplementary Figures 4B and 5B). Interestingly, significant differences were found in the lymphocyte cell fraction among the four cancer types, with the BRCA1/2 mutation group showing notably higher levels compared to the wild type group (p < 0.05). Specifically, in ovarian cancer, a significant increase appeared in the stromal fraction within the mutation group compared to the wild type group (p = 0.042) (Figure 4C). Conversely, in breast cancer, the mutation group exhibited a higher fraction of necrotic cells (p = 4.77 × 10−9) (Figure 5C). Prostate cancer differed in that the BRCA1/2 mutations group showed a significant increase in stromal cells (p = 0.049), while the BRCA1/2 wild type group showed a significant increase in epithelial cells (p = 0.049) (Supplementary Figure 4C). In pancreatic cancer, the mutation group showed a higher stromal fraction (p = 0.024) and necrotic cell fraction (p = 0.031) (Supplementary Figure 5C). Taken together, these findings suggested that BRCA1/2 mutations were associated with high-grade tumors and an increased lymphocyte fraction. Despite these similarities across the four cancer types, our results also indicated distinct differences in the microenvironments of each cancer.

Individualized prognosis stratification in ovarian cancer

Next, we focused on individualized prognosis stratification in ovarian cancer. Patients with BRCA1/2 mutations exhibited a wide range of PFS after receiving PARPi treatment, with a median PFS of 44.87 months (95% CI: 37.55–52.18). This variation suggested the need for risk stratification based on targeted therapy prognosis. To achieve this, the MIAM-C model was adopted to generate a risk score for each patient. The risk score was then binarized, with a cutoff of 0.95 determined by X-tile analysis. Patients with risk scores above 0.95 were classified as high risk, while the remaining patients were classified as low risk. The high-risk group had a median PFS of 36.53 months (95% CI: 25.67–47.38), whereas the low-risk group had a median PFS of 48.03 months (95% CI: 35.71–52.18). Survival curves between the two groups showed a statistically significant difference (p = 0.020) (Figure 6A), indicating their differing risk of disease progression after targeted therapy. However, the MIAM model was unable to stratify patients with BRCA1/2 mutations undergoing PARPi therapy (p = 0.067) (Figure 6B).

Figure 6. Prognostic value of the MIAM-C model in ovarian cancer with BRCA1/2 mutations. (A and B) Progression-free survival curves of high-risk and low-risk groups, respectively, defined by the MIAM-C and MIAM model in patients with BRCA1/2 mutant ovarian cancer. (C and D) Univariate Cox regression and multivariate Cox regression analyses of factors associated with progression-free survival, respectively. The risk score was calculated by the MIAM-C model.

MIAM-C: multi-instance attention model-C.

Further analysis was conducted to determine the prognostic significance of the MIAM-C model’s risk score compared to other important factors in the clinic. These factors included age, family history, International Federation of Gynecology and Obstetrics (FIGO) stage, grade, and histological type. We used Cox regression analysis to assess their relationship. Our analysis included both univariate and multivariate Cox regression analysis. In the univariate Cox regression analysis, we found that the risk score had a statistically significant impact on prognosis (hazard ratio: 0.37, 95% CI: 0.15–0.89, p = 0.030) (Figure 6C). In the multivariate Cox regression analysis, we confirmed that the risk score was an independent predictor for PFS (hazard ratio: 0.40, 95% CI: 0.16–0.99, p = 0.047) (Figure 6D). As a result, the MIAM-C model was proved to be effective in categorizing patients with BRCA1/2 mutations based on their prognosis after PARPi treatment. This could be valuable in guiding decisions regarding targeted therapy and served as a useful addition to gene sequencing.

Discussion

Accurate determination of BRCA1/2 gene status and individualized response to PARPi therapy is critical for the targeted therapy of ovarian, breast, prostate, and pancreatic cancers. We developed an innovative multimodal model integrating tissue, cell, and clinical features. This model directly identifies BRCA1/2 gene status and effectively stratifies BRCA-mutant ovarian cancer patients receiving PARPi treatment. Comparing the MIAM-C model with the unimodal DL model MIAM, the results indicate that MIAM-C demonstrates superior performance. Visualization and quantitative analysis enhance model interpretability for the medical field.

DL algorithms can infer molecular features from histological images, as they leave footprints in histomorphology [22,41]. An increasing number of studies have utilized AI techniques to detect BRCA1/2 gene status from pathological images. For example, Wang et al. employed a residual neural network (ResNet 18) to predict BRCA1/2 status in breast cancer using H&E images from 62 patients across two centers. The model achieved AUC values ranging from 0.635 to 0.828 at various magnification levels [30]. In another study, Camilla et al. developed clustering-constrained-attention multiple-instance learning (CLAM) to detect BRCA1/2 mutations in ovarian cancer using H&E pathological images from 664 patients. The model achieved an AUC of 0.70 in the training set and 0.55 in the internal testing set [31]. Previous studies mainly focused on tissue features when constructing DL models and overlooked cell features and clinical characteristics. However, studies suggest that BRCA1/2 gene mutations are associated not only with tissue features but also with cell features and clinical characteristics [35,36]. We developed the MIAM-C model, which outperforms the MIAM model based solely on tissue features. The MIAM-C model extracts tissue features while incorporating cell features and clinical characteristics. Feature contribution analysis has revealed the pivotal role of tissue features in the model, while the integration of cell features and clinical factors significantly improves the model’s predictive performance, thereby enhancing accuracy and reliability.

Gene sequencing is widely acknowledged as the current standard for detecting genotypes. However, its application in oncology workflows faces obstacles such as time-consuming procedures, technical complexity, and high costs. Furthermore, gene sequencing does not effectively identify patients who may have a poor prognosis in response to targeted therapy. We have developed the MIAM-C model, which serves as a valuable complement to gene sequencing. The MIAM-C model not only predicts the BRCA1/2 genotype but also provides personalized PFS predictions for patients undergoing PARPi therapy. This approach enables the stratification of patients with BRCA1/2 mutations based on their response to treatment. In related research, Wang et al. proposed a fully automated artificial intelligence system (FAIS) that can detect EGFR genotype and identify patients with an EGFR mutation at high risk for tyrosine kinase inhibitor resistance [42]. Additionally, Tian et al. presented a DL framework that assesses PD-L1 expression in non-small cell lung cancer and predicts clinical outcomes in response to immunotherapy [43]. These studies highlight the valuable insights that AI approaches can provide into molecular status and the prediction of outcomes for specific targeted therapies.

DL models have shown impressive performance in the medical field but face a significant challenge: they are ‘black-box’ models. The lack of interpretability, trust, and transparency makes it difficult to apply DL models in clinical practice [44,45]. One alternative approach to model interpretability is visualizing highly ranked image tiles, which can help identify novel morphological features [46,47]. Using this approach, we discovered that the model primarily identifies features such as high-grade tumors and lymphocytic infiltration, which are closely associated with BRCA1/2 mutations. Previous studies have also revealed morphological characteristics associated with BRCA1/2 mutations, including poorly differentiated tumors, lymphocytic infiltration, higher mitotic rates, and cellular pleomorphism [35,48]. The morphological features identified by our model align with those reported in previous studies. Notably, we further analyzed tumor microenvironments in the BRCA1/2 mutation group and the BRCA1/2 wild type group, specifically evaluating the differences in cell type fraction between the two groups. Our findings showed that the lymphocyte fraction in the BRCA1/2 mutation group was higher than in the BRCA1/2 wild type group for ovarian, breast, prostate, and pancreatic cancers. However, the fraction of stromal and necrotic cells exhibited variability between the two groups in these four cancer types. This discrepancy could be due to various factors, such as the tumor size, treatment modalities, and tumor grade or stage, all of which can impact the specific tumor microenvironment. In high-grade serous ovarian cancer, BRCA1/2-mutated tumors showed significantly increased levels of CD3+ and CD8+ tumor-infiltrating lymphocytes compared to homologous recombination (HR)-proficient tumors [49]. Additionally, the stromal fraction in the BRCA1/2 mutant group was significantly higher than in the HR wild type group in breast cancer [50]. By analyzing histomorphological features and the tumor microenvironment, we extensively explored the medical interpretability of the model.

Our study has several limitations. First, we only considered BRCA1/2 mutations and did not account for other genes in the HRD pathway. Therefore, we cannot rule out mutations in other genes that may be associated with specific histomorphological features. A methodology that enables the simultaneous detection of multiple genes would be valuable. Second, we did not distinguish between germline and somatic variations within the BRCA1/2 mutations. Third, although our model was validated in four cancer types, the dataset remains limited, especially with a small number of cases for prostate and pancreatic cancers. This may affect the robustness of the model’s predictions for these specific cancer types. In the future, larger samples from multiple centers will be necessary to validate the accuracy and robustness of the model.

Conclusions

In conclusion, our study has developed a multimodal model for detecting BRCA1/2 gene status in various types of cancer. We have also elucidated how the model identifies crucial morphological features that are linked to BRCA1/2 gene status. Additionally, the model demonstrates the ability to predict the response of patients with BRCA1/2 mutations to targeted therapy.

Supplementary Material

Supplemental Material

Ethical approval

This study was approved by the Chongqing University Cancer Hospital of Medicine Institutional Review Board (Ethical Application Ref: CZLS2023213-A).

Author contributions

Bo Xu and Bo Lin conceived and designed the study. Yi Li, Xiaomin Xiong, and Mengke Xu collected and analyzed the data. Xiaoju Li, Boping Yang, and Yu Li contributed to data interpretation. Bo Lin and Xiaohua Liu provided technical support. Yi Li, Bo Lin, and Bo Xu drafted and revised the manuscript. All authors have reviewed and approved the final version of the manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data used to support the findings of this study are available from the corresponding author upon request. However, the data cannot be shared publicly due to the privacy of individuals who participated in the study. The related code and pre-trained models for the study are publicly available on GitHub (https://github.com/LIYI0720/MIAM-C).
==== Refs
References

1 Wooster R, Bignell G, Lancaster J, et al. Identification of the breast cancer susceptibility gene BRCA2. Nature. 1995;378 (6559 ):789–792. doi: 10.1038/378789a0.8524414
2 Miki Y, Swensen J, Shattuck-Eidens D, et al. A strong candidate for the breast and ovarian cancer susceptibility gene BRCA1. Science. 1994;266 (5182 ):66–71. doi: 10.1126/science.7545954.7545954
3 Kuchenbaecker KB, Hopper JL, Barnes DR, et al. Risks of Breast, Ovarian, and contralateral breast cancer for BRCA1 and BRCA2 mutation carriers. JAMA. 2017;317 (23 ):2402–2416. doi: 10.1001/jama.2017.7112.28632866
4 Roy R, Chun J, Powell SN. BRCA1 and BRCA2: different roles in a common pathway of genome protection. Nat Rev Cancer. 2011;12 (1 ):68–78. doi: 10.1038/nrc3181.22193408
5 Milne RL, Osorio A, Cajal TRY, et al. The average cumulative risks of breast and ovarian cancer for carriers of mutations in BRCA1 and BRCA2 attending genetic counseling units in Spain. Clin Cancer Res. 2008;14 (9 ):2861–2869. doi: 10.1158/1078-0432.CCR-07-4436.18451254
6 Oh M, Alkhushaym N, Fallatah S, et al. The association of BRCA1 and BRCA2 mutations with prostate cancer risk, frequency, and mortality: a meta-analysis. Prostate. 2019;79 (8 ):880–895. doi: 10.1002/pros.23795.30900310
7 Hu C, Hart SN, Polley EC, et al. Association between inherited germline mutations in cancer predisposition genes and risk of pancreatic cancer. JAMA. 2018;319 (23 ):2401–2409. doi: 10.1001/jama.2018.6228.29922827
8 Momozawa Y, Sasai R, Usui Y, et al. Expansion of cancer risk profile for BRCA1 and BRCA2 Pathogenic Variants. JAMA Oncol. 2022;8 (6 ):871–878. doi: 10.1001/jamaoncol.2022.0476.35420638
9 Tutt ANJ, Garber JE, Kaufman B, et al. Adjuvant Olaparib for Patients with BRCA1- or BRCA2-Mutated Breast Cancer. N Engl J Med. 2021;384 (25 ):2394–2405. doi: 10.1056/NEJMoa2105215.34081848
10 Abida W, Patnaik A, Campbell D, et al. Rucaparib in men with metastatic castration-resistant prostate cancer harboring a BRCA1 or BRCA2 gene alteration. J Clin Oncol. 2020;38 (32 ):3763–3772. doi: 10.1200/JCO.20.01035.32795228
11 Golan T, Hammel P, Reni M, et al. Maintenance Olaparib for Germline BRCA-mutated metastatic pancreatic cancer. N Engl J Med. 2019;381 (4 ):317–327. doi: 10.1056/NEJMoa1903387.31157963
12 Liu JF, Brady MF, Matulonis UA, et al. Olaparib with or without Cediranib Versus platinum-based chemotherapy in recurrent platinum-sensitive Ovarian Cancer (NRG-GY004): a randomized, open-label, phase III trial. J Clin Oncol. 2022;40 (19 ):2138–2147. doi: 10.1200/JCO.21.02011.35290101
13 Robson M, Goessl C, Domchek S. Olaparib for Metastatic Germline BRCA-mutated breast cancer. N Engl J Med. 2017;377 (18 ):1792–1793. doi: 10.1056/NEJMc1711644.
14 Matsubara N, de Bono J, Olmos D, et al. Olaparib efficacy in patients with metastatic castration-resistant prostate cancer and BRCA1, BRCA2, or ATM alterations identified by testing circulating tumor DNA. Clin Cancer Res. 2023;29 (1 ):92–99. doi: 10.1158/1078-0432.CCR-21-3577.36318705
15 Kindler HL, Hammel P, Reni M, et al. Overall survival results from the POLO Trial: a phase III study of active maintenance Olaparib Versus Placebo for Germline BRCA-mutated metastatic pancreatic cancer. J Clin Oncol. 2022;40 (34 ):3929–3939. doi: 10.1200/JCO.21.01604.35834777
16 Vergote I, González-Martín A, Ray-Coquard I, et al. European experts’ consensus group, European experts consensus: BRCA/homologous recombination deficiency testing in first-line ovarian cancer. Ann Oncol. 2022;33 (3 ):276–287. doi: 10.1016/j.annonc.2021.11.013.34861371
17 Daly MB, Pilarski R, Yurgelun MB, et al. NCCN guidelines insights: genetic/familial high-risk assessment: breast, ovarian, and pancreatic, Version 1.2020. J Natl Compr Canc Netw. 2020;18 (4 ):380–391. doi: 10.6004/jnccn.2020.0017.32259785
18 Daly MB, Pal T, Berry MP, et al. Genetic/familial high-risk assessment: breast, Ovarian, and pancreatic, Version 2.2021, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw. 2021;19 (1 ):77–102. doi: 10.6004/jnccn.2021.0001.33406487
19 Kamaraju S, Conroy M, Harris A, et al. Challenges to genetic testing for germline mutations associated with breast cancer among African Americans Authors. Cancer Treat Rev. 2024;124 :102695. doi: 10.1016/j.ctrv.2024.102695.38325071
20 Park HS, Park S-J, Kim JY, et al. Next-generation sequencing of BRCA1/2 in breast cancer patients: potential effects on clinical decision-making using rapid, high-accuracy genetic results. Ann Surg Treat Res. 2017;92 (5 ):331–339. doi: 10.4174/astr.2017.92.5.331.28480178
21 Banerjee S, Gonzalez-Martin A, Harter P, et al. First-line PARP inhibitors in ovarian cancer: summary of an ESMO Open – Cancer Horizons round-table discussion. ESMO Open. 2020;5 (6 ):e001110. doi: 10.1136/esmoopen-2020-001110.33310779
22 Kather JN, Heij LR, Grabsch HI, et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat Cancer. 2020;1 (8 ):789–799. doi: 10.1038/s43018-020-0087-6.33763651
23 Veta M, van Diest PJ, Willems SM, et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal. 2015;20 (1 ):237–248. doi: 10.1016/j.media.2014.11.010.25547073
24 Yu K-H, Zhang C, Berry GJ, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. 2016;7 (1 ):12474. doi: 10.1038/ncomms12474.27527408
25 Diao JA, Wang JK, Chui WF, et al. Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes. Nat Commun. 2021;12 (1 ):1613. doi: 10.1038/s41467-021-21896-9.33712588
26 Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018;24 (10 ):1559–1567. doi: 10.1038/s41591-018-0177-5.30224757
27 Noorbakhsh J, Farahmand S, Foroughi Pour A, et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat Commun. 2020;11 (1 ):6367. doi: 10.1038/s41467-020-20030-5.33311458
28 Yamashita R, Long J, Longacre T, et al. Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol. 2021;22 (1 ):132–141. doi: 10.1016/S1470-2045(20)30535-0.33387492
29 Zhao Y, Wang W, Ji Y, et al. Computational pathology for prediction of isocitrate dehydrogenase gene mutation from whole slide images in adult patients with diffuse glioma. Am J Pathol. 2024;194 (5 ):747–758. doi: 10.1016/j.ajpath.2024.01.009.38325551
30 Wang X, Zou C, Zhang Y, et al. Prediction of BRCA gene mutation in breast cancer based on deep learning and histopathology images. Front Genet. 2021;12 :661109. doi: 10.3389/fgene.2021.661109.34354733
31 Nero C, Boldrini L, Lenkowicz J, et al. Deep-learning to predict BRCA mutation and survival from digital H&E slides of epithelial Ovarian Cancer. Int J Mol Sci. 2022;23 (19 ):11326. doi: 10.3390/ijms231911326.36232628
32 Spurdle AB, Healey S, Devereau A, et al. ENIGMA–evidence-based network for the interpretation of germline mutant alleles: an international initiative to evaluate risk and clinical significance associated with sequence variation in BRCA1 and BRCA2 genes. Hum Mutat. 2012;33 (1 ):2–7. doi: 10.1002/humu.21628.21990146
33 Tricarico R, Kasela M, Mareni C, et al. Assessment of the InSiGHT interpretation criteria for the clinical classification of 24 MLH1 and MSH2 Gene Variants. Hum Mutat. 2017;38 (1 ):64–77. doi: 10.1002/humu.23117.27629256
34 Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans Med Imag. 2016;35 (8 ):1962–1971. doi: 10.1109/TMI.2016.2529665.
35 Kim E-K, Park SY, Kim S-W. Clinicopathological characteristics of BRCA-associated breast cancer in Asian patients. J Pathol Transl Med. 2020;54 (4 ):265–275. doi: 10.4132/jptm.2020.04.07.32397691
36 Huang G, Lu H, Chen Q, et al. Prevalence and factors associated with BRCA1/2 gene mutation in chinese populations with breast cancer. Int J Gen Med. 2022;15 :6783–6789. doi: 10.2147/IJGM.S378706.36042935
37 Innella G, Godino L, Erini G, et al. Factors predicting BRCA1/2 pathogenic variants in patients with ovarian cancer: a systematic review with meta-analysis. J Clin Pathol. 2023;76 (8 ):510–517. doi: 10.1136/jcp-2022-208753.37221047
38 Lundberg SM, Erion G, Chen H, et al. From local explanations to global understanding with explainable ai for trees. Nat Mach Intell. 2020;2 (1 ):56–67. doi: 10.1038/s42256-019-0138-9.32607472
39 Graham S, Vu QD, Raza SEA, et al. Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal. 2019;58 :101563. doi: 10.1016/j.media.2019.101563.31561183
40 Chen RJ, Lu MY, Williamson DFK, et al. Pan-cancer integrative histology-genomic analysis via multimodal deep learning. Cancer Cell. 2022;40 (8 ):865–878.e6. doi: 10.1016/j.ccell.2022.07.004.35944502
41 Fu Y, Jung AW, Torne RV, et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat Cancer. 2020;1 (8 ):800–810. doi: 10.1038/s43018-020-0085-8.35122049
42 Wang S, Yu H, Gan Y, et al. Mining whole-lung information by artificial intelligence for predicting EGFR genotype and targeted therapy response in lung cancer: a multicohort study. Lancet Digit Health. 2022;4 (5 ):e309–e319. doi: 10.1016/S2589-7500(22)00024-3.35341713
43 Tian P, He B, Mu W, et al. Assessing PD-L1 expression in non-small cell lung cancer and predicting responses to immune checkpoint inhibitors using deep learning on computed tomography images. Theranostics. 2021;11 (5 ):2098–2107. doi: 10.7150/thno.48027.33500713
44 Salahuddin Z, Woodruff HC, Chatterjee A, et al. Transparency of deep neural networks for medical image analysis: a review of interpretability methods. Comput Biol Med. 2022;140 :105111. doi: 10.1016/j.compbiomed.2021.105111.34891095
45 Chen H, Gomez C, Huang C-M, et al. Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review. NPJ Digit Med. 2022;5 (1 ):156. doi: 10.1038/s41746-022-00699-2.36261476
46 Naik N, Madani A, Esteva A, et al. Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains. Nat Commun. 2020;11 (1 ):5727. doi: 10.1038/s41467-020-19334-3.33199723
47 Qu H, Zhou M, Yan Z, et al. Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis Oncol. 2021;5 (1 ):87. doi: 10.1038/s41698-021-00225-9.34556802
48 Sønderstrup IMH, Jensen MB, Ejlertsen B, et al. Evaluation of tumor-infiltrating lymphocytes and association with prognosis in BRCA-mutated breast cancer. Acta Oncol. 2019;58 (3 ):363–370. doi: 10.1080/0284186X.2018.1539239.30614364
49 Strickland KC, Howitt BE, Shukla SA, et al. Association and prognostic significance of BRCA1/2-mutation status with neoantigen load, number of tumor-infiltrating lymphocytes and expression of PD-1/PD-L1 in high grade serous ovarian cancer. Oncotarget. 2016;7 (12 ):13587–13598. doi: 10.18632/oncotarget.7277.26871470
50 Launonen I-M, Lyytikäinen N, Casado J, et al. Single-cell tumor-immune microenvironment of BRCA1/2 mutated high-grade serous ovarian cancer. Nat Commun. 2022;13 (1 ):835. doi: 10.1038/s41467-022-28389-3.35149709
