==== Front J Gynecol OncolJ Gynecol OncolJGOJournal of Gynecologic Oncology2005-03802005-0399Asian Society of Gynecologic Oncology; Korean Society of Gynecologic Oncology 3002263010.3802/jgo.2018.29.e66Original ArticleOvaryArtificial intelligence weights the importance of factors predicting complete cytoreduction at secondary cytoreductive surgery for recurrent ovarian cancer https://orcid.org/0000-0001-8373-8569Bogani Giorgio https://orcid.org/0000-0002-0565-5829Rossetti Diego https://orcid.org/0000-0002-5684-8225Ditto Antonino https://orcid.org/0000-0002-4863-1747Martinelli Fabio https://orcid.org/0000-0002-1811-209XChiappa Valentina https://orcid.org/0000-0003-2622-4195Mosca Lavinia https://orcid.org/0000-0002-3744-2668Leone Roberti Maggiore Umberto https://orcid.org/0000-0001-7166-6238Ferla Stefano https://orcid.org/0000-0003-0981-0598Lorusso Domenica https://orcid.org/0000-0002-4740-7794Raspagliesi Francesco Department of Gynecologic Oncology, IRCCS National Cancer Institute, Milan, Italy.Correspondence to Giorgio Bogani. Department of Gynecologic Oncology, IRCCS National Cancer Institute, Via Venezian 1, 20133 Milan, Italy. giorgiobogani@yahoo.it; giorgio.bogani@istitutotumori.mi.it9 2018 23 4 2018 29 5 e6606 3 2018 29 3 2018 07 4 2018 Copyright © 2018. Asian Society of Gynecologic Oncology, Korean Society of Gynecologic Oncology2018Asian Society of Gynecologic Oncology, Korean Society of Gynecologic OncologyThis is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.Objective Accumulating evidence support that complete cytoreduction (CC) at the time of secondary cytoreductive surgery (SCS) improves survival in patients affected by recurrent ovarian cancer (ROC). Here, we aimed to determine whether artificial intelligence (AI) might be useful in weighting the importance of clinical variables predicting CC and survival. Methods This is a retrospective study evaluating 194 patients having SCS for ROC. Using artificial neuronal network (ANN) analysis was estimated the importance of different variables, used in predicting CC and survival. ANN simulates a biological neuronal system. Like neurons, ANN acquires knowledge through a learning-phase process and allows weighting the importance of covariates, thus establishing how much a variable influences a multifactor phenomenon. Results Overall, 82.9% of patients had CC at the time of SCS. Using ANN, we observed that the 3 main factors driving the ability of achieve CC included: disease-free interval (DFI) (importance: 0.231), retroperitoneal recurrence (importance: 0.178), residual disease at primary surgical treatment (importance: 0.138), and International Federation of Gynecology and Obstetrics (FIGO) stage at presentation (importance: 0.088). Looking at connections between different covariates and overall survival (OS), we observed that DFI is the most important variable influencing OS (importance: 0.306). Other important variables included: CC (importance: 0.217), and FIGO stage at presentation (importance: 0.100). Conclusion According to our results, DFI should be considered as the most important factor predicting both CC and OS. Further studies are needed to estimate the clinical utility of AI in providing help in decision making process. Ovarian NeoplasmsCytoreductive SurgeryArtificial Intelligence ==== Body INTRODUCTION Ovarian cancer represents the most lethal gynecological cancer in developed countries [1]. Although growing surgical and medical attempts are improving the efficacy of first line treatments, the majority of ovarian cancer patients developed recurrent disease. The gold standard treatment for recurrent ovarian cancer (ROC) is chemotherapy (CT) [2]. However, accumulating data suggested that in selected patients secondary cytoreductive surgery (SCS) might be a valuable treatment modality. Although mature evidence is lacking, several investigations highlighted that SCS might improve outcome of patients suffering from platinum-sensitive recurrence [34567]. Moreover, the preliminary results of the ongoing Arbeitsgemeinschaft Gynaëkologische Onkologie Descriptive Evaluation of preoperative Selection of (K)Criteria for Operability in recurrent Ovarian cancer (AGO OVAR DESKTOP) III/ENGOTov20 study (comparing SCS plus CT with CT alone in patients with ROC) suggested that SCS improves platinum-free interval and disease-free survival (DFS) in comparison with CT alone [5]. Patient selection is paramount in order to reduce ineffective surgical attempts that might delay effective treatments such as CT. Several models were built in order to identify patients who deserve SCS [67]. However, from the other point of view, these models could be strict and a beneficial treatment could be denied in patients who do not met inclusion criteria. Here, we aimed to evaluate if the application of artificial intelligence (AI) might be useful in weight the importance of individual patients' and diseases' variables, thus identifying ROC patients who deserve SCS [8]. In the present paper, using the results achieved by AI we aimed to assess importance of different variables in predicting complete cytoreduction (CC) at SCS. As secondary outcomes we aimed to evaluate whether the application of AI might replace the results achieved by conventional statistical analysis. MATERIALS AND METHODS This is a retrospective evaluation of patients undergoing SCS due to platinum-sensitive ROC. The National Cancer Institute (NCI) Institutional Review Board approved this study (approval number: INT/MI/006812). The paper included data from a series of consecutive patients with ROC undergoing SCS at NCI of Milan, from January 1, 2001 through December 31, 2015. Patients who did not provide consent to use their medical information for research purposes were excluded from the present analysis. While, all the included patients signed written consent for research purpose. Data were retrospectively retrieved from a computerized surgical database, containing data on every surgical procedure performed for the patients enrolled into the study. The computerized database was maintained prospectively and it is of research quality and has been updated by trained residents and nurses. Data included in the present study were already presented into a recently published paper evaluating predictors of CC in patients undergoing SCS for ROC [9] Patients' records were screened in order to identify baseline patients' and diseases' characteristics. Inclusion criteria were: 1) age ≥18 years; 2) histologically-proven recurrence of invasive epithelial ovarian, primary peritoneal and Fallopian tube cancer (i.e., ROC); and 3) execution of SCS. Exclusion criteria were: 1) consent withdrawn; 2) absence of gross disease at the time of SCS; and 3) performance status (PS) not allowing surgical treatment. Central pathology review was performed in all cases, before having SCS. Details about our surgical protocol are reported elsewhere [9]. Generally, SCS was offered to patients with recurrent disease with a disease-free interval (DFI) longer than 6 months. DFI was the time between the end of platinum-based adjuvant CT and the diagnosis of recurrence. Although no specific guidelines are available for the section to SCS, the surgical attempts were offered after the evaluation of the parameters included in both the AGO OVAR and Memorial Sloan Kettering (MSK) criteria: 1) no residual disease (RD) at first surgery; 2) good PS; 3) absence of ascites; 4) site of recurrence; and 5) DFI [3456]. According to the AGO OVAR criteria, ascites was recorded when free fluid was 500 mL or more. The diagnosis of carcinomatosis included the presence of diffuse peritoneal spread of the disease, into multiple anatomical sites. Since the primary goal of SCS is achieve CC, the primary outcome measure of our research is to identify factors associated with CC. Additionally, as secondary endpoints, we aimed to weight the importance of those factors in influencing survival outcomes. The innovation of the present paper is based on the use of AI (artificial neuronal network [ANN] analysis). ANN was used in order to weight the importance of associated variables, thus predicting the effect of each variable on achieving CC and survival outcomes. This analysis is a system of interconnections based on simple mathematical model associated with learning algorithms. ANN comprised a 4-layer (1 input layer, 2 hidden layers, and 1 output layer) feed-forward analysis. To develop the ANN, cases were randomly assigned to the training group (80%) or to the testing group (20%) through a generator of random numbers. The back propagation of the error was applied as a learning rule using the online training method. Synaptic weights were updated after each training data record. Detailed description of ANN is reported elsewhere [810]. Basic descriptive statistics were used. Kaplan-Meier and Cox proportional hazard models were used appropriately in order to evaluate the impact of the predicting variables on survival outcomes. Hazard ratio (HR) and 95% confidence intervals (CIs) were calculated for each comparison. Univariate and multivariate analysis were performed when appropriate. All covariates with a p-value less than 0.10, based on univariate analysis were included in the multivariate model. Statistical analyses were performed using the GraphPad Prism version 6.0 for Mac (GraphPad Software, San Diego, CA, USA) and IBM-Microsoft SPSS version 22.0 for Mac (IBM Corp., Armonk, NY, USA). All p-values were 2-sided. The p-values <0.05 were considered statistically significant. RESULTS Overall, 199 patients, had SCS due to the diagnosis of ROC. Five patients were excluded since they did not provide consent to use their medical information for research purposes, thus leaving 194 (97.4%) patients available for the analysis. Fig. 1 shows the flow of patients into the study design. Baseline patients' characteristics are reported in Table 1. In our population, 161 (82.9%) patients had CC at the time of SCS. Using ANN, we observed that the 3 main factors driving the ability of achieve CC included: DFI (importance: 0.231), retroperitoneal recurrence only (importance: 0.178), RD at primary surgical treatment (importance: 0.138) and International Federation of Gynecology and Obstetrics (FIGO) stage at presentation (importance: 0.088). Fig. 2 displays the results of ANN and the importance of various predicting factors. Fig. 1 Study design. ROC, recurrent ovarian cancer; SCS, secondary cytoreductive surgery. Table 1 Baseline patients' characteristics (n=194) Characteristics Values Age (yr) 61.6 (±12.5) BMI (kg/m2) 24.7 (±4.2) ECOG PS 0 151 (77.8) 1 43 (22.2) CA-125 at first diagnosis (UI/L) 328.5 (±682.2) Type of ovarian cancer High grade serous 140 (72.5) Endometrioid 19 (9.8) Clear cells 9 (4.6) Undifferentiated 20 (10.3) Other 6 (3.0) FIGO stage Early-stage 14 (7.2) Advanced-stage 160 (82.5) Unknown 20 (10.3) Cytoreductuction at primary treatment No macroscopic tumor at the end of surgery 146 (75.2) Macroscopic tumor at the end of surgery 42 (21.6) Unknown 6 (3.1) Site of recurrent disease Retroperitoneal 39 (20.1) Single peritoneal 62 (31.9) Multiple peritoneal 77 (39.7) Carcinomatosis 16 (8.3) Ascites Yes 23 (11.9) No 124 (63.9) Unknown 47 (24.2) DFI (mo) 43.9 (±38.5) Follow-up (mo) 32.2 (±20.9) Data are shown as mean (SD) or number (%). BMI, body mass index; DFI, disease-free interval; ECOG, Eastern Cooperative Oncology Group; FIGO, International Federation of Gynecology and Obstetrics; PS, performance status; SD, standard deviation. Fig. 2 Importance of factors predicting for CC. CC, complete cytoreduction; DFI, disease-free interval; ECOG, Eastern Cooperative Oncology Group; RD, residual disease. Looking at connections between covariates and overall survival (OS), we observed that DFI is the most important variable influencing OS (importance: 0.306). Other important variables included: CC (importance: 0.217), and FIGO stage at presentation (importance: 0.100). Fig. 3 shows the results of ANN related to the connections between predictive variables and OS. Patients having CC experienced a median OS of 40.5 vs. 23.0 months for patients having incomplete resection, respectively (p<0.001, log-rank test). Although it did not reach statistical significance, patients with early stage disease at presentation (FIGO stage I–II) experienced better survival than patients with advanced stage (FIGO stage III–IV) of disease: median OS 59.7 vs. 33.0 months, respectively (p=0.120, log-rank test). Fig. 3 Importance of factors predicting for survival. CC, complete cytoreduction; DFI, disease-free interval; ECOG, Eastern Cooperative Oncology Group; RD, residual disease. DISCUSSION The present study investigated how the adoption of AI might be useful to identify characteristics that might choose to undergo SCS due to ROC. We observed that among our population DFI is the main factor predicting for CC at SCS. Another important factor for the outcome of SCS is site of recurrence. In particular, our data underlined that the presence of retroperitoneal disease alone is associated with an increased ability to achieve CC in comparison to the presence of peritoneal disease. Interestingly, the presence of single peritoneal, multiple peritoneal nodules and carcinomatosis had a limited impact on the ability to achieve CC, thus highlighting that the presence of carcinomatosis should not be considered a contraindication for SCS. CC is the main variable influencing patients' survival, followed by DFI, and FIGO stage at presentation. Site of recurrence (peritoneal vs. retroperitoneal) had no impact on survival outcomes. Accumulating data support that in patients with ROC, the execution of SCS is related to important improvement in survival outcomes when CC is achieved. Therefore, patients' selection is paramount in identifying patients who deserve surgical attempts. As aforementioned, few investigators tried to build model useful in patients' selection [1112131415]. The AGO OVAR DESKTOP I and II studies evaluated the importance of achieving CC at the time of SCS and proposed positive predictors in order to enrich a population in whom SCS has a high probability for CC [345]. These criteria included: 1) absence of RD at first surgery; 2) good PS; and 3) absence of ascites (<500 mL). Similarly, the “MSK criteria” represent a useful tool in predicting CC [67]. These criteria are based on the site of recurrence (i.e., single, multiple, and carcinomatosis) and DFI [67]. However, the available model might be too strict thus prohibiting a beneficial treatment in patients who do not meet these criteria. Recently, our study group published a combined score using the variables included in both the 2 most reliable models (i.e., the AGO OVAR and MSK criteria) [9]. Adopting our nomogram, we were able to predict the probability to achieve CC based on the AGO OVAR (RD at primary surgery, PS, and presence of ascites) and MSK (DFI and site of recurrence) criteria [9]. Other investigators included in their models the evaluation of serum markers levels (i.e., CA-125 and HE4) and other characteristics of the disease at presentation such as FIGO stage and histology [1112]. Although level A evidence supporting the role of SCS in ROC is not still available, preliminary data of the OVAR DESKTOP III/ENGOTov20 study suggested that SCS is a valuable option for ROC patients, since it improves DFS. Other trials are ongoing, including the SOC 1 trial [16]. While, the SOCceR trial was spotted prematurely in August 2015 due to the lagging of inclusion [17]. In the present study, adopting ANN we observed that DFI is the main factor impacting the ability to obtain CC. In contrast with other group, we observed that site of recurrence is an important factor in predicting CC. In particular, both the AGO OVAR and MSK criteria did not focused on the presence of retroperitoneal localization. In our experience, we observed a great difference between retroperitoneal and peritoneal recurrences. Moreover, we also observed that also FIGO stage at presentation is an important factor that might have to be taken into account when we manage patients with ROC. The presence of ascites and carcinomatosis had per se a limited impact on the ability to achieve CC. The adoption of ANN represents the main novelty of the present investigation. The utilization of AI provides more insight in identifying variables for predicting CC and OS. Although our study confirmed data observed in other studies [67], ANN weighted how a variable impact on outcomes. However, the inherent biases of the single-center retrospective study design represent the main weaknesses of our study. Moreover, our population included a selected group of ROC patients and it is not fully projectable on the whole group of ovarian cancer patients presenting with recurrent disease. SCS represents a valuable therapeutic option for a selected group of ovarian cancer presenting with platinum-sensitive recurrence [131415]. Patients selection is the key point for SCS. In the present paper, we observed that applying AI helped to weight the importance of various predicting variables. Through AI we estimated that DFI and the presence of retroperitoneal recurrence are the main factors enabling to predict CC and OS. Additionally, CC is strictly related to oncologic outcomes. Further innovative methods are needed to provide an improvement in patients' selection. Conflict of Interest: No potential conflict of interest relevant to this article was reported. Author Contributions: Conceptualization: B.G. Data curation: R.D., D.A., M.F., C.V., M.L., L.R.M.U., F.S., L.D., R.F. Formal analysis: B.G. Methodology: R.F. Writing - original draft: B.G. Writing - review & editing: R.D., D.A., M.F., C.V., M.L., L.R.M.U., F.S., L.D., R.F. ==== Refs 1 Kumar A Torres ML Cliby WA Kalli KR Bogani G Aletti G Inflammatory and nutritional serum markers as predictors of peri-operative morbidity and survival in ovarian cancer Anticancer Res 2017 37 3673 3677 28668859 2 Colombo N Lorusso D Scollo P Impact of recurrence of ovarian cancer on quality of life and outlook for the future Int J Gynecol Cancer 2017 27 1134 1140 28640766 3 Harter P du Bois A Hahmann M Hasenburg A Burges A Loibl S Surgery in recurrent ovarian cancer: the Arbeitsgemeinschaft Gynaekologische Onkologie (AGO) DESKTOP OVAR trial Ann Surg Oncol 2006 13 1702 1710 17009163 4 Harter P Sehouli J Reuss A Hasenburg A Scambia G Cibula D Prospective validation study of a predictive score for operability of recurrent ovarian cancer: the Multicenter Intergroup Study DESKTOP II. A project of the AGO Kommission OVAR, AGO Study Group, NOGGO, AGO-Austria, and MITO Int J Gynecol Cancer 2011 21 289 295 21270612 5 AGO Study Group (DE) Study comparing tumor debulking surgery versus chemotherapy alone in recurrent platinum-sensitive ovarian cancer (DESKTOP III). ClinicalTrials.gov Identifier: NCT01166737 [Internet] Bethesda, MD National Institutes of Health 2010 cited 2017 Sep 11 Available from: https://clinicaltrials.gov/ct2/show/NCT01166737 6 Chi DS McCaughty K Diaz JP Huh J Schwabenbauer S Hummer AJ Guidelines and selection criteria for secondary cytoreductive surgery in patients with recurrent, platinum-sensitive epithelial ovarian carcinoma Cancer 2006 106 1933 1939 16572412 7 Cowan RA Eriksson AG Jaber SM Zhou Q Iasonos A Zivanovic O A comparative analysis of prediction models for complete gross resection in secondary cytoreductive surgery for ovarian cancer Gynecol Oncol 2017 145 230 235 28285846 8 Bogani G Ditto A Martinelli F Signorelli M Chiappa V Leone Roberti Maggiore U Artificial intelligence estimates the impact of human papillomavirus types in influencing the risk of cervical dysplasia recurrence: progress toward a more personalized approach Eur J Cancer Prev Forthcoming 2018 9 Bogani G Tagliabue E Signorelli M Ditto A Martinelli F Chiappa V A score system for complete cytoreduction in selected recurrent ovarian cancer patients undergoing secondary cytoreductive surgery: predictors- and nomogram-based analyses J Gynecol Oncol 2018 29 e40 29533023 10 Angioli R Capriglione S Aloisi A Ricciardi R Scaletta G Lopez S A predictive score for secondary cytoreductive surgery in recurrent ovarian cancer (SeC-Score): a single-centre, controlled study for preoperative patient selection Ann Surg Oncol 2015 22 4217 4223 25808099 11 Tian WJ Chi DS Sehouli J Tropé CG Jiang R Ayhan A A risk model for secondary cytoreductive surgery in recurrent ovarian cancer: an evidence-based proposal for patient selection Ann Surg Oncol 2012 19 597 604 21732142 12 da Costa AA Valadares CV Mantoan H Saito A Salvadori MM Guimarães AP The value of secondary cytoreductive surgery in recurrent ovarian cancer and application of a prognostic score Int J Gynecol Cancer 2016 26 449 455 26825830 13 Eriksson AG Graul A Yu MC Halko A Chi DS Zivanovic O Minimal access surgery compared to laparotomy for secondary surgical cytoreduction in patients with recurrent ovarian carcinoma: perioperative and oncologic outcomes Gynecol Oncol 2017 146 263 267 28527673 14 Uzan C Morice P Rey A Pautier P Camatte S Lhommé C Outcomes after combined therapy including surgical resection in patients with epithelial ovarian cancer recurrence(s) exclusively in lymph nodes Ann Surg Oncol 2004 11 658 664 15197013 15 Janco JM Kumar A Weaver AL McGree ME Cliby WA Performance of AGO score for secondary cytoreduction in a high-volume U.S. center Gynecol Oncol 2016 141 140 147 26836496 16 Shanghai Gynecologic Oncology Group (CN) Surgery or chemotherapy in recurrent ovarian cancer (SOC 1 trial)? ClinicalTrials.gov Identifier: NCT01611766 [Internet] Bethesda, MD National Institutes of Health 2012 cited 2018 Mar 28 Available from: https://clinicaltrials.gov/ct2/show/NCT01611766 17 van de Laar R Kruitwagen RF Zusterzeel PL Van Gorp T Massuger LF Correspondence: premature stop of the soccer trial, a multicenter randomized controlled trial on secondary cytoreductive surgery: Netherlands trial register number: NTR3337 Int J Gynecol Cancer 2017 27 2 28002207