
==== Front
Cancer Inform
Cancer Inform
CIX
spcix
Cancer Informatics
1176-9351
SAGE Publications Sage UK: London, England

10.1177/11769351241275889
10.1177_11769351241275889
Original Research
Development of Prediction Model for 5-year Survival of Colorectal Cancer
https://orcid.org/0000-0003-3770-2375
Nopour Raoof
Department of Health Information Management, Student Research Committee, School of Health Management and Information Sciences Branch, Iran University of Medical Sciences, Tehran, Iran
Raoof Nopour, Department of Health Information Management, Student Research Committee, School of Health Management and Information Sciences Branch, Iran University of Medical Sciences, Tehran, 1449614535, Iran. Email: raoof.n1370@gmail.com
4 9 2024
2024
23 117693512412758893 6 2024
28 7 2024
© The Author(s) 2024
2024
SAGE Publications Ltd unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses
https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Objectives:

This study aims to introduce a prediction model based on a machine learning approach as an efficient solution for prediction purposes to better prognosis and increase CRC survival.

Methods:

In the current retrospective study, we used the data of 1062 CRC cases to analyse and establish a prediction model for the 5-year CRC survival. The machine learning algorithms were used to develop prediction models, including random Forest, XG-Boost, bagging, logistic regression, support vector machine, artificial neural network, decision tree, and K-nearest neighbours.

Results:

The current study revealed that the XG-Boost with AU-ROC of 0.906 and 0.813 for internal and external conditions gave us better insight into predictability and generalizability than other algorithms.

Conclusion:

XG-Boost can be utilised as a knowledge source for implementing intelligent systems as an assistive tool for clinical decision-making in healthcare settings to improve prognosis and increase CRC survival through various clinical solutions that doctors can achieve.

colorectal cancer
survival rate
prognostic factor
machine learning
prediction model
cover-dateJanuary-December 2024
typesetterts1
==== Body
pmcHighlights

• Machine learning algorithms were leveraged to establish prediction models for the 5-year survival of CRC.

• A combination of pathological, laboratory, therapy, socioeconomics, and lifestyle factors was used to predict this topic.

• XG-Boost is a satisfactory model for predicting the 5-year survival of CRC.

• The pathological and therapy factors are remarkable for prediction on this topic.

• This study demonstrated a favourable generalizability of the XG-Boost model in different clinical environments.

Introduction

Colorectal cancer (CRC) is the development and growth of tumour masses as abnormal cells in any region from the colon to the rectum.1,2 This cancer type, with 1 400 000 and roughly 700 000 deaths, is considered the third most prevalent and fourth cause of mortality among other cancers worldwide.3-5 As a primary concern of public health, CRC, with 135 430 and 50 260 rates related to new cases and death, is regarded as the third most common cancer and the second cause of death in the USA.6,7 This disease is increasing in developing countries, especially those achieving the Western lifestyle.8,9 Also, despite the increasing trends of CRC in incidence and mortality rate in low and middle-income countries, it is stable or in the downward state in high-income ones.10,11

CRC accounts for the third most common cancer in Iran and is considered the fourth most prevalent cancer in men after stomach, bladder, and prostate and the second in women following breast cancer.12,13 This disease constitutes 10% and 9.4% of cancer incidence and death, respectively, reaching 3.2 million cases globally by 2040.14,15 This sharp escalation is due to the increase in the population of older adults and human development.15,16 It’s demonstrated that the CRC has an ascending incidence trend in Iran.12,17 The GLOBOCAN announced that the CRC incidence will double in Iran before 2040.18,19

One way to evaluate the efficiency of the healthcare measures for disease control and the effects of various treatment plans is by estimating CRC survival. 20 The 5-year CRC survival has different amounts between nations globally; for example, this rate is less than 8% in African countries to 64%-65% in South Korea and the USA. Also, The 5-year CRC survival ranges from 27% to 85% in different points of Iran.9,21,22 The 5-year CRC survival has increased from 37.9% in 1998-2002 to 78.6% in recent years due to the early detection of this disease in the localised tumour condition. 23 Also, the current plan for CRC prognosis uses the classic Tumor, Node, and Metastasis staging classification system, so we require a more accurate and efficient prediction model to provide better insight into the prognosis and increase the treatment efficiency. 24

So far, machine learning (ML) techniques have gained competency in establishing efficient prediction models with high accuracy in various aspects of medicine.25,26 They have shown more potential in prediction aims than other methods, such as deep learning, especially when dealing with routine clinical data, despite images, signals, or videos. 27 Also, they have demonstrated more efficient and accurate survival predictability associated with cancer diseases, and they have been introduced as an alternative and convenient way to some conventional statistical methods, such as Cox, for prediction purposes even without prior knowledge of data. 28 As mentioned, developing a prediction model based on the prognostic factors has a significant role in promoting clinical solutions such as treatment strategies by early prediction prognosis of this disease at an earlier stage. Therefore, the current study aims to establish a prediction model based on machine learning to predict the 5-year CRC survival based on the prognostic factors to a better prognosis and clinical efficiency and then increase CRC survival among patients.

Methods

Community of study and database characteristics

As a data-driven and retrospective of the current study, the community research comprised CRC-confirmed cases referred to Masoud Clinical Center in Tehran City from January 2018 to December 2023. During 6 years of referral, the data of 1062 CRC cases were stored in one database. Six hundred sixty-one and 401 cases were associated with the non-survived and survived instances, respectively. The surviving cases were patients who had the CRC-positive diagnosis in their records at that centre and were alive 5 years after the primary diagnosis. On the contrary, the non-survived cases had similar characteristics but died after 5 years.

Database preprocessing

In the current study, we first prepared the current database to establish the optimal prediction model for CRC survival. First, any duplicate and noisy data were removed from the study. Second, we confronted the missing values in cases in the database. We had two scenarios for the missing values: 1-for the instances with more than 5% missing values, we excluded these cases from the study to prevent bias in further analysis. The rationale for using this amount for deleting the cases was associated with two factors: The current database had almost complete data, and considering this amount didn’t challenge our database with losing a high case omission rate. Also, with this low amount considered for case omission, the characteristics of the original database would be more preserved. 2-If the lost data in cases were less than 5%, we leveraged the mode of each value associated with the corresponding feature to fill in the missing values.

Input and outcome variables

The prognostic factors included in the current database were categorised into demographic factors, including age, sex, place of residence, Body Mass Index (BMI), lifestyle factors such as smoking and alcohol consumption, personal history of diabetes and hypertension, familial history of CRC, therapy factors such as surgery, chemotherapy, radiotherapy, and hormonotherapy, pathological factors including, tumor stage, tumor recurrence, tumor differentiation, lymphovascular invasion, perineural invasion, tumor location, and laboratory factors such as hemoglobin and white blood cells (WBC) count. The outcome variable was the 5-year survival status of CRC patients categorised into two situations of survived and non-survived, specified as 0 and 1 codes in the current database, respectively.

Feature selection

Before performing the ML process, we leveraged the feature selection (FS) technique. FS is defined as eliminating irrelevant features to clean and increase the quality of the database. 29 It is beneficial in preventing the overfitting of the ML algorithms, enhancing the speed of calculations, increasing the accuracy of algorithms, better understating the data, and improving generalisation. 30 In the current study, we used binary logistic regression (BLR) as a combinatory correlation analysis and weighting by Gini Index (GI) score to obtain the best prognostic factors influencing the 5-year CRC survival. The P < .05 was considered to select the best factors. We put the GI = 0.5 as a threshold to select the factors based on the GI. In other words, the factors with GI ⩽ 0.5 (more robust classification of cases by variable) were considered influential factors in predicting the 5-year CRC survival among patients. On the contrary, the factors obtained GI > 0.5 (weaker classification of cases by variable) were excluded from the data analysis and ML model building.

Model establishment and assessment

We leveraged chosen ML algorithms to build the prediction models for the 5-year CRC survival. In this respect, the algorithms of Random Forest (RF), XG-Boost, bagging, logistic regression (LR), Support Vector Machine (SVM), Artificial Neural Networks (ANN), Decision Tree (DT), and K-nearest Neighbors (KNN) were utilised to establish the prediction models. These ML algorithms were leveraged due to their high applicability and favourable performance for prediction purposes, as demonstrated in other biomedical research.31,32 The performance of the chosen algorithms was evaluated by using the positive predictive value (PPV) (equation (1)), negative predictive value (NPV) (equation (2)), sensitivity (equation (3)), specificity (equation (4)), accuracy (equation (5)), F-Score (equation (6)), and the Area Under the Receiver Operator Characteristics (AU-ROC). In equations, the TP and TN refer to non-survived and survived cases correctly categorised by the ML algorithms. FN and FP are these cases that are incorrectly classified.

The Grid search method was used to adjust the ML algorithms’ hyperparameters during the learning process. In this method, each algorithm is tested manually using different ranges of hyperparameter values, and the best algorithm is chosen based on the best combination of values, possessing higher performance. We used the K-fold cross-validation as a data-splitting strategy for the learning process. In this strategy, the data are split into K sections. One section tests the algorithms’ performance, and (K-1) sections are used for training them. This process randomly occurs in K epochs, and the data are chosen randomly for training and testing processes in each epoch. The algorithms’ performance is equal to the average performance of epochs. The stratified K-fold cross-validation is more common considering the data imbalance based on the class distribution. This way, the data are sampled randomly based on the class distribution, which is crucial in reducing the bias created during the learning process. 33 In the current study, the (K = 10) was used to split the data as a widely used data-splitting strategy in other studies, 34 so the stratified 10-fold cross-validation is leveraged for testing and training purposes.

(1) PPV=TPTP+FP

(2) NPV=TNTN+FN

(3) Sensitivity=TPTP+FN

(4) Specificity=TNTN+FP

(5) Accuracy=TP+TNTP+FN+FP+TN

(5) F−Score=TPTP+(1/2(FN+FP))

External validation assessment

In the current study, we used the external validation assessment to measure the generalizability and applicability of our prediction model in other clinical settings. We used 108 CRC cases from the Imam Khomeini Hospital in Sari City to evaluate the generalizability. Among these CRC cases, 68 and 40 cases were associated with non-survived and survived instances, respectively. To assess the generalizability of the algorithm, we used the classifiability metrics of TP, FP, FN, and TN, as well as the AU-ROC in the internal and external modes. Moreover, we highlighted the importance of the prognostic factors in the internal and external modes. In this regard, we used the relative importance of the two modes (RI) obtained by the ML algorithm.

Result

Database preparation and sample characteristics

First, by identifying the duplicate cases belonging to one patient with the same ID in the current database, eight cases were removed from the study, including two and six cases associated with the survived and non-survived cases. Second, 11 cases, including 4 and 7 cases belonging to survived and non-survived patients with more than 5% missing data, were excluded from the current study. The lost data of 31 cases with less than 5% missing data were filled by the mode of the values of the corresponding feature. Finally, the 1043 CRC cases were included in the current study. Among these cases, 648 and 395 belonged to the non-survived and survived cases, respectively. The statistical characteristics of the cases included in the data analysis are presented in Table 1. The difference between the two survived and non-survived groups is given by the Chi-square (P < .05).

Table 1. Characteristics of cases included in the analysis.

Features	Values	Total cases n = 1043	Non-survived cases n = 648	Survived cases n = 395	P-value	
Age (years)	<55	379	247	132	<0.001*	
>55	664	401	263	
Sex	Male	496	333	163	0.1	
Female	547	315	232	
Place of residence	Rural	646	397	249	0.08	
Urban	397	251	146	
BMI (Kg/m)*	<18.5	334	254	80	<0.001*	
18.5-25	423	333	90	
25-30	212	43	169	
>30	74	18	56	
Smoking	Yes	548	416	132	<0.001*	
No	495	232	263	
Alcohol consumption	Yes	227	167	60	0.1	
No	816	481	335	
Diabetes	Yes	623	503	120	<0.001*	
No	420	145	275		
Hypertension	Yes	479	329	150	<0.001*	
No	564	319	245	
Familial history of CRC	Yes	448	309	139	<0.001*	
No	595	339	256	
Surgery	Yes	876	540	336	<0.001*	
No	167	108	59	
Chemotherapy	Yes	799	490	309	<0.001*	
No	244	158	86	
Radiotherapy	Yes	622	370	252	<0.001*	
No	421	278	144	
Hormonotherpay	Yes	355	224	131	0.04*	
No	688	421	267	
Tumor stage	I	127	42	85	<0.001*	
II	496	258	238	
III	324	289	36	
IIII	96	59	37	
Tumor recurrence	Yes	325	248	77	<0.001*	
No	718	400	318	
Tumor differentiation	Grade 1	397	198	199	<0.001*	
Grade 2	438	316	122	
Grade 3	208	134	74	
lymphovascular invasion	Yes	652	349	303	<0.001*	
No	391	299	92	
Perineural invasion	Yes	489	317	172	<0.001*	
No	554	331	223	
Tumor location	Rectum	207	86	121	<0.001*	
Right and Transverse colon	386	318	68	
	Left colon	397	210	187	
	Sigmoid	53	34	19	
Hemoglobin level	Low	688	434	254	<0.001*	
Normal	355	214	141	
WBC count (/mL**)	<55 000	691	394	297	<0.001*	
>55 000	352	254	98	
* Kilogram/metre. **per microliter.

According to Table 1, the prognostic factors including age, BMI, smoking, diabetes, hypertension, familial history of CRC, surgery, chemotherapy, radiotherapy, hormonotherapy, tumor stage, tumor recurrence, tumor differentiation, lymphovascular invasion, perineural invasion, tumor location, hemoglobin level, and WBC obtained difference between the survived and non-survived cases at P < .05. On the contrary, the sex, place of residence, and alcohol consumption didn’t significantly differ between the two groups.

Feature selection

The results of feature selection based on the BLR are presented in Table 2.

Table 2. The BLR to screen important factors.

Features	β	OR	95% CI of OR	P -value	
Age	.563	1.248	[1.113-1.684]	<.001	
Sex	.106	1.017	[0.963-1.078]	.12	
Place of residence	−.261	0.894	[0.745-1.284]	.2	
BMI	−.548	0.693	[0.559-0.827]	<.001	
Smoking	.447	1.224	[1.126-1.487]	<.001	
Alcohol consumption	.321	1.106	[0.894-1.185]	0.1	
Diabetes	.281	1.095	[1.07-1.113]	<.001	
Hypertension	.119	1.025	[0.898-1.05]	0.1	
Familial history of CRC	.552	1.288	[1.201-1.455]	<.001	
Surgery	.761	1.453	[1.394-2.217]	<.001	
Chemotherapy	.807	1.526	[1.438-2.543]	<.001	
Radiotherapy	.61	1.399	[1.296-1.984]	<.001	
Hormonotherpay	.221	1.069	[1.05-1.1]	<.001	
Tumour stage	.746	1.427	[1.334-1.761]	<.001	
Tumor recurrence	.597	1.303	[1.206-1.829]	<.001	
Tumor differentiation	.664	1.474	[1.328-1.683]	<.001	
lymphovascular invasion	.499	1.266	[1.201-1.384]	<.001	
Perineural invasion	.315	1.102	[1.077-1.192]	.01	
Tumor site	.689	1.494	[1.399-1.548]	<.001	
Hemoglobin level	.577	1.295	[1.202-1.384]	<.001	
WBC count	.621	1.374	[1.284-1.475]	<.001	

In Table 2, β implies the regression coefficient, OR is the odd ratio, and CI is the confidence interval. As this table shows, the factors including age (β = .563, OR = 1.248, 95% CI of OR = [1.113-1.684]), BMI(β = −.548, OR = 0.693, 95% CI of OR = [0.559-0.827]), smoking (β = .447, OR = 1.224, 95% CI of OR = [1.126-1.487]), diabetes (β = .281, OR = 1.095, 95% CI of OR = [1.07-1.113]), familial history of CRC (β = .552, OR = 1.288, 95% CI of OR = [1.201-1.455]), surgery (β = .761, OR = 1.453, 95% CI of OR = [1.394-2.217]), chemotherapy (β = .807, OR = 1.526, 95% CI of OR = [1.438-2.543]), radiotherapy (β = .61, OR = 1.399, 95% CI of OR = [1.296-1.984]), hormonotherapy (β = .221, OR = 1.069, 95% CI of OR = [1.05-1.1]), tumor stage (β = .746, OR = 1.427, 95% CI of OR = [1.334-1.761]), tumor recurrence (β = .597, OR = 1.303, 95% CI of OR = [1.206-1.829]), tumor differentiation (β = .664, OR = 1.474, 95% CI of OR = [1.328-1.683]), lymphovascular invasion (β = .499, OR = 1.266, 95% CI of OR = [1.201-1.384]), perineural invasion (β = .315, OR = 1.102, 95% CI of OR = [1.077-1.192]), tumor site (β = .689, OR = 1.494, 95% CI of OR = [1.399-1.548]), hemoglobin level (β = .577, OR = 1.295, 95% CI of OR = [1.202-1.384]), and WBC count (β = .621, OR = 1.374, 95% CI of OR = [1.284-1.475]) obtained the significant relationship with 5-year CRC survival (P < .05). On the contrary, the prognostic factors, including sex, place of residence, alcohol consumption, and hypertension, were excluded from further analysis.

Figure 1 shows the importance of prognostic factors based on the GI scoring technique.

Figure 1. Scoring the prognostic factors by using the GI score.

As Figure 1 shows, the factors including age (GI = 0.35), smoking (GI = 0.29), diabetes (GI = 0.47), familial history of CRC (GI = 0.37), surgery (GI = 0.25), chemotherapy (GI = 0.22), radiotherapy (GI = 0.29), hormonotherapy (GI = 0.33), tumor stage (GI = 0.17), tumor recurrence (GI = 0.26), tumor differentiation (GI = 0.24), lymphovascular invasion (GI = 0.18), perineural invasion (GI = 0.35), tumor location (GI = 0.26), hemoglobin level (GI = 0.33), and WBC count (GI = 0.39) were considered as the best factors for predicting the 5-year CRC survival by obtaining GI < 0.5. On the contrary, the factors including sex (GI = 0.64), place of residence (GI = 0.68), BMI (GI = 0.53), alcohol consumption (GI = 0.56), and hypertension (GI = 0.62) with GI > 0.5 were not considered as critical prognostic factors based on the GI.

Model development and assessment

The results of the performance evaluation of chosen algorithms by using 10-fold cross-validation as a data splitting strategy with the optimised hyperparameters by the Grid search method in three conditions of all features, selected features by BLR, and features chosen by using the GI are presented in Tables 3 and 4, respectively.

Table 3. The performance evaluation of chosen algorithms.

Algorithm	Feature selection	PPV (%)	NPV (%)	Sensitivity (%)	Specificity (%)	Accuracy (%)	F-Score (%)	AU-ROC	
RF	BLR	84.69	70.16	80.25	76.2	78.72	82.41	0.811	
GI	84.16	70.81	81.17	74.94	78.81	82.64	0.783	
None	85.83	74.75	84.1	77.22	81.5	84.96	0.825	
XG-Boost	BLR	95.93	91.34	94.6	93.42	94.15	95.26	0.892	
GI	95.03	95.51	97.38	91.65	95.21	96.19	0.906	
None	92.19	88.21	92.9	87.09	90.7	92.54	0.898	
Bagging	BLR	88.36	84.99	91.36	80.25	87.15	89.83	0.865	
GI	89.46	85.75	91.67	82.28	88.11	90.55	0.883	
None	87.90	76.87	85.19	80.76	83.51	86.52	0.824	
SVM	BLR	86.45	79.27	87.65	77.47	83.8	87.05	0.815	
GI	84.42	73.57	83.64	74.68	80.25	84.03	0.795	
None	82.12	68.61	80.09	71.39	76.8	81.09	0.763	
ANN	BLR	78.36	62.93	76.54	65.32	72.29	77.44	0.724	
GI	77.82	58.71	71.45	66.58	69.61	74.5	0.701	
None	75.61	57.24	71.76	62.03	68.07	73.63	0.668	
DT	BLR	83.67	65.71	75.93	75.7	75.84	79.61	0.726	
GI	79.02	62.15	75	67.34	72.1	76.96	0.714	
None	77.16	57.52	70.37	65.82	68.65	73.61	0.679	
KNN	BLR	81.45	69.44	81.33	69.62	76.89	81.39	0.786	
GI	79.61	68.88	81.94	65.57	75.74	80.76	0.765	
None	78.95	67.46	81.02	64.56	74.78	79.97	0.744	
LR	BLR	72.76	52.38	67.59	58.48	64.14	70.08	0.727	
GI	71.28	49.89	65.12	56.96	62.03	68.06	0.712	
None	69.45	47.26	62.81	54.68	59.73	65.96	0.694	

Table 4. The hyperparameters optimised by Grid search.

FS-Algorithm	Hyperparameters	
BLR-RF	Maximum depth = 6, number of estimators = 12, maximum number of features = 5, maximum leaf nodes = 3	
GI-RF	Maximum depth = 9, number of estimators = 15, maximum number of features = 9, maximum leaf nodes = 3	
RF (none-FS)	Maximum depth = 11, number of estimators = 17, maximum number of features = 11, maximum leaf nodes = 3	
BLR-XG-Boost	Booster = gradient boosted tree, eta = 0.15, minimum child weight = 1, maximum depth = 5	
GI-XG-Boost	Booster = gradient boosted tree, eta = 0.25, minimum child weight = 1, maximum depth = 8	
XG-Boost (None-FS)	Booster = gradient boosted tree, eta = 0.35, minimum child weight = 1, maximum depth = 12	
BLR-Bagging	Base classifier = J-48, number of iterations = 15, calculate out of bag = false	
GI-Bagging	Base classifier = J-48, number of iterations = 20, calculate out of bag = false	
Bagging (None-FS)	Base classifier = REP-Tree, number of iterations = 30, calculate out of bag = true	
BLR-SVM	Control parameter (C) = 10, kernel type = RBF, RBF_gamma = 0.15, gamma = 2	
GI-SVM	Control parameter (C) = 10, kernel type = RBF, RBF_gamma = 0.25, gamma = 1.5	
SVM (None-FS)	Control parameter (C) = 20, kernel type = Linear, gamma = 1	
BLR-ANN	Hidden layers = 10, learning rate = 1, maximum epoch = 50	
GI-ANN	Hidden layers = 8, learning rate = 0.7, maximum epoch = 30	
ANN (None-FS)	Hidden layers = 20, learning rate = 0.5, maximum epoch = 200	
BLR-DT	Confidence factor = 0.1, minimum number of object = 1, binary splitting = false, reduced error pruning = true	
GI-DT	Confidence factor = 0.2, minimum number of object = 1, binary splitting = false, reduced error pruning = true	
DT (None-FS)	Confidence factor = 0.35, minimum number of object = 1, binary splitting = false, reduced error pruning = true	
BLR-KNN	3 < K < 9, distance computation = Euclidean metric, cross validate = true, distance weighting = 1/distance.	
GI-KNN	3 < K < 9, distance computation = Euclidean metric, cross validate = true, distance weighting = 1/distance.	
KNN (None-FS)	3 < K < 9, distance computation = Euclidean metric, cross validate = true, distance weighting = 1/distance.	
BLR-LR	Maximum number of iterations = 20, number of decimal places = 4	
GI-LR	Maximum number of iterations = 30, number of decimal places = 5	
LR (None-FS)	Maximum number of iterations = 50, number of decimal places = 5	

Based on Tables 3 and 4, using the best-optimised hyperparameters, first, we compared the performance of algorithms in conditions and then determined the best one for prediction purposes. The RF with PPV of 85.83%, NPV of 74.75%, sensitivity of 84.1%, specificity of 77.22%, accuracy of 81.5%, F-score of 84.96%, and AU-ROC of 0.825 in the non-FS state obtained high performance than other states. XG-Boost with PPV of 95.93% and specificity of 93.42% in the BLR FS method and NPV of 95.51%, sensitivity of 97.38%, accuracy of 95.21%, F-score of 96.19%, and AU-ROC of 0.906 in the GI FS method outperformed other states.

Bagging with PPV of 89.46%, NPV of 85.75%, sensitivity of 91.67%, specificity of 82.28%, accuracy of 88.11%, F-score of 90.55% and AU-ROC of 0.883 in the GI FS method obtained higher performance than other states. SVM with PPV of 86.45%, NPV of 79.27%, sensitivity of 87.65%, specificity of 77.47%, accuracy of 83.8%, F-score of 87.05%, and AU-ROC of 0.815 with BLR FS method obtained better performance than other states. ANN with PPV of 78.36%, NPV of 62.93%, sensitivity of 76.54%, accuracy of 72.29%, F-Score of 77.44, AU-ROC of 0.724 in the BLR FS method, and specificity of 66.58 in the GI FS method achieved better-performing capability than other FS strategies. DT with PPV of 83.67%, NPV of 65.71%, sensitivity of 75.93%, specificity of 75.7%, accuracy of 75.84%, F-score of 79.61, and AU-ROC of 0.726 in the BLR FS method obtained better performance than others. KNN with PPV of 81.45%, NPV of 69.44%, specificity of 69.62%, accuracy of 76.89%, F-score of 81.39%, and AU-ROC of 0.786 in the BLR state and sensitivity of 81.94% in the GI FS method obtained better performance than other states for predicting the 5-year CRC survival.

Comparing the algorithms’ performance gave us insight into the XG-Boost with PPV of 95.93%, NPV of 95.51%, sensitivity of 97.38%, specificity of 93.42%, accuracy of 95.21%, F-score of 96.19%, and AU-ROC of 0.906 had the highest performance efficiency than other ML algorithms for predicting the 5-year CRC survival. On the contrary, the ANN with PPV of 75.61%, NPV of 57.24%, sensitivity of 71.45%, specificity of 62.03%, accuracy of 68.07%, and AU-ROC of 0.668 obtained the worst performance capability than other ML algorithms. The DT, with F-score of 73.61%, had the lowest performance in terms of this performance criterion. In addition, the LR with PPV of 72.76%, NPV of 52.38%, sensitivity of 67.59%, specificity of 58.48%, accuracy of 64.14%, F-Score of 70.08%, and AU-ROC of 0.727 in BLR-FS mode, PPV of 71.28%, NPV of 49.89%, sensitivity of 65.12%, specificity of 56.96%, accuracy of 62.03%, F-Score of 68.06%, and AU-ROC of 0.712 in GI mode, and PPV of 69.45%, NPV of 47.26%, sensitivity of 62.81%, specificity of 54.68%, accuracy of 59.73%, F-Score of 65.96%, and AU-ROC of 0.694 had the lowest performance than other ML in all states.

External validation test and feature assessment

As cited in the methods section, we tested our model’s performance capability to predict the 5-year CRC survival using unfamiliar data. Hence, we used the 108 CRC cases, including 68 and 40 cases associated with non-survived and survived instances, to demonstrate the best-performing model’s generalizability by feeding these data to the model. In this regard, the XG-Boost model was selected for external validity purposes. The results of classifying the external cases based on the TP, FP, FN, and TN in the XG-Boost in three conditions, including the BLR and GI as FS methods and XG-Boost without any FS, are presented in Table 5.

Table 5. The results of the external cases classification.

Algorithm	TP	FN	FP	TN	
BLR-XG-Boost	58	10	7	33	
GI-XG-Boost	55	13	13	27	
XG-Boost (None-FS)	51	17	17	23	

Based on Table 5, The XG-Boost model with TP = 58, FN = 10, FP = 7, and TN = 33 using the BLR as FS method obtained a higher performance than other conditions. The second rank belonged to the XG-Boost with GI as FS with TP = 55, FN = 13, FP = 13, and TN = 27. The XG-Boost without FS gained the worst performance with TP = 51, FN = 17, FN = 17, and TN = 23 with more error-classified cases. The results of comparing the model’s predictability based on the AU-ROC curve are depicted in Figure 2.

Figure 2. The XG-Boost in internal and external validations.

Figure 2 shows that the XG-Boost with BLR as FS by AU-ROC of 0.813 obtained better predictability than other conditions (closer curve to the sensitivity vertices). The XG-Boost with GI as FS by AU-ROC of 0.787 obtained the second rank, and this model without any FS method with AU-ROC of 0.763 obtained the worst predictability. Generally, by comparing this curve between internal and external conditions, we comprehended that the performance reduction of the XG-Boost was almost 10% in AU-ROC, indicating the favourable generalizability in other clinical environments (The values of AU-ROC of XG-Boost in internal conditions were presented in Table 3). The XG-Boost with BLR as FS method was considered the best model and favourable generalizability for predicting the 5-year CRC survival in the current study. We used the BLR-XG-Boost model as the best-performing algorithm to assess the importance of each prognostic factor. Hence, we assessed their importance based on the Relative Importance (RI) gained by this model in the internal and external conditions. The results of scoring each prognostic factor based on the RI are shown in Figure 3.

Figure 3. The importance of prognostic factors in internal and external conditions.

According to Figure 3, almost all pathological factors obtained high importance. The pathological factors, including tumor differentiation (internal RI = 0.39, external RI = 0.41), tumor recurrence (internal RI = 0.44, external RI = 036), and lymphovascular invasion (internal RI = 0.43, external RI = 0.41) obtained higher importance than other pathological factors in terms of predicting 5-year survival. The therapy factors, including chemotherapy (internal RI = 0.48, external RI = 0.45) and surgery (internal RI = 0.45, external RI = 0.42), achieved more strength in predicting the 5-year survival than other prognostic factors. On the contrary, the age (internal RI = 0.15, external RI = 0.21) and BMI (internal RI = 0.18, external RI = 0.21) obtained less importance in this respect.

Discussion

In the current study, we intended to establish a prediction model for the 5-year CRC survival disease for a better prognosis, especially for the high-risk group of CRC patients with a poor prognosis, influencing the survival status. In this respect, we got assistance from one single-centred database containing prognostic factors with the help of the ML approach for effective and efficient predictability in various clinical situations. To achieve this aim, we first investigated the current database concerning any redundancy, noisy, and missing data to prepare it for data analysis. Next, we used two different strategies of the feature selection process to choose the best factors influencing the five-year CRC survival. After that, we used selected ML algorithms, including RF, XG-Boost, bagging, SVM, ANN, DT, LR, and KNN, to establish prediction models for predicting survival. We compared their performance to get the best ones in terms of predictive efficiency. Based on the best-performing trained algorithm, we used the external data cases to test the generalizability of the algorithm in other clinical environments. Also, the prognostic factors were assessed by using the best-performing algorithm in internal and external validations.

We had two feature selection strategies, including BLR and GI, as filtering methods to gain the best factors influencing the 5-year CRC survival. Based on the BLR, the factors including age, BMI, smoking, diabetes, familial history of CRC, surgery, chemotherapy, radiotherapy, hormonotherapy, tumor stage, tumor recurrence, tumor differentiation, lymphovascular invasion, perineural invasion, tumor location, hemoglobin level, and WBC were obtained as the essential factors to this aim. GI showed that the age, smoking, diabetes, familial history of CRC, surgery, chemotherapy, radiotherapy, hormonotherapy, tumor stage, tumor recurrence, tumor differentiation, lymphovascular invasion, perineural invasion, tumor location, hemoglobin level, and WBC count as best prognostic factors.

The current study gave us insight into the XG-Boost with PPV of 95.93%, NPV of 95.51%, sensitivity of 97.38%, specificity of 93.42%, accuracy of 95.21%, F-score of 96.19%, and AU-ROC of 0.906 obtained the best performance than others. Also, this algorithm with AU-ROC of 0.813 obtained favourable generalizability in predicting the 5-year CRC survival based on the external cases. Based on the XG-Boost, the prognostic factors, including tumor differentiation (internal RI = 0.39, external RI = 0.41), tumor recurrence (internal RI = 0.44, external RI = 036), lymphovascular invasion, chemotherapy (internal RI = 0.48, external RI = 0.45), and surgery (internal RI = 0.45, external RI = 0.42) were considered as the most important factors concerning 5-year CRC survival prediction. As shown, XG-Boost is an algorithm that has the potential to predict 5-year CRC survival. This algorithm could be considered an effective and efficient model for predicting 5-year CRC survival and be embedded as a knowledge base in intelligent systems, such as clinical decision support systems in clinical environments. The doctors in those settings could enter the CRC patients’ prognostic characteristics into the system and get the results of the CRC survival risk. Some clinical solutions, such as the early identification of CRC recurrence and interventional and non-interventional therapies, can be performed for high-risk patients. This way, the prognosis of these patients would be enhanced, and consequently, their survival would be increased.

So far, several studies have been conducted on CRC survival based on ML algorithms. Cardoso et al. used the ML technique to predict CRC patients’ survival. Based on their results, the XG-Boost with an AU-ROC of 0.857 obtained the best performance for prediction purposes. The clinical stage in their study was recognised as the best predictor of CRC survival. 35 In the current study, despite Cardoso’s study, which used more treatment factors, we focused more on both pathological and treatment factors. The pathological factors obtained higher competency than other factors for prediction purposes. Also, we tested the XG-Boost for external data cases, showing the favourable interoperability of the model in different clinical settings; this subject wasn’t considered in Cardoso’s study.

Yang et al. attempted to establish ML models to predict CRC survival using multi-omics data from the Cancer Genome Atlas (TCGA). They leveraged the bioinformatics analysis to omics data and then trained ML algorithms on these data. Their best-performing model obtained an AU-ROC of 0.755 with 10-fold cross-validation. 36 In the current study, leveraging the other factors, including the treatment, pathological, laboratory, demographic, and others, gave us better insight into the ML’s performance concerning CRC survival, even XG-Boost with an AU-ROC of 0.813 in the external validation state. Bibaault et al. built a prediction model for CRC survival based on the gradient-boosting algorithm using tumour characteristics, socioeconomics, and lifestyle factors. In their study, the model obtained an AU-ROC of 0.84 for prediction. In the current study, we focused more on pathological factors and concluded that they significantly enhance predictability. Also, our results indicated that the XG-Boost model with the AU-ROC of 0.906 and 0.813 for internal and external validation conditions performed more effectively in predicting 5-year CRC survival and usability in other clinical environments. 37

Achilonu et al. used ML and statistical approaches to predict CRC recurrence and patient survival. They showed that the ANN with an AU-ROC of 0.82 had better performance capability for predicting survival than other ML approaches. Also, their study recognised the factors, including histology, as the best factor influencing survival. 38 In the current study, similar to Achilonu’s study, pathological factors gained more importance than other factors, especially sociodemographic ones. Despite Achilonu’s study, we used more pathological and therapy factors, and in this condition, the performance of our ML model with an AU-ROC of 0.906 was more favourable than Achilonu’s study.

Pourhoseingholi compared ensemble and non-ensemble ML techniques for predicting the five-year CRC survival. In this respect, some prognostic factors, including tumour characteristics and therapy factors, were used to establish prediction models. Their study demonstrated that the voting algorithm with an AU-ROC of 0.96 is the best-performing model for predicting survival. 39 In this study, we focused more on pathological data and weren’t satisfied only by TNM data and the performance gained by internal data cases. Hence, the external data were used to test the prediction model’s generalizability and demonstrate our prediction model’s clinical usability in other clinical environments. BalajiVicharapu’s study used pathological, laboratory, and lifestyle factors to build a prediction model for this topic. The RF with AU-ROC of nearly 0.82 obtained better performance than other ML techniques. 40 The treatment factors such as performing surgery, radiotherapy, and chemotherapy are crucial prognostic factors in this respect, and in BalajiVicharapu’s study, they weren’t considered.

In the current study, despite the previous ones, we attempted to use the combinations of prognostic factors, including pathological factors in addition to TNM data, laboratory data, therapy factors, socioeconomics, and lifestyle, to establish a prediction model for the 5-year CRC survival. Also, we used the external data to test the current prediction model’s performance ability in other clinical environments, which was lacking in the previous studies. These testing scenarios gave us insight into the XG-Boost, which has favourable performance in different environments, assuring the clinical usability of the prediction model in other clinical centres.

Limitation

Despite the mentioned benefits, the current study had some limitations. First, we used the database from one clinical centre, which may somewhat influence the generalizability of the current prediction model. Second, we used some preprocessing steps to adapt the current database to establish prediction models that affect the model’s accuracy and generalizability. Despite this limitation, the external performance of the current prediction model was favourable, indicating low bias in the prediction performance (almost a 10% reduction in AU-ROC value). Third, the current study didn’t consider some factors, including genomic data and tumour markers, due to a lack of information in the database. Fourth, we had limitations in leveraging essential factors that may not be considered due to this study’s retrospective nature. Fifth, one critical step in estimating the bias of the prediction model is leveraging the external validation cohort. This way, we can assess the clinical usability and generalizability of the model to other clinical settings. Although we used this method for this aim, the samples used for this work were almost small, which may not give us a complete insight into the generalizability of the prediction model.

Conclusion

An effective and efficient prediction of CRC survival can potentially enhance CRC patients’ prognosis. This study demonstrated that XG-Boost with an AU-ROC of 0.906 can better prognosis and increase CRC survival. The model can be leveraged as a knowledge source to establish an efficient prediction system to achieve this aim.

Future Directions

For future studies, we recommend using more data from several centres for better predictability and generalizability, using actual data to fill in missing values instead of preprocessing steps to enhance the generalizability of the prediction models. The inclusion of some factors, including genomic data and tumour markers, may influence the performance to some extent, so we recommend using these factors for more enhanced survival predictability. We also suggest using a cohort study to investigate all aspects of the research and the other essential factors that should be included. Leveraging more clinical data from more clinical centres for external validity is indispensable to assess the bias and increase the prediction model’s generalizability more confidently, and it should be considered.

We thank the people and specialists who assisted us in all steps of this study.

Author Contribution: R.N. performed the writing, review, and editing of this manuscript.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.

Availability of Data and Materials: The research data are available from the corresponding author upon reasonable request.

Ethics Approval and Consent to Participate: This study was approved by the Ethics Committee of Tehran University of Medical Sciences (TUMS) (No: IR.TUMS.SPH.REC.1398.191). Due to the retrospective nature of this study, it’s waived from the informed consent.

Consent for Publication: Not applicable

ORCID iD: Raoof Nopour https://orcid.org/0000-0003-3770-2375
==== Refs
References

1 Ghani RA Osman NMF . Knowledge and awareness of diabetes mellitus as a risk factor of colorectal cancer among international Islamic University Malaysia Kuantan students. Asian J Med Biomed. 2022;6 (2 ):143-153.
2 Testa U Pelosi E Castelli G. Colorectal cancer: genetic abnormalities, tumor progression, tumor heterogeneity, clonal evolution and tumor-initiating cells. Med Sci. 2018;6 (2 ):31.
3 Eileen M Melina A Gini A , et al . Global burden of colorectal cancer in 2020 and 2040: incidence and mortality estimates from GLOBOCAN. Gut. 2023;72 (2 ):338. doi:10.1136/gutjnl-2022-327736 36604116
4 Gandomani HS Aghajani M Mohammadian-Hafshejani A Tarazoj AA Pouyesh V Salehiniya H. Colorectal cancer in the world: incidence, mortality and risk factors. Biomed Res Therapy. 2017;4 (10 ):1656-1675.
5 Koilakou S Petrou P. Economic evaluation of monoclonal antibodies in metastatic colorectal cancer: a systematic review. Mol Diag Therapy. 2021;25 :715-734.
6 Ansa BE Coughlin SS Alema-Mensah E Smith SA. Evaluation of colorectal cancer incidence trends in the United States (2000–2014). J Clin Med. 2018;7 (2 ):22. doi:10.3390/jcm7020022 29385768
7 Schlottmann F Strassle P Cairns A Herbella F Fichera A Patti M. Disparities in emergent colectomy for colorectal cancer contribute to inequalities in postoperative morbidity and mortality in the US health care system. Scand J Surg. 2020;109 (2 ):102-107.30696360
8 Rawla P Sunkara T Barsouk A. Epidemiology of colorectal cancer: incidence, mortality, survival, and risk factors. Gastroenterol Rev/Przegląd Gastroenterol. 2019;14 (2 ):89-103.
9 Keum N Giovannucci E. Global burden of colorectal cancer: emerging trends, risk factors and prevention strategies. Nature Rev Gastroenterol & Hepatol. 2019;16 (12 ):713-732.31455888
10 Aguiar S Jr Oliveira MM Mello CA Calsavara VF Curado MP. Survival of patients with colorectal cancer in a cancer center. Arq de Gastroenterol. 2020;57 :172-177.
11 Arnold M Sierra MS Laversanne M Soerjomataram I Jemal A Bray F. Global patterns and trends in colorectal cancer incidence and mortality. Gut. 2017;66 (4 ):683-691.26818619
12 Dolatkhah R Somi MH Kermani IA , et al . Increased colorectal cancer incidence in Iran: a systematic review and meta-analysis. BMC Public Health. 2015;15 (1 ):997. doi:10.1186/s12889-015-2342-9 26423906
13 Amirkhah R Naderi-Meshkin H Mirahmadi M Allahyari A Sharifi HR. Cancer statistics in Iran: towards finding priority for prevention and treatment. Cancer Press Journal. 2017;3 (2 ):27-38.
14 Basudan AM Basuwdan AM Abudawood M Farzan R Alfhili MA. Comprehensive retrospective analysis of colorectal cancer incidence patterns in Saudi Arabia. Life. 2023;13 (11 ):2198.
15 Xi Y Xu P. Global colorectal cancer burden in 2020 and projections to 2040. Translat Oncol. 2021;14 (10 ):101174.
16 Vabi BW Gibbs JF Parker GS. Implications of the growing incidence of global colorectal cancer. J Gastro Oncol. 2021;12 (Suppl 2 ):S387.
17 Abdifard E Amini S Bab S Masroor N Khachian A Heidari M. Incidence trends of colorectal cancer in Iran during 2000-2009: a population-based study. Med J Islam Rep Iran. 2016;30 :382.
18 Hoseini B Rahmatinejad Z Goshayeshi L , et al . Colorectal Cancer in North-Eastern Iran: a retrospective, comparative study of early-onset and late-onset cases based on data from the Iranian hereditary colorectal cancer registry. BMC Cancer. 2022;22 (1 ):48. doi:10.1186/s12885-021-09132-5 34998373
19 Roshandel G Ghasemi-Kebria F Malekzadeh RJC . Colorectal cancer: epidemiology, risk factors, and prevention. Cancers. 2024;16 (8 ):1530.
20 Li J Qin S Xu R-H , et al . Effect of fruquintinib vs placebo on overall survival in patients with previously treated metastatic colorectal cancer: the FRESCO randomized clinical trial. Jama. 2018;319 (24 ):2486-2496.29946728
21 Maajani K Khodadost M Fattahi A , et al . Survival rate of colorectal cancer in Iran: a systematic review and meta-analysis. Asian Pac J Cancer Prev. 2019;20 (1 ):13-21. doi:10.31557/apjcp.2019.20.1.13 30677864
22 Jiang Y Yuan H Li Z , et al . Global pattern and trends of colorectal cancer survival: a systematic review of population-based registration data. Cancer Biol Med. 2022;19 (2 ):175.
23 Dulskas A Gaizauskas V Kildusiene I Samalavicius NE Smailyte G. Improvement of survival over time for colorectal cancer patients: a population-based study. J Clin Med. 2020;9 (12 ). doi:10.3390/jcm9124038
24 Lo C-M Yang Y-W Lin J-K , et al . Modeling the survival of colorectal cancer patients based on colonoscopic features in a feature ensemble vision transformer. Comp Med Imag Graph. 2023;107 :102242. doi: 10.1016/j.compmedimag.2023.102242
25 Rahmani AM Yousefpoor E Yousefpoor MS , et al . Machine learning (ML) in medicine: review, applications, and challenges. Mathematics. 2021;9 (22 ):2970.
26 Handelman G Kok H Chandra R , et al . eD octor: machine learning and the future of medicine. J Int Med. 2018;284 (6 ):603-619.
27 Bote-Curiel L Munoz-Romero S Gerrero-Curieses A Rojo-Álvarez JL. Deep learning and big data in healthcare: a double review for critical beginners. Appl Sci. 2019;9 (11 ):2331.
28 Cruz JA Wishart DS. Applications of machine learning in cancer prediction and prognosis. Cancer Inform. 2006;2 :117693510600200030. doi:10.1177/117693510600200030
29 Venkatesh B Anuradha J. A review of feature selection and its methods. Cyb Inform Technol. 2019;19 (1 ):3-26.
30 Chandrashekar G Sahin F. A survey on feature selection methods. Comp Elect Eng. 2014;40 (1 ):16-28.
31 Nopour R. Prediction of five-year survival among esophageal cancer patients using machine learning. Heliyon. 2023;9 :1-15.
32 Nopour R. Design of risk prediction model for esophageal cancer based on machine learning approach. Heliyon. 2024;10 :1-11.
33 Wu CC Yeh WC Hsu WD , et al . Prediction of fatty liver disease using machine learning algorithms. Comp Meth Prog Biomed. 2019;170 :23-29. doi: 10.1016/j.cmpb.2018.12.032
34 Nopour R. Establishment of prediction model for mortality risk of pancreatic cancer: a retrospective study. BMC Med Inform Dec Mak. 2024;24 (1 ):181. doi:10.1186/s12911-024-02590-4
35 Buk Cardoso L Cunha Parro V Verzinhasse Peres S , et al . Machine learning for predicting survival of colorectal cancer patients. Sci Rep. 2023;13 (1 ):8874.37264045
36 Yang M Yang H Ji L , et al . A multi-omics machine learning framework in predicting the survival of colorectal cancer patients. Comp Biol Med. 2022;146 :105516. doi: 10.1016/j.compbiomed.2022.105516
37 Bibault J-E Chang DT Xing L. Development and validation of a model to predict survival in colorectal cancer using a gradient-boosted machine. Gut. 2021;70 (5 ):884-889.32887732
38 Achilonu OJ Fabian J Bebington B , et al . Predicting colorectal cancer recurrence and patient survival using supervised machine learning approach: a South African population-based study. Original Research. Frontiers in Public Health. 2021;9 :694306.34307286
39 Pourhoseingholi MA Kheirian S Zali MR. Comparison of basic and ensemble data mining methods in predicting 5-year survival of colorectal cancer patients. Acta Inform Med. 2017;25 (4 ):254-258. doi:10.5455/aim.2017.25.254-258 29284916
40 BalajiVicharapu A Patnala SCM . A study on various machine learning techniques used for colorectal cancer disease prediction and survival. Annals of the Romanian Society for Cell Biology. 2020 :748-763.
