
==== Front
Risk Manag Healthc Policy
Risk Manag Healthc Policy
rmhp
Risk Management and Healthcare Policy
1179-1594
Dove

478826
10.2147/RMHP.S478826
Original Research
Predictive Model of Internal Bleeding in Elderly Aspirin Users Using XGBoost Machine Learning
Chen et al
Chen et al
Chen Tenggao 1 *
Lei Wanlin 2 *
http://orcid.org/0000-0002-3166-0461
Wang Maofeng 2
1 Department of Colorectal Surgery, Affiliated Dongyang Hospital, Wenzhou Medical University, Dongyang, Zhejiang, 322100, People’s Republic of China
2 Department of Biomedical Sciences Laboratory, Affiliated Dongyang Hospital, Wenzhou Medical University, Dongyang, Zhejiang, 322100, People’s Republic of China
Correspondence: Maofeng Wang, Department of Biomedical Sciences Laboratory, No. 60 Wuning West Road, Affiliated Dongyang Hospital, Wenzhou Medical University, Dongyang, 322100, People’s Republic of China, Email wzmcwmf@wmu.edu.cn
* These authors contributed equally to this work

18 9 2024
2024
17 22552269
17 7 2024
15 9 2024
© 2024 Chen et al.
2024
Chen et al.
https://creativecommons.org/licenses/by-nc/3.0/ This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution – Non Commercial (unported, v3.0) License (http://creativecommons.org/licenses/by-nc/3.0/). By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms (https://www.dovepress.com/terms.php).
Objective

This study aimed to develop a predictive model for assessing internal bleeding risk in elderly aspirin users using machine learning.

Methods

A total of 26,030 elderly aspirin users (aged over 65) were retrospective included in the study. Data on patient demographics, clinical features, underlying diseases, medical history, and laboratory examinations were collected from Affiliated Dongyang Hospital of Wenzhou Medical University. Patients were randomly divided into two groups, with a 7:3 ratio, for model development and internal validation, respectively. Least absolute shrinkage and selection operator (LASSO) regression, extreme gradient boosting (XGBoost), and multivariate logistic regression were employed to develop prediction models. Model performance was evaluated using area under the curve (AUC), calibration curves, decision curve analysis (DCA), clinical impact curve (CIC), and net reduction curve (NRC).

Results

The XGBoost model exhibited the highest AUC among all models. It consisted of six clinical variables: HGB, PLT, previous bleeding, gastric ulcer, cerebral infarction, and tumor. A visual nomogram was developed based on these six variables. In the training dataset, the model achieved an AUC of 0.842 (95% CI: 0.829–0.855), while in the test dataset, it achieved an AUC of 0.820 (95% CI: 0.800–0.840), demonstrating good discriminatory performance. The calibration curve analysis revealed that the nomogram model closely approximated the ideal curve. Additionally, the DCA curve, CIC, and NRC demonstrated favorable clinical net benefit for the nomogram model.

Conclusion

This study successfully developed a predictive model to estimate the risk of bleeding in elderly aspirin users. This model can serve as a potential useful tool for clinicians to estimate the risk of bleeding in elderly aspirin users and make informed decisions regarding their treatment and management.

Keywords

aspirin
bleeding
haemorrhage
predictive model
extreme gradient boosting
nomogram
Zhejiang Provincial Natural Science Foundation of China This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LTGY23H200002.
==== Body
pmcIntroduction

Aspirin is extensively utilized in the management and prevention of various diseases, particularly coronary artery disease.1 However, the use of aspirin in elderly patients poses a challenge due to an elevated risk of bleeding.2 Balancing the prevention of cardiovascular events with minimizing bleeding risks is a major concern.3

Elderly individuals are prone to aspirin-induced gastric injury,4 Recent evidence suggests that daily aspirin use does not improve survival in healthy elderly individuals (> 70 years old). Conversely, the aspirin group had a higher incidence of major hemorrhage compared to control group.5 Several bleeding risk scores have been developed to assist in selecting appropriate treatment regimens and durations, providing valuable insights for clinical practice.6 The PRECISE-DAPT risk score accurately predicts bleeding risk in aspirin users and has been recommended (Class IIB) for identifying high-risk patients susceptible to bleeding.7 The bleeding score effectively stratifies bleeding and ischemic risk across diverse study populations, consistently providing benefit-risk difference stratification.8 European guidelines emphasize a personalized approach to balancing bleeding and ischemic risks instead of a generalized strategy for aspirin use.9 Although American and European guidelines primarily recommend PARIS and PRECISE scores, they have limitations due to variations in patient cohorts.10,11 New clinical models have recently emerged to improve hemorrhagic event prediction, incorporating commonly used scoring systems such as CRUSADE,12 ARC-HBR,13 ACUITY-HORIZONS,14 BleeMACS,15 TIMI risk score,16 HAS-Bled score,17 GRACE score,12 and CHA2DS2-VASC score.18 These scores evaluate various clinical characteristics including coronary anatomy, surgical procedures, genotyping, lifestyle factors, and treatment adherence.19 Inferior vena cava (IVC) filter placement is associated with important long-term complications.20 Predictive models for filter-related complications may help guide clinical decision-making but remain limited.20 Given the presence of multiple bleeding risk scores, there is an urgent need for a highly accurate clinical model that can adjust aspirin type and duration to minimize ischemic risk while avoiding increased bleeding risk. Each score has its advantages and limitations based on the characteristics of the patient cohorts used for development and validation, making them applicable to specific patients, clinical contexts, and timeframes.19 We previously published a study examining bleeding risk in all aspirin-using patients.21 Our current article specifically investigates bleeding risk in elderly aspirin users, recognizing this subgroup’s heightened susceptibility to bleeding. Balancing the harm and benefit remains a challenge for clinicians, especially when managing bleeding risk in elderly aspirin users. Further research is required to determine the association between bleeding events and the application of bleeding risk scores.

This study aimed to develop new bleeding risk scores for elderly aspirin users and assess their predictive capabilities.

Methods

Study Population

Participants for this study were retrospective recruited from the Affiliated Dongyang Hospital of Wenzhou Medical University. The inclusion criteria included hospitalized individuals aged over 65 years who had documented aspirin use in electronic medical records (EMRs) from January 2008 to December 2017. This study was reviewed and approved by the Ethics Committee of the Affiliated Dongyang Hospital of Wenzhou Medical University (approval #2023-YX-408). As this was a retrospective clinical data analysis, the necessity of informed consent was waived by the ethics committee. All patient medical data were anonymized and de-identified before the analysis. This research involving human participants was conducted in accordance with the principles of the Declaration of Helsinki.

Outcome Definition

The occurrence of any recorded bleeding events, such as cerebral hemorrhage, gastrointestinal bleeding, mucosal bleeding, or other common types of bleeding, within 5 years after the administration of aspirin was examined using hospital EMRs discharge records. In this study, the presence of bleeding was categorized as positive, while the absence of bleeding was considered negative.

Risk Factors

From the hospital EMRs, we collected the following information of the subjects: gender, age, height, weight, BMI, and past medical history including smoking, hypertension, drinking, surgical history, diabetes, tumors, previous bleeding events, Percutaneous Coronary Interventions (PCI), cerebral infarction, acute myocardial infarction, use of gastric protective drugs, gastric ulcers, anticoagulant usage, portal hypertension, and various clinical test indicators such as peripheral hemoglobin (HGB), platelet count (PLT), white blood cell count (WBC), and glomerular filtration rate (GFR), cardiac ejection fraction (EF). The lowest values of clinical test indicators within one month prior to starting aspirin were considered. Other past medical histories were recorded if they occurred before the initiation of aspirin.

Data Pre-Processing

The clinical research big data obtained underwent rigorous cleaning procedures, including the removal of outliers and imputation of missing values. Indicators with missing values exceeding 20%, such as height, weight, BMI, EF, and GFR, were excluded from the analysis. Multiple imputation techniques were applied to handle missing predictor values. The data was then divided into a training set (70% of the data) and a test set (remaining portion) for model training and evaluation. The classification model was trained using the training set, while the performance of the model was assessed using the test set.

Model Building

Least absolute shrinkage and selection operator (LASSO) regression and extreme gradient boosting (XGBoost) machine learning algorithms were employed to identify the optimal predictive features. Shapley additive explanation (SHAP) values were used to assess feature importance. Logistic regression modeling was performed on the 5 or 10 most significant parameters from the XGBoost model, as well as the parameters selected by LASSO regression. These parameters were grouped into three models. Performance comparison of these models included metrics such as area under the receiver operating characteristic (ROC) curve (AUC), Net Reclassification Improvement (NRI), and Integrated Discrimination Improvement (IDI). The best-performing model was selected based on these metrics. A nomogram was then developed using this model to predict bleeding risk.22–24

Model Evaluation

The model’s discrimination performance was evaluated by assessing its sensitivity, specificity, and the area under the ROC curve (AUC). Calibration was examined using calibration curves. The efficacy of the identified risk factors in predicting bleeding risk was verified through decision curve analysis (DCA), clinical impact curve (CIC), and net reduction curve (NRC). These analyses considered the net benefit for patients under different risk thresholds. Furthermore, the model was validated by comparing it to individual indicators in terms of discrimination. For a visual representation of the model construction and validation process, refer to Figure 1 in the flowchart.Figure 1 Study process flowchart.

Statistical Methods

Statistical analysis and data visualization were performed using R4.2.1 software on Windows. Continuous variables were presented as means with standard deviations or medians with interquartile ranges and compared using Student’s t-test or Mann–Whitney U-test. Categorical variables were expressed as frequencies with percentages and compared using χ²-test or Fisher’s exact test. Multiple imputation techniques were implemented using the “mice” package. The “comparegroups” package was used for baseline description and differences analysis. LASSO regression utilized the “glmnet” package, while multivariable logistic regression employed the “glm” package. The nomogram was created with the “regplot” package, and NRI calculations utilized the “nricens” package. IDI analysis was conducted using the “PredictABEL” package. Discrimination analysis utilized the “pROC”, “ggROC”, and “fbroc” packages. Calibration assessment was performed with the “rms” and “riskregression” packages. DCA and CIC were carried out using the “rmda”, “dca.R”, and “dcurves” packages. ROC analysis comparisons of multiple models were conducted with the “ROCR” package, while DCA comparisons were performed using the “Dcurves” package. Diagnostic evaluation utilized the “reportROC” package. All statistical tests were two-sided, and a significance level of P < 0.05 was considered statistically significant.

Results

Study Population Characteristics

A total of 26,030 elderly aspirin users were included in the study, with 1565 experiencing bleeding. Among the 25 variables examined, WBC, HGB, PLT, Height, Weight, BMI, EF, and GFR were continuous variables. Variables with missing information in over 20% of patients (Height, Weight, BMI, EF, and GFR) were excluded, resulting in 20 variables with less than 20% missing data (detailed in Appendix 1). No significant differences were observed in terms of myocardial infarction and PCI between cohorts with and without bleeding. Table 1 presents the baseline characteristics of the aspirin users. A random 7:3 division allocated patients to the training set (n=18,221) and the test set (n=7809). Table 2 displays the baseline characteristics of patients in both sets, showing no significant differences for each indicator between the two cohorts.Table 1 Baseline Characteristics of Subjects

Variables	Total N=26030	No Bleeding N=24465	Bleeding N=1565	p	
Sex				<0.001	
Female	11676 (44.9%)	11,113 (45.4%)	563 (36.0%)		
Male	14354 (55.1%)	13,352 (54.6%)	1002 (64.0%)		
DAPT, n (%)				<0.001	
No	16071 (61.7%)	15,255 (62.4%)	816 (52.1%)		
Yes	9959 (38.3%)	9210 (37.6%)	749 (47.9%)		
Age (years)	74.0 [69.0;79.0]	73.0 [69.0;79.0]	75.0 [70.0;81.0]	<0.001	
Smoke, n (%)				<0.001	
No	11212 (43.1%)	10,708 (43.8%)	504 (32.2%)		
Yes	14818 (56.9%)	13,757 (56.2%)	1061 (67.8%)		
Drink, n (%)				<0.001	
No	11212 (43.1%)	10,708 (43.8%)	504 (32.2%)		
Yes	14818 (56.9%)	13,757 (56.2%)	1061 (67.8%)		
DM, n (%)				0.006	
No	22916 (88.0%)	21,573 (88.2%)	1343 (85.8%)		
Yes	3114 (12.0%)	2892 (11.8%)	222 (14.2%)		
Hypertension, n (%)				<0.001	
No	15187 (58.3%)	14,479 (59.2%)	708 (45.2%)		
Yes	10843 (41.7%)	9986 (40.8%)	857 (54.8%)		
Operation, n (%)				<0.001	
No	24943 (95.8%)	23,519 (96.1%)	1424 (91.0%)		
Yes	1087 (4.18%)	946 (3.87%)	141 (9.01%)		
Tumor, n (%)				0.004	
No	25227 (96.9%)	23,730 (97.0%)	1497 (95.7%)		
Yes	803 (3.08%)	735 (3.00%)	68 (4.35%)		
MI, n (%)				0.691	
No	24746 (95.1%)	23,262 (95.1%)	1484 (94.8%)		
Yes	1284 (4.93%)	1203 (4.92%)	81 (5.18%)		
PCI, n (%)				0.666	
No	25892 (99.5%)	24,337 (99.5%)	1555 (99.4%)		
Yes	138 (0.53%)	128 (0.52%)	10 (0.64%)		
Previous bleeding, n (%)				0.000	
No	25575 (98.3%)	24,313 (99.4%)	1262 (80.6%)		
Yes	455 (1.75%)	152 (0.62%)	303 (19.4%)		
WBC (109/L)	4.87 [3.98;5.93]	4.90 [4.02;5.97]	4.30 [3.50;5.24]	<0.001	
HGB (g/L)	119 [105;131]	120 [107;131]	95.0 [73.0;114]	<0.001	
PLT (109/L)	159 [125;197]	160 [127;198]	134 [97.0;170]	<0.001	
Gastric protective medicine, n (%)				<0.001	
No	19966 (76.7%)	18,974 (77.6%)	992 (63.4%)		
Yes	6064 (23.3%)	5491 (22.4%)	573 (36.6%)		
Gastric ulcer, n (%)				<0.001	
No	25852 (99.3%)	24,324 (99.4%)	1528 (97.6%)		
Yes	178 (0.68%)	141 (0.58%)	37 (2.36%)		
Cerebral infarction, n (%)				<0.001	
No	14560 (55.9%)	14,021 (57.3%)	539 (34.4%)		
Yes	11470 (44.1%)	10,444 (42.7%)	1026 (65.6%)		
Portal hypertension, n (%)				0.014	
No	26021 (100.0%)	24,459 (100.0%)	1562 (99.8%)		
Yes	9 (0.03%)	6 (0.02%)	3 (0.19%)		
Anticoagulants, n (%)				<0.001	
No	16777 (64.5%)	15,946 (65.2%)	831 (53.1%)		
Yes	9253 (35.5%)	8519 (34.8%)	734 (46.9%)		
Abbreviations: DAPT, dual antiplatelet therapy; DM, Diabetes Mellitus; MI, Myocardial infarction; PCI, Percutaneous Coronary Intervention; WBC, White blood cell count; HGB, hemoglobin; PLT, platelet count.

Table 2 The Baseline Characteristics of the Training and Test Set

Variables	Total N=26030	Test N=7809	Train N=18221	p	
Sex				0.950	
Female	11676 (44.9%)	3500 (44.8%)	8176 (44.9%)		
Male	14354 (55.1%)	4309 (55.2%)	10,045 (55.1%)		
DAPT, n (%)				0.473	
No	16071 (61.7%)	4795 (61.4%)	11,276 (61.9%)		
Yes	9959 (38.3%)	3014 (38.6%)	6945 (38.1%)		
Age (years)	74.0 [69.0;79.0]	74.0 [69.0;79.0]	73.0 [69.0;79.0]	0.282	
Smoke, n (%)				0.980	
No	11212 (43.1%)	3365 (43.1%)	7847 (43.1%)		
Yes	14818 (56.9%)	4444 (56.9%)	10,374 (56.9%)		
Drink, n (%)				0.980	
No	11212 (43.1%)	3365 (43.1%)	7847 (43.1%)		
Yes	14818 (56.9%)	4444 (56.9%)	10,374 (56.9%)		
DM, n (%)				0.137	
No	22916 (88.0%)	6911 (88.5%)	16,005 (87.8%)		
Yes	3114 (12.0%)	898 (11.5%)	2216 (12.2%)		
Hypertension, n (%)				0.856	
No	15187 (58.3%)	4549 (58.3%)	10,638 (58.4%)		
Yes	10843 (41.7%)	3260 (41.7%)	7583 (41.6%)		
Operation, n (%)				0.808	
No	24943 (95.8%)	7487 (95.9%)	17,456 (95.8%)		
Yes	1087 (4.18%)	322 (4.12%)	765 (4.20%)		
Tumor, n (%)				0.080	
No	25227 (96.9%)	7591 (97.2%)	17,636 (96.8%)		
Yes	803 (3.08%)	218 (2.79%)	585 (3.21%)		
MI, n (%)				0.372	
No	24746 (95.1%)	7409 (94.9%)	17,337 (95.1%)		
Yes	1284 (4.93%)	400 (5.12%)	884 (4.85%)		
PCI, n (%)				1.000	
No	25892 (99.5%)	7768 (99.5%)	18,124 (99.5%)		
Yes	138 (0.53%)	41 (0.53%)	97 (0.53%)		
Previous bleeding, n (%)				1.000	
No	25575 (98.3%)	7672 (98.2%)	17,903 (98.3%)		
Yes	455 (1.75%)	137 (1.75%)	318 (1.75%)		
WBC (109/L)	4.87 [3.98;5.93]	4.85 [3.97;5.93]	4.87 [3.99;5.93]	0.514	
HGB (g/L)	119 [105;131]	119 [105;131]	119 [105;131]	0.574	
PLT (109/L)	159 [125;197]	158 [124;197]	159 [125;197]	0.446	
Gastric protective medicine, n (%)				0.105	
No	19966 (76.7%)	6041 (77.4%)	13,925 (76.4%)		
Yes	6064 (23.3%)	1768 (22.6%)	4296 (23.6%)		
Gastric ulcer, n (%)				0.333	
No	25852 (99.3%)	7762 (99.4%)	18,090 (99.3%)		
Yes	178 (0.68%)	47 (0.60%)	131 (0.72%)		
Cerebral infarction, n (%)				0.796	
No	14560 (55.9%)	4378 (56.1%)	10,182 (55.9%)		
Yes	11470 (44.1%)	3431 (43.9%)	8039 (44.1%)		
Portal hypertension, n (%)				0.295	
No	26021 (100.0%)	7808 (100.0%)	18,213 (100.0%)		
Yes	9 (0.03%)	1 (0.01%)	8 (0.04%)		
Anticoagulants, n (%)				0.180	
No	16777 (64.5%)	5081 (65.1%)	11,696 (64.2%)		
Yes	9253 (35.5%)	2728 (34.9%)	6525 (35.8%)		
Abbreviations: DAPT, dual antiplatelet therapy; DM, Diabetes Mellitus; MI, Myocardial infarction; PCI, Percutaneous Coronary Intervention; WBC, White blood cell count; HGB, hemoglobin; PLT, platelet count.

Selected Predictors and Construction Model

Three variables (previous bleeding, HGB, and cerebral infarction) were selected for inclusion in the model using LASSO regression with ten-fold cross-validation. The pathway of variable shrinkage and cross-validation is illustrated in Figure 2A and B, respectively. The XGBoost model explained feature importance using SHAP, as shown in Figure 3. We performed separate multivariate logistic regression modeling using the five most important indicators, the ten most important indicators from SHAP, and three indicators selected by LASSO regression. After multivariate logistic regression with the “backward” process, several variables were included in the final models (Table 3). Model 4 included four indicators, while Model 6 included six indicators.Table 3 Comparison of Discrimination Between Different Models

Model	Include parameters	AUC (95% CI)	
mod Lasso	Previous.bleeding+HGB+cerebral.infarction	0.836(0.823–0.849	
mod 4	Previous.bleeding+HGB+cerebral.infarction+PLT	0.841(0.829–0.854)	
mod 6	Previous.bleeding+HGB+cerebral.infarction+PLT+gastric.ulcer+tumor	0.842(0.829–0.855)	
Abbreviations: HGB, hemoglobin; PLT, platelet count.

Figure 2 Variable selection was conducted using LASSO regression. (A) Coefficient profile plots were plotted against the log (lambda) sequence to visualize the variable selection process and identify nonzero coefficient variables based on the optimal lambda value. (B) Dotted vertical lines represent optimal values determined using the 1 standard error of the minimum criteria (lambda.1se).

Figure 3 Feature importance analysis using the Shapley additive explanation (SHAP).

Abbreviations: HGB, hemoglobin; WBC, White blood cell count; PLT, platelet count. GPM, Gastric protective medicine; DM, Diabetes Mellitus; Double anti, dual antiplatelet therapy; PCI, Percutaneous Coronary Intervention.

The ROC curves for the LASSO model, Model 4, and Model 6 in the training set are presented in Figure 4. Results show that Model 6 exhibited significantly superior discrimination compared to the other two models based on DeLong’s test (p<0.001). Additionally, a comparison of Model 6 and Model 4 using NRI and IDI metrics indicated that Model 6 had significantly higher values, indicating improved efficacy (Table 4). Finally, a nomogram model was constructed using six variables: HGB, PLT, previous bleeding, gastric ulcer, cerebral infarction, and tumor (Table 5). In the training cohort, our model achieved a sensitivity of 72.1%, specificity of 80.5%, positive predictive value of 18.7%, and negative predictive value of 97.9%.Table 4 Comparison of NRI and IDI Between Two Models

	Model 4	p	
Model 6	NRI (Categorical) [95% CI]
[0,0.5) [0.5,1]	0.0102 [0.0032–0.0173]	0.005	
NRI (Continuous) [95% CI]	0.5333 [0.4782–0.5884]	<0.001	
IDI [95% CI]	0.0032 [0.0013–0.0051]	0.001	

Table 5 Final Model Coefficients

Characteristics	B	SE	OR	CI	p	
(Intercept)	1.392	0.15866	4.021	4.021 (2.944–5.485)	<0.001	
Previous bleeding	3.432	0.14043	30.951	30.95 (23.55–40.86)	<0.001	
HGB	−0.04	0.00151	0.961	0.960 (0.958–0.963)	<0.001	
PLT	−0.004	0.00064	0.996	0.996 (0.995–0.997)	<0.001	
Gastric ulcer	1.021	0.26713	2.775	2.775 (1.613–4.610)	<0.001	
Tumor	−0.48	0.17998	0.619	0.618 (0.429–0.870)	0.008	
Cerebral infarction	0.753	0.07367	2.123	2.122 (1.838–2.454)	<0.001	
Abbreviations: HGB, hemoglobin; PLT, platelet count.

Figure 4 Receiver operating characteristic (ROC) curves of different models for distinguishing bleeding from non-bleeding in the training set.

Model Visualization

The nomogram depicted in Figure 5 provides a visual representation of the logistic regression analysis, allowing for the prediction of bleeding risk in elderly aspirin users. By locating the value of each risk factor on the corresponding vertical line, points can be obtained. The total points are calculated by summing up the points from each risk factor. To determine the bleeding prediction for a specific elderly aspirin user, a vertical line is drawn from the total points axis, intersecting with the corresponding probability on the nomogram. For example, if an elderly aspirin user has no previous bleeding, gastric ulcer, cerebral infarction, or tumor, with platelet counts of 143×109/L and an HGB level of 95 g/L, the total score would be 336. Drawing a vertical line from the total score of 336 intersects the probability axis at approximately 0.049, indicating an estimated bleeding probability of 4.9% (Figure 5).Figure 5 Development of a nomogram based on logistic regression analysis using a combination of six indicators. A total score of 336 corresponds to a bleeding probability of 0.049 (highlighted in red).

Abbreviations: HGB: Hemoglobin; PLT: Platelet count.

Model Validation

Model discrimination was evaluated by calculating the AUC of the ROC curve. In Figure 6A, the training dataset exhibited an AUC of 0.842 (95% CI: 0.829–0.855), while in Figure 6B, the test dataset showed an AUC of 0.820 (95% CI: 0.800–0.840). Calibration curves in Figure 6C and D demonstrated excellent agreement between predicted bleeding probability and actual observations in both the training and test sets. The Hosmer and Lemeshow goodness of fit (GOF) test also indicated good consistency. Figure 7 displayed DCA, CIC, and NRC results for the developed model. DCA confirmed a favorable net benefit in predicting bleeding risk among elderly aspirin users. The threshold probability ranged from 5.0–67% in the training dataset (Figure 7A) and 5.0–68% in the test dataset (Figure 7B). Lower risk thresholds corresponded to higher net benefits. However, CIC analysis revealed that decreasing the risk threshold increased false positive rates and unnecessary interventions (Figure 7C and D). Therefore, considering both DCA and CIC results is crucial for making optimal decisions, striking a balance between high net benefit and low false positive rate. NRC plots in Figure 7E and F demonstrated a good fit for the model in both the training and validation sets.Figure 6 ROC and Calibration curves of the nomogram. (A) ROC curves in the training set; (B) in the validation set. (C) Calibration curves in the training set; (D) in the validation set. Calibration curves demonstrate the alignment between predicted bleeding risk (x-axis) and actual diagnosed cases (y-axis). The red line represents the performance of the training set (C) and validation set (D), while diagonal lines indicate ideal predictions. Closer alignment with the diagonal lines indicates better prediction performance.

Figure 7 Decision curve analysis (DCA), clinical impact curves (CIC) and the net reduction curves (NRC) of the nomogram. (A) DCA in the training set; (B) DCA in the validation set; (C) CIC in the training set; (D) CIC in the validation set; (E) NRC in the training set; (F) NRC in the validation set. In DCAs, the y-axis represents net benefit. The horizontal lines labeled “None” assume no participants experienced bleeding, while “All” assumes all participants had bleeding. The blue line represents our predictive model developed in this study. In CICs, the red curve indicates the number of individuals classified as positive (high risk) by the model at each threshold probability, representing the number of high-risk individuals. The blue curve represents the number of true positives (individuals with the outcome) at each threshold probability. In NRCs, the y-axis values represent the number of patients that could be reduced under the same effect size by utilizing a specific threshold probability of diagnosis, indicated by the x-axis value.

Model Compare with Single Indicator

We compared the discriminative ability of our constructed model (nomogram) with that of a single indicator. Figure 8 clearly shows that our model outperforms a single indicator in terms of discriminative ability.Figure 8 Comparison between nomogram and individual indicators.

Abbreviations: PB, Previous bleeding; HGB, hemoglobin; GU, Gastric ulcer; PLT, platelet count.

Discussion

In this study, we developed and validated a model for evaluating bleeding risk in elderly aspirin users. The final model included six indicators: HGB, PLT, previous bleeding, gastric ulcer, cerebral infarction, and tumor. Our model demonstrated strong discrimination, calibration, and net clinical benefit. The derived nomogram visually represents the logistic regression analysis results, providing clinicians with a tool to estimate bleeding risk in elderly aspirin users.

Several factors have been identified as influential in bleeding among aspirin users, including HGB,25 PLT,26 previous bleeding,25 cerebral infarction,27 gastric ulcer,28 tumor.29 With the aim of incorporating comprehensive information, our risk models for predicting bleeding considered these factors. Previous studies have highlighted the significance of decreased HGB levels as a strong predictor of major bleeding,30,31 and our study reaffirmed this as a critical indicator for assessing bleeding risk. Additionally, we identified PLT levels as another significant predictor, where lower levels indicated higher bleeding risk and poorer prognosis.32 Our predictive model aligned with findings from previous studies. Previous bleeding has long been recognized as a crucial factor in guiding treatment plans for aspirin users,25 and our risk model revealed that it increases bleeding risk in elderly aspirin users. Artery occlusion cerebral infarction was found to be associated with an elevated risk of hemorrhage transformation,33 consistent with our results and potentially linked to increased antithrombotic medication usage. Furthermore, gastric ulcer was independently associated with bleeding risk in elderly aspirin users in our risk model, aligning with prior research.34 In addition to the mentioned factors, tumor has also been recognized as a potential predictor of bleeding,35 Interestingly, our findings indicated that tumor is a significant factor that reduces the risk of bleeding, contrary to previous research findings,36 This discrepancy may be attributed to the impact of retrospective study bias and the absence of bleeding records in hospital EMRs due to premature death of tumor patients.

Tailored management strategies are crucial for elderly aspirin users with varying bleeding risks. Various bleeding scoring systems, such as the Glasgow Blatchford Score (GBS),37 TIMI score,16 CRUSADE score,12 and HAS-BLED score,17 are widely used in clinical practice but exhibit limitations due to their semi-quantitative nature and specific patient tailoring. In contrast, our model employs a quantitative approach, facilitating a more precise assessment of bleeding risk. This methodology enhances risk calculation accuracy compared to the subjective existing systems. Additionally, our model incorporates a comprehensive set of clinical indicators, capturing a broader range of patient-specific factors, which leads to a more thorough evaluation of bleeding risk. The model’s high AUC of 84% demonstrates its robust predictive performance, indicating superior accuracy and effective discrimination among patients at varying bleeding risk levels, thus surpassing the limitations of current scoring systems. Our predictive model, developed using machine learning techniques like LASSO regression and XGBoost, incorporates key variables such as HGB, PLT, previous bleeding, and cerebral infarction. The model’s output is visualized through a user-friendly nomogram, enabling clinicians to estimate individual bleeding risk and make personalized treatment decisions. Evaluation metrics, including clinical decision curve, clinical impact curve, and net reduction curve analyses, demonstrate a high net clinical benefit, suggesting potential improvements in patient outcomes and reduced healthcare costs. Considering aspirin’s critical role as an antiplatelet agent in coronary disease management, the model’s capacity to accurately assess bleeding risk is essential for optimizing its use. By identifying patients at elevated bleeding risk, the model facilitates a nuanced approach to aspirin therapy, empowering clinicians to make informed decisions about its initiation, continuation, or alternative treatments. For high-risk patients, we advocate for a personalized antiplatelet therapy strategy guided by the model’s recommendations.

However, it is important to consider the limitations of our study. Firstly, in reflecting on the historical nature of our sampled cohort, it is important to acknowledge the significant advancements in diagnostic and therapeutic techniques that have emerged since the data was collected. Innovations such as MRI, endoscopy, and various diagnostic immunoassays, along with improvements in pharmacological treatments, may influence the applicability of our findings to current clinical practice. This highlights the need for further research to explore how these advancements impact patient outcomes and treatment protocols. Secondly, our reliance on electronic medical records (EMRs) dating back to 2008 presents certain limitations. The accuracy and completeness of data captured in EMRs may be affected by the quality of manual transcription from historical medical records. As such, there may be gaps or inconsistencies in the data that could impact our study’s conclusions. Additionally, our study focused specifically on elderly aspirin users, and further investigation is required to generalize our findings to other patient populations.

Conclusion

We developed and validated a model for evaluating bleeding risk in elderly aspirin users. The final model included six indicators: HGB, PLT, previous bleeding, gastric ulcer, cerebral infarction, and tumor. Our model offers valuable insights for managing bleeding in elderly aspirin users, aiming to optimize treatment planning and improve patient outcomes. Future research should focus on external validation across diverse patient cohorts and explore additional variables and machine learning algorithms to enhance predictive accuracy.

Data Availability Statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics Approval

This study received approval from the Medical Ethics Committee of the Affiliated Dongyang Hospital of Wenzhou Medical University (approval #2023-YX-408). Informed consent was waived, and patient records/information were anonymized and deidentified before analysis.

Disclosure

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
==== Refs
References

1. Costa F, van Klaveren D, James S, et al. Derivation and validation of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy (PRECISE-DAPT) score: a pooled analysis of individual-patient datasets from clinical trials. Lancet. 2017;389 (10073 ):1025–1034. doi:10.1016/s0140-6736(17)30397-5 28290994
2. Yeh JS, Chen WT, Tomlinson B, Tam WC, Chien LN. Comparing the effectiveness and safety of dual antiplatelet with ticagrelor or clopidogrel in elderly Asian patients with acute myocardial infraction. Front Cardiovasc Med. 2023;10 :1143509. doi:10.3389/fcvm.2023.1143509 37008324
3. Lv M, Zheng X, Wu T, et al. A new score for predicting acute gastrointestinal bleeding in patients administered oral antiplatelet drugs. Front Pharmacol. 2020;11 :571605. doi:10.3389/fphar.2020.571605 33519436
4. Cheng Y, Lin J, Liu J, Wang Y, Yan W, Zhang M. Decreased vascular endothelial growth factor expression is associated with cell apoptosis in low-dose aspirin-induced gastric mucosal injury. Am J Med Sci. 2015;349 (2 ):110–116. doi:10.1097/maj.0000000000000409 25607509
5. McNeil JJ, Woods RL, Nelson MR, et al. Effect of aspirin on disability-free survival in the healthy elderly. N Engl J Med. 2018;379 (16 ):1499–1508. doi:10.1056/NEJMoa1800722 30221596
6. Mihatov N, Kirtane AJ, Stoler R, et al. Bleeding and ischemic risk prediction in patients with high bleeding risk (an EVOLVE Short DAPT Analysis). Am J Cardiol. 2023;207 :370–379. doi:10.1016/j.amjcard.2023.06.036 37778226
7. Celik A, Esin FK, Emren SV, Kiris T, Karaca M. The clinical value of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy score in predicting long-term major adverse cardiovascular and cerebrovascular events and major bleeding in acute coronary syndrome patients who underwent percutaneous coronary intervention. Angiology. 2023;74 (10 ):958–969. doi:10.1177/00033197221127073 36113050
8. Zhao X, Li J, Liu F, et al. The PRECISE-DAPT score and 5-year outcomes after percutaneous coronary intervention: a large-scale, real-world study from China. Eur Heart J Qual Care Clin Outcomes. 2022;8 (8 ):812–820. doi:10.1093/ehjqcco/qcab068 34558620
9. Capodanno D, Alfonso F, Levine GN, Valgimigli M, Angiolillo DJ. ACC/AHA Versus ESC guidelines on dual antiplatelet therapy: JACC guideline comparison. J Am Coll Cardiol. 2018;72 (23 Pt A ):2915–2931. doi:10.1016/j.jacc.2018.09.057 30522654
10. Mihatov N, Secemsky EA, Kereiakes DJ, et al. Utility of the dual antiplatelet therapy score to guide antiplatelet therapy: a systematic review and meta-analysis. Catheter Cardiovasc Interv. 2021;97 (4 ):569–578. doi:10.1002/ccd.29352 33111495
11. Vries MJ, van der Meijden PE, Henskens YM, ten Cate-Hoek AJ, ten Cate H. Assessment of bleeding risk in patients with coronary artery disease on dual antiplatelet therapy. A systematic review. Thromb Haemost. 2016;115 (1 ):7–24. doi:10.1160/th15-04-0355 26272731
12. Choi SY, Kim MH, Serebruany V. Comparison of ACUITY, CRUSADE, and GRACE risk scales for predicting clinical outcomes in patients treated with dual-antiplatelet therapy. TH Open. 2018;2 (4 ):e399–e406. doi:10.1055/s-0038-1675576 31249967
13. Kadiyala V, Long S, Has P, et al. PRECISE-DAPT and ARC-HBR predict in-hospital outcomes in patients who underwent percutaneous coronary intervention. Am J Cardiol. 2023;191 :43–50. doi:10.1016/j.amjcard.2022.12.004 36640599
14. Sim DS, Jeong MH, Kim HS, et al. Utility of GRACE and ACUITY-HORIZONS risk scores to guide dual antiplatelet therapy in Korean patients with acute myocardial infarction undergoing drug-eluting stenting. J Cardiol. 2018;72 (5 ):411–419. doi:10.1016/j.jjcc.2018.04.006 29764713
15. Grodecki K, Huczek Z, Scisło P, et al. Gender-related differences in post-discharge bleeding among patients with acute coronary syndrome on dual antiplatelet therapy: a BleeMACS sub-study. Thromb Res. 2018;168 :156–163. doi:10.1016/j.thromres.2018.06.022 30064688
16. Bonaca MP, Bhatt DL, Cohen M, et al. Long-term use of ticagrelor in patients with prior myocardial infarction. N Engl J Med. 2015;372 (19 ):1791–1800. doi:10.1056/NEJMoa1500857 25773268
17. Costa F, Tijssen JG, Ariotti S, et al. Incremental value of the CRUSADE, ACUITY, and HAS-BLED risk scores for the prediction of hemorrhagic events after coronary stent implantation in patients undergoing long or short duration of dual antiplatelet therapy. J Am Heart Assoc. 2015;4 (12 ). doi:10.1161/jaha.115.002524
18. Capodanno D, Rossini R, Musumeci G, et al. Predictive accuracy of CHA2DS2-VASc and HAS-BLED scores in patients without atrial fibrillation undergoing percutaneous coronary intervention and discharged on dual antiplatelet therapy. Int J Cardiol. 2015;199 :319–325. doi:10.1016/j.ijcard.2015.07.064 26241637
19. Brinza C, Burlacu A, Tinica G, Covic A, Macovei L. A systematic review on bleeding risk scores’ accuracy after percutaneous coronary interventions in acute and elective settings. Healthcare. 2021;9 (2 ):148. doi:10.3390/healthcare9020148 33540514
20. Li B, Eisenberg N, Beaton D, et al. Predicting inferior vena cava filter complications using machine learning. J Vasc Surg Venous Lymphat Disord. 2024:101943. doi:10.1016/j.jvsv.2024.101943 39084408
21. Liang C, Wanling L, Maofeng W. LASSO-derived model for the prediction of bleeding in aspirin users. Sci Rep. 2024;14 (1 ):12507. doi:10.1038/s41598-024-63437-6 38822153
22. Tang M, Wang S, Wang Y, et al. Development and validation of a nomogram predicting postoperative recurrent lumbar disc herniation based on activity factors. Risk Manag Healthc Policy. 2024;17 :689–699. doi:10.2147/rmhp.S453819 38544530
23. Wang Q, Jiang XX, Que YZ, Wan XY, Lin CQ. Development and validation of a risk prediction model for female stress urinary incontinence in Rural Fujian, China. Risk Manag Healthc Policy. 2024;17 :1101–1112. doi:10.2147/rmhp.S457332 38707519
24. Yan X, Xia P, Tong H, et al. Development and validation of a dynamic nomogram for predicting 3-month mortality in acute ischemic stroke patients with atrial fibrillation. Risk Manag Healthc Policy. 2024;17 :145–158. doi:10.2147/rmhp.S442353 38250220
25. Kamran H, Jneid H, Kayani WT, et al. Oral antiplatelet therapy after acute coronary syndrome: a review. JAMA. 2021;325 (15 ):1545–1555. doi:10.1001/jama.2021.0716 33877270
26. Morici N, Tavecchia GA, Antolini L, et al. Use of PRECISE-DAPT score and admission platelet count to predict mortality risk in patients with acute coronary syndrome. Angiology. 2019;70 (9 ):867–877. doi:10.1177/0003319719848547 31088127
27. Huang J, Liao F, Tang J, Shu X. Risk factors for gastrointestinal bleeding in patients with cerebral infarction after dual antiplatelet therapy. Clin Neurol Neurosurg. 2023;231 :107802. doi:10.1016/j.clineuro.2023.107802 37295199
28. Zhang Y, Fang XM, Chen GX. Clinical use of low-dose aspirin for elders and sensitive subjects. World J Clin Cases. 2019;7 (20 ):3168–3174. doi:10.12998/wjcc.v7.i20.3168 31667166
29. Hirai R, Kawano S, Inoo S, et al. Postoperative bleeding risk after gastric endoscopic submucosal dissection in patients receiving a P2Y12 receptor antagonist. Gut Liver. 2023;17 (3 ):404–411. doi:10.5009/gnl220196 36172713
30. Ng AK, Ng PY, Ip A, Ling IW, Lam LT, Siu CW. Incidence, prediction, and outcomes of major bleeding after percutaneous coronary intervention in Chinese patients. JACC Asia. 2022;2 (3 ):341–350. doi:10.1016/j.jacasi.2021.12.009 36338416
31. Chilbert MR, Reidy SE, Clark CM, Guszkowski M, Gargala E, Woodruff AE. Comparison of bleeding rates between oral anticoagulants in combination with dual antiplatelet therapy (Triple Therapy) in a real-world cohort. Hosp Pharm. 2022;57 (2 ):253–259. doi:10.1177/00185787211024602 35601720
32. Lambert C, Maitland H, Ghanima W. Risk-based and individualised management of bleeding and thrombotic events in adults with primary immune thrombocytopenia (ITP). Eur J Haematol. 2023;112 (4 ):504–515. doi:10.1111/ejh.14154 38088207
33. Kumazawa R, Jo T, Matsui H, Fushimi K, Yasunaga H. Direct oral anticoagulants versus warfarin for secondary prevention of cerebral infarction and bleeding in older adults with atrial fibrillation. J Am Geriatr Soc. 2022;70 (7 ):2029–2039. doi:10.1111/jgs.17770 35397114
34. Palmer K. Acute upper gastrointestinal haemorrhage. Br Med Bull. 2007;83 (1 ):307–324. doi:10.1093/bmb/ldm023 17942452
35. Lucà F, Parrini I, Abrignani MG, et al. Management of acute coronary syndrome in cancer patients: it’s high time we dealt with it. J Clin Med. 2022;11 (7 ):1792. doi:10.3390/jcm11071792 35407399
36. Kwok CS, Wong CW, Kontopantelis E, et al. Percutaneous coronary intervention in patients with cancer and readmissions within 90 days for acute myocardial infarction and bleeding in the USA. Eur Heart J. 2021;42 (10 ):1019–1034. doi:10.1093/eurheartj/ehaa1032 33681960
37. Di Gioia G, Sangineto M, Paglia A, et al. Limits of pre-endoscopic scoring systems in geriatric patients with upper gastrointestinal bleeding. Sci Rep. 2024;14 (1 ):20225. doi:10.1038/s41598-024-70577-2 39215015
