
==== Front
Heliyon
Heliyon
Heliyon
2405-8440
Elsevier

S2405-8440(24)12846-0
10.1016/j.heliyon.2024.e36815
e36815
Research Article
An interpretable machine learning method for risk stratification of patients with acute coronary syndrome
Zhu Xing-Yu ab
Zhang Kai-Jie a
Li Xiao a
Su Fei-Fei b
Tian Jian-Wei tianjianwei029@aliyun.com
b⁎
a Graduate School of Hebei North University, Zhangjiakou, 075031, Hebei Province, China
b Department of Cardiovascular Medicine, Air Force Medical Center, Chinese People's Liberation Army, Beijing, 100142, Beijing, China
⁎ Corresponding author. tianjianwei029@aliyun.com
23 8 2024
15 9 2024
23 8 2024
10 17 e368156 4 2024
22 8 2024
22 8 2024
© 2024 The Authors
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Abstracts

Backgrounds

Risk stratification for major adverse cardiovascular events (MACE) within one year in patients with acute coronary syndrome (ACS) undergoing percutaneous coronary intervention (PCI) remains a challenge. Although several predictive models based on machine learning have emerged, they are difficult to understand. This study aimed to develop a machine learning prediction model that is easy to understand and trustworthy by lay people to assess the risk of MACE in ACS patients undergoing PCI within one year of the procedure.

Methods

This retrospective cohort study used medical data from 1105 patients to construct a machine-learning model. To ensure thoroughness and multidimensionality of model parsing, Shapley Additive explanations (SHAP) analysis and Local interpretable model-agnostic explanations (LIME) interpretation techniques were used to systematically and deeply interpret the constructed models from a global to a detailed level.

Results

The study assessed 12 machine learning methods' prediction models and found that the Random Forest model was the most effective in predicting the risk of MACE in ACS patients after undergoing PCI. The model achieved an AUC value of 0.807 in the validation set, with an accuracy of 0.82, and a stable F1 score of 0.51. SHAP analysis ranked eight key feature variables, such as LVEF, in global importance. The weights of each feature range in the prediction model were revealed using LIME analysis.

Conclusion

The machine learning prediction model we developed is capable of accurately predicting the likelihood of patients with ACS experiencing a MACE within one year of surgery.

Graphical abstract

Abstract Map: ACS: Acute Coronary Syndrome; PCI: Percutaneous Coronary Intervention; MACE: Major adverse cardiovascular events; RF: Random Forest; SVM: Support Vector Machine; GBM: Gradient Boosting Machine; XGB: Extreme Gradient Boosting; KNN: K-Nearest Neighbors.By Figdraw.Image 1
==== Body
pmc1 Introduction

Over the past 30 years, the prevalence and mortality of patients with ACS have continued to rise in China [1]. In the United States, there are 605,000 new cases of acute myocardial infarction each year, as well as 200,000 recurrent cases [2]. Each year, over 7 million people worldwide are diagnosed with ACS [3]. Cardiovascular disease remains the leading cause of death worldwide, with nearly half of all deaths caused by ischemic heart disease, despite significant breakthroughs in the diagnosis and treatment of ACS [4,5]. The incidence of MACE in patients with ACS who undergo PCI not only increases financial pressure on patients but also places a significant psychological burden on them [6]. Accurate prediction of MACE is crucial in making medical decisions and selecting treatment options after PCI.

Existing risk prediction models often rely on traditional univariate or multivariate statistical regression analyses. However, these methods can struggle to effectively extract risk identifiers when dealing with high-dimensional data [7,8]. However, recent developments in machine learning (ML) technology have made it possible to build risk prediction models without prior assumptions about which imaging or clinical markers have predictive value [9,10]. These machine learning algorithms have been meticulously designed and optimized to learn from large and complex high-dimensional datasets, enabling them to uncover profound connections between patient clinical variables and outcomes. However, using advanced machine learning models in clinical practice is still somewhat restricted, primarily due to the absence of intuitive and explicit interpretations of their decision-making process [11]. When building algorithmic models, we typically evaluate the model's effectiveness by measuring the precision and recall of the test set. However, when communicating with non-professionals, relying solely on a single number may not be sufficient to establish credibility. Therefore, it is important to demonstrate the internal logic and principles of the model to enhance its credibility. It is worth noting that not all models are based on explicit rules. Although certain black-box models may have higher prediction accuracy, they can be difficult for the general public to understand and trust due to their opacity, which makes it challenging to provide a concrete decision-making basis. Fortunately, several tools have been introduced recently that can reveal the decision logic of machine learning models and effectively identify critical predictors [12,13]. With these tools, we can better understand the working mechanisms of ML models. This will pave the way for their wider application in clinical practice.

This exploration aims to construct a machine learning model with high explanatory power to predict the potential risk of MACE in patients with ACS after undergoing PCI. In addition, we utilized two advanced technological tools, SHAP and LIME, to provide a comprehensive analysis and exploration of our constructed machine-learning prediction models at both the holistic and local levels. SHAP is a widely used and versatile tool for assessing the influence of features in machine learning models. It is a method of model hindsight interpretation [14,15]. The main objective is to measure the impact of features on the model's final output, providing a comprehensive analysis of the ‘black box model’ from both a holistic and individual perspective [16]. The LIME algorithm approximates any classifier or regressor with an interpretable model that accurately interprets its predictions.

2 Methods

2.1 Study population

This is a comprehensive analysis of prognostic data for patients with ACS who underwent PCI at the Chinese People's Liberation Army Air Force Medical Centre (Abstract Map). The study adhered to the principles outlined in the Declaration of Helsinki and was formally approved by the Ethical Review Committee of the PLA Air Force Centre for Specialty Medicine. This retrospective study included consecutive patients with ACS admitted between October 2019 and January 2023. The patients' comprehensive data were extracted from a medical database dedicated to the Department of Cardiovascular Medicine. All patients were meticulously categorized based on the occurrence of MACE. This study defines MACE as encompassing a wide range of scenarios, including the recurrence of angina pectoris, severe arrhythmias, re-hospitalization for cardiovascular problems, episodes of acute myocardial infarction, the onset of heart failure, and death due to coronary artery disease. Comprehensive profiling of patient demographic data, comorbidity status, angiographic findings, procedural details, in-hospital occurrences, and post-discharge follow-up data was conducted. To ensure the rigor of the study and the accuracy of the data, 54 patients with hematological disorders, malignancies, and incomplete follow-up data were strictly excluded. The study's flow chart is presented in Fig. 1.Fig. 1 Flow chart.

Fig. 1

Handling of missing data: In clinical trials, data loss is not uncommon due to various reasons. This absence of data can compromise the integrity of the original dataset, thereby weakening the robustness and validity of the study's conclusions. Therefore, we used the KNN interpolation method to handle missing data. We employed the ranging technique to screen k samples with spatial similarity or proximity in the dataset. These samples were then used to estimate the values of the missing data points. For the missing values in each sample, we used the mean value of the neighborhood found in the dataset to fill in and ensure the completeness and accuracy of the data. The study found that all selected variables had less than 25 % missing data. To fill in the missing data, the ‘DMwR2′ package in R4.2.3 was used, assuming that the missing data was random. The term “missing at random” is used to describe a situation in which the probability of a variable being missing is solely dependent on the observed values within a given dataset, and is not influenced by the unobserved variables. To illustrate, a patient's weight may be correlated with other factors, such as age, disease status, etc., but is not influenced by missing weight values.

Statistical analyses: Statistical analyses were conducted using R version 4.2.3 software. A normality test was conducted during the preliminary stage of data analysis. To describe continuous variables that follow a normal distribution, we used the mean and standard deviation (x‾ ±s). We compared parameter values between groups using independent samples t-tests. For continuous variables that do not conform to a normal distribution, we used the median (M (Q1, Q3)) to express them in detail and applied the Mann-Whitney U test to compare them. For categorical variables, we presented the data as percentages and used the chi-squared test to explore differences.

Predictive model development and validation: Recursive Feature Elimination (RFE) is a model-based feature filtering strategy. It iteratively trains the model and gradually eliminates the features with the lowest weights to filter out the best feature combinations. The algorithm was used to identify 8 features that significantly predict the occurrence of MACE events in ACS patients undergoing PCI. Redundant and noisy features were effectively excluded. This study employs twelve machine learning models, including Logistic Regression, Neural Network, Support Vector Machine, Multilayer Perceptron, Gaussian Process, Gradient Boosting Machine, Naive Bayes, Adaptive Boosting, Extreme Gradient Boosting, Random Forest, Decision Tree, and K-Nearest Neighbors, to develop prediction models. To construct the prediction model, we employed a stochastic approach to finely partition the patient data within the dataset into a training set and a validation set in a 7:3 ratio. We ensured the accuracy of parameter tuning and model robustness by implementing a tenfold cross-validation technique for network search and parameter optimization. The metric used to assess the predictive efficacy of the models was the area under the working characteristic curve of the subjects. Additionally, we calculated sensitivity, specificity, positive predictive value, negative predictive value, recall, precision, and F1 score for each model. To demonstrate the effectiveness of the optimal model, we conducted a decision curve analysis (DCA) [17].

2.2 Interpretive tools for machine learning

SHAP is a widely used and versatile tool for assessing the influence of features in machine learning models. It is a method of model hindsight interpretation [14,15]. The main objective is to measure the impact of features on the model's final output, providing a comprehensive analysis of the ‘black box model’ from both a holistic and individual perspective [16]. The SHAP framework provides a clear explanation of how each feature contributes to the target variable, using the term 'contributors'. SHAP values reveal the exact positive or negative impact of each predictor [16]. This model is easy to understand and provides insight into the reasoning behind the predictions. LIME provides interpretations for individual samples by training local agent models. To interpret black-box models, we select example samples and perturb them within their neighborhoods to generate new data points. We then obtain predicted values for these new data points from the black-box model. Finally, we use this new data to train an interpretable model that accurately approximates the black-box model. To enhance our understanding of our risk prediction model, we used SHAP and LIME to carefully examine the impact of various clinical variables. The model generates a predictive value for each sample, and the SHAP value accurately reflects the numerical contribution of each feature within that sample. LIME also provides the contribution of the range of feature variables in the predictive model.

3 Result

3.1 Patient characteristics

The study included 1105 patients who were randomly divided into two clusters. 70 % (777) of the data was used to construct the predictive model, while the remaining 30 % (328) was used for validation. The study divided ACS patients into two groups: those who experienced MACE within one year (MACE group) and those who did not (non-MACE group). Table 1 presents the baseline characteristics of these groups in the training set. To improve the accuracy of predicting MACE events in ACS patients after PCI, this study used a recursive feature elimination algorithm to select eight influential feature metrics. These metrics were considered the core elements for constructing the model. The eight key features that are specifically included are left ventricular ejection fraction (LVEF), albumin, myoglobin, high-sensitivity C-reactive protein (hs-CRP), age, creatinine, D-dimer, and B-type natriuretic peptide (BNP). Table 1 presents data showing that patients with high levels of myoglobin, hsCRP, age, creatinine, D-dimer, and BNP are at a significantly increased risk of developing MACE. In contrast, patients with ACS had reduced LVEF and albumin levels, which also increased their risk of MACE.Table 1 Baseline characteristics of the training set.

Table 1Variables	All Patients (N = 777)	Non-MACE (N = 579)	MACE (N = 198)	P values	
Unstable angina, n (%)	557(72)	438(76)	119(60)	<0.001	
NSTEMI, n (%)	71(9)	46(8)	25(13)	0.067	
STEMI, n (%)	149(19)	95(16)	57(27)	0.001	
Dyslipidemia, n (%)	460(59)	361(62)	99(50)	0.003	
Hypertension, n (%)	510(66)	373(64)	137(69)	0.257	
Smoking, n (%)	146(19)	106(18)	40(20)	0.629	
Diabetes mellitus, n (%)	286(37)	213(37)	73(37)	1	
Female, n (%)	600(77)	453(78)	147(74)	0.29	
Age, Mean ± SD	63.63 ± 11.3	62.4 ± 10.67	67.23 ± 12.57	<0.001	
Heart. Rate, M (Q1, Q3)	76(68,85)	76(68,84)	78(69,86.75)	0.108	
SBP, M (Q1, Q3)	129(118,140)	129(119,141)	126.5(115,139.75)	0.051	
DBP, M (Q1, Q3)	76(68,85)	77(69,85)	76(68,84)	0.315	
Height, M (Q1, Q3)	1.7(1.64,1.74)	1.7(1.65,1.74)	1.69(1.63,1.73)	0.051	
Weight, M (Q1, Q3)	72.5(65,80)	73(65.25,80)	72(63.25,79)	0.147	
BMI, M (Q1, Q3)	25.59(23.53,27.51)	25.54(23.62,27.44)	25.66(22.9,27.62)	0.678	
leucocyte, [ × 109/L, M (Q1, Q3)]	6.77(5.56,8.59)	6.66(5.5,8.31)	7(5.68,9.4)	0.031	
neutrophil, [ × 109/L, M (Q1, Q3)]	4.36(3.35,5.9)	4.26(3.3,5.7)	4.54(3.6,7)	0.003	
lymphocyte, [ × 109/L, M (Q1, Q3)]	1.6(1.2,2.05)	1.64(1.28,2.1)	1.5(1.09,1.91)	0.002	
monocyte, [ × 109/L, M (Q1, Q3)]	0.45(0.37,0.6)	0.44(0.37,0.59)	0.5(0.37,0.62)	0.047	
hemoglobin, [g/L, M (Q1, Q3)]	139(126,149)	140(128,149)	135(119.25,147)	<0.001	
Blood platelet, [ × 109/L, M(Q1,Q3)]	210(176,245)	208(177,243)	215(160.5250.25)	0.731	
Hs-CRP, [mg/L, M (Q1, Q3)]	2.38(0.6,6.29)	2.06(0.56,4.95)	4.31(0.93,13.92)	<0.001	
Prothrombin time, [sec, M (Q1, Q3)]	11.3(10.8,11.8)	11.23(10.8,11.8)	11.4(10.8,12.07)	0.042	
APTT, [sec, M (Q1, Q3)]	31.5(29.5,33.4)	31.5(29.7,33.4)	31.45(29.13,33.4)	0.61	
fibrinogen, [g/L, M (Q1, Q3)]	3.19(2.81,3.71)	3.14(2.8,3.66)	3.33(2.87,3.94)	0.007	
D-Dimer, [ng/mL, M (Q1, Q3)]	89(56,160)	78.86(52,132)	148(79,233.86)	<0.001	
Potassium, [mmol/L, M (Q1, Q3)]	4(3.8,4.2)	4(3.8,4.2)	4(3.8,4.3)	0.249	
Sodium, [mmol/L, M (Q1, Q3)]	140(138,142)	140(138.4142)	140(138,142)	0.299	
AST, [U/L, M (Q1, Q3)]	22(17.5,31.9)	22(17.75,32)	22.25(17,31)	0.969	
ALT, [U/L, M (Q1, Q3)]	21(15,33)	22(16,34)	19.8(13,30)	0.01	
creatinine, [μmol/L,M(Q1,Q3)]	74(63,86)	73(63,83)	79(65.17,96.53)	<0.001	
Uric acid,[μmol/L,M(Q1,Q3)]	340(286,405)	334(283,394.44)	363.73(300.5435.25)	<0.001	
albumin, [g/L, M (Q1, Q3)]	42.6(40.1,44.7)	43.26(40.8,45.1)	40.6(38.7,43.39)	<0.001	
myoglobin, [ng/mL, M (Q1, Q3)]	53.7(37,88)	50.46(35,71)	75.5(45.2186)	<0.001	
CK-MB, [ng/mL, M (Q1, Q3)]	2.1(2,5.8)	2(2,3.6)	3.15(2,15.75)	<0.001	
Troponin, [ng/mL, M (Q1, Q3)]	0.02(0.01,0.83)	0.01(0.01,0.42)	0.07(0.01,1.3)	<0.001	
PNI, M (Q1, Q3)	51(47.25,54.3)	51.55(48.15,54.8)	48.48(45.06,52.14)	<0.001	
Total cholesterol, [mmol/L, M (Q1, Q3)]	3.95(3.37,4.75)	3.97(3.41,4.75)	3.91(3.26,4.68)	0.361	
triglyceride, [mmol/L, M (Q1, Q3)]	1.43(1.07,2.04)	1.44(1.08,2.02)	1.41(1.02,2.06)	0.393	
HDL-C, [mmol/L, M (Q1, Q3)]	1.02(0.89,1.2)	1.02(0.89,1.2)	1.01(0.87,1.18)	0.261	
LDL-C, [mmol/L, M (Q1, Q3)]	2.19(1.72,2.86)	2.2(1.73,2.84)	2.1(1.7,2.89)	0.709	
BNP, M (Q1, Q3)	61.1(25.9195.9)	48.73(22.4121.45)	139.3(55.45,447.15)	<0.001	
LVEF, M (Q1, Q3)	59(55,62)	59(57,62)	55(49.25,59)	<0.001	
GRACE, M (Q1, Q3)	88(73,104)	83(70.5100)	101(83,118)	<0.001	
MACE: major adverse cardiovascular event; NSTEMI: Non-ST-segment elevation myocardial infarction; STEMI: ST-segment elevation myocardial infarction; SBP: systolic blood pressure; DBP: diastolic blood pressure; BMI: Body mass index; Hs-CRP: hypersensitive C-reactive protein; APTT: Activated partial thromboplastin time; AST: Aspartate Transaminase; ALT: Alanine Aminotransferase; CK-MB: creatine kinase isoenzymes; PNI: Prognostic Nutritional Index; HDL-C: high-density lipoprotein-cholesterol; LDL-C: low-density lipoprotein-cholesterol, LVEF: left ventricular ejection fraction; GRACE: Global registry of acute coronary events.

3.2 Model performance and interpretation

We created twelve machine learning models for the training dataset, such as logistic regression, neural networks, support vector machines, multilayer perceptrons, Gaussian processes, gradient boosters, plain Bayes, adaptive augmentation, extreme gradient boosting, random forests, decision trees, and K-nearest neighbor algorithms. Table 2 and Fig. 2 detail the key metrics for these models on the validation dataset, including sensitivity, specificity, positive predictive value, negative predictive value, recall, F1 score, AUC value, and precision. After a thorough comparison, the random forest model showed the best prediction performance, with an AUC value of 0.807, an accuracy of 0.82, and an F1 score of 0.51, making it the top performer in this evaluation. For the test dataset, we plotted the DCA curve of the RF machine learning model to explore in depth the net gain performance of the model. In the clinical context, net gain is defined as the minimum probability of a patient's disease occurring when further medical action is required [18]. Fig. 3 illustrates three lines: the orange line, the black line, and the blue line. The orange line represents the extreme case where no treatment is applied to any individuals, resulting in a net benefit of zero. The black line represents the opposite extreme, where all individuals are treated, and the net benefit decreases as the threshold probability increases. The blue line represents the change in net benefit of our decision model for different threshold probabilities. If the blue line is close to the orange and black reference lines, it indicates that the model lacks practical application value. Conversely, if the blue line is consistently higher than the reference lines across a wide range of threshold intervals, it proves that the model has a better net benefit. Fig. 3 shows the construction of a machine learning forecasting model with good net returns.Table 2 Predictive power of twelve machine learning models.

Table 2CLS	Sensitivity (95 % CI)	Specificity (95 % CI)	Pos Pred Value (95 % CI)	Neg Pred Value (95 % CI)	Recall	F1	AUC	Accuracy (95 % CI)	
LR	0.36 (0.25, 0.48)	0.95 (0.91, 0.97)	0.67 (0.50, 0.81)	0.84 (0.79, 0.88)	0.36	0.47	0.772	0.82(0.77,0.86)	
NN	0.33 (0.23, 0.45)	0.95 (0.92, 0.98)	0.67 (0.49, 0.81)	0.84 (0.79, 0.88)	0.33	0.44	0.762	0.82(0.77,0.85)	
SVM	0.28 (0.18, 0.40)	0.95 (0.91, 0.97)	0.61 (0.42, 0.77)	0.82 (0.78, 0.87)	0.30	0.418	0.705	0.80(0.75,0.84)	
MLP	0.51 (0.39, 0.63)	0.88 (0.83, 0.92)	0.54 (0.42, 0.67)	0.87 (0.82, 0.90)	0.51	0.52	0.764	0.80(0.75,0.84)	
GP	0.33 (0.23, 0.45)	0.96 (0.93, 0.98)	0.71 (0.53, 0.85)	0.84 (0.79, 0.88)	0.34	0.46	0.771	0.82(0.78,0.86)	
GBM	0.43 (0.31, 0.55)	0.90 (0.86, 0.94)	0.55 (0.41, 0.69)	0.85 (0.80, 0.89)	0.43	0.48	0.766	0.80(0.75,0.84)	
NB	0.43 (0.31, 0.55)	0.88 (0.84, 0.92)	0.51 (0.38, 0.64)	0.85 (0.80, 0.89)	0.43	0.46	0.750	0.78(0.74,0.82)	
Adaboost	0.47 (0.35, 0.59)	0.89 (0.85, 0.93)	0.55 (0.42, 0.68)	0.86 (0.81, 0.90)	0.47	0.50	0.798	0.80(0.75,0.84)	
XGB	0.39 (0.28, 0.51)	0.91 (0.87, 0.95)	0.56 (0.41, 0.70)	0.84 (0.79, 0.88)	0.39	0.46	0.772	0.80(0.75,0.84)	
RF	0.43 (0.31, 0.55)	0.93 (0.89, 0.95)	0.62 (0.47, 0.75)	0.85 (0.81, 0.89)	0.43	0.51	0.807	0.82(0.77,0.86)	
DT	0.20 (0.11, 0.31)	0.94 (0.90, 0.96)	0.47 (0.28, 0.66)	0.81 (0.76, 0.85)	0.22	0.31	0.584	0.79(0.74,0.83)	
KNN	0.19 (0.11, 0.30)	0.96 (0.93, 0.98)	0.61 (0.39, 0.80)	0.81 (0.76, 0.85)	0.26	0.39	0.733	0.79(0.76,0.72)	
CLS, classifiers; AUC, Area Under the Curve; CI, confidence interval; LR, Logistic Regression; NN: Neural Network; SVM, Support Vector Machine; MLP: Multilayer Perceptron; GP: Gaussian Process; GBM: Gradient Boosting Machine; NB: Naive Bayes; Adaboost: Adaptive Boosting; XGB: EXtreme Gradient Boosting; RF: Random Forest; DT: Decision Tree; KNN: K-Nearest Neighborsk

Fig. 2 A Twelve types of ROC curves for machine learning prediction of MACE in patients with ACS within one year of PCI.B ROC curve of GRACE score to predict MACE in patients with ACS within one year after PCI.

Fig. 2

Fig. 3 Decision curve analysis of a clinical prediction model for the occurrence of MACE 1 year after PCI in ACS patients.

Fig. 3

The SHAP model is a versatile approach to model interpretability, allowing for the exploration of associations between model predictions and specific features at both the global and local levels. SHAP demonstrates its unique strengths by considering the marginal contributions of each feature across all possible feature sequences. This process calculates the average to obtain the SHAP baseline value for the feature, ensuring accurate measurement and interpretation of each feature's contribution to the model. After careful selection and tuning, we determined that the RF classifier would provide the best performance for this dataset. Therefore, we adopted this classifier to investigate the core features of the dataset that are closely related to MACE prediction. Fig. 4A shows the global feature importance mapping generated by inputting the SHAP value matrix into the bar graph function. The plot represents the global importance of each feature by its average absolute value across all samples, while the Y-axis ranking visualizes its criticality in the model. After conducting a thorough analysis, we determined that the left ventricular ejection fraction (LVEF) was the primary factor in our model construction among the eight characteristic variables. Referring to Fig. 4B, each point represents a patient. The x-axis coordinates correspond to the predicted values from the model, while the color of the dots represents the measured values. For instance, the prediction of outcomes is significantly negatively affected by left ventricular ejection fraction (LVEF). This means that as the value of LVEF increases, the likelihood of experiencing a major adverse cardiovascular event (MACE) decreases. The impact of feature rank on the results is shown through the yellow and purple dots on the graph. Yellow dots represent high LVEF values on the negative side of the X-axis, while purple dots represent low LVEF values on the positive side of the X-axis. Additionally, the model indicates a strong negative correlation with albumin. Several clinical variables significantly contribute to prognostic prediction. As the measurements of these variables rise, the probability of MACE also increases. These variables include age, BNP, hs-CRP, D-Dimer, myoglobin, and creatinine.Fig. 4 A shows the SHAP values for macro feature importance, while Fig. 4B displays a scatterplot of macro feature density. Each row of the plot represents a feature, with the SHAP value as the horizontal coordinate. The features are ranked according to the average absolute value of SHAP, with the most important feature for the model being at the top. Wide areas indicate a large number of samples clustered together. The figure displays a ranking graph of the feature's importance, with yellow indicating a larger value and purple indicating a smaller value. Fig. 4C and D shows the micro single-sample feature influence diagram.

Fig. 4

Fig. 4C and D displays the personalized trait attribution analysis for two randomly selected patients. The mean effect value, Ef(x), across all sample data is 1.25. The impact of each feature variable is visually represented by arrows, with the direction indicating whether it increases or decreases the probability of the outcome. This can either decrease the likelihood of a negative outcome or increase the chance of a positive outcome. The arrows are arranged based on their impact on the outcome and color-coded to differentiate between positive (purple) and negative (yellow) effects. The length of each arrow is proportional to the SHAP value of the corresponding feature, accurately reflecting its strength of influence. Fig. 4C presents the characteristic attributions of a patient who developed MACE. The identified high-risk factors for MACE include low LVEF value, high hs-CRP level, high age, high BNP value, high D-Dimer level, high myoglobin, and high creatinine level. In comparison, Fig. 4D illustrates the attribution of characteristics in a patient who did not develop MACE. While advanced age is still considered a risk factor for MACE, the reduced levels of BNP, hs-CRP, D-Dimer, myoglobin, and creatinine in this patient significantly decrease the risk of developing MACE. This comparative analysis emphasizes the complex interaction of different characteristics in cardiovascular risk assessment.

To provide more detail on our constructed models, we utilized LIME, an interpretable tool in the machine learning field [19]. Fig. 5A illustrates the process of interpreting individual predictions, and physicians are better able to make decisions using the model when an understandable interpretation is provided. In Fig. 5A, the blue color represents features that support the predicted outcome, while the red color indicates features that do not support the predicted outcome. The horizontal coordinates reflect the magnitude of the weights for supporting (blue) or not supporting (red), and the vertical coordinates rank the features according to their importance. In the first example, a patient who developed MACE was over 70 years old and had a D-dimer level between 89 and 160, which provided support for predicting outcomes. However, the other variables did not provide strong support for the prediction results. Ultimately, the predictive accuracy in this case was only 0.68. The next three cases involved patients who did not have a MACE event. We can see in detail how the specific range of each feature affects the final prediction. It is worth noting that the prediction accuracies of these three cases are as high as 1, 0.99, and 0.97 respectively, fully demonstrating the excellent performance of our prediction model. Fig. 5B, on the other hand, shows the range of features and their corresponding weights for these four cases in a comprehensive manner. As can be seen, the gradient from red to blue shows the dynamic evolution of the feature weights. The dark blue region represents strong support for the prediction result, while the increasingly reddish region implies a gradual weakening of the support for the prediction result. In the construction of the prediction model, the range of features is marked by the vertical coordinate, the hue of which directly represents the importance and influence of the feature in the model.Fig. 5A 5B Interpreting individual forecasts.

Fig. 5A 5B

4 Discussion

This study aimed to construct a highly explanatory machine learning model to analyze in depth the potential association between multiple risk factors and the occurrence of MACE within one year in ACS patients undergoing PCI. In doing so, we expect to achieve effective risk stratification of patients with ACS after PCI. In the population of post-PCI ACS patients included in the study, we found that the predictive model based on the Random Forest algorithm demonstrated superior performance in predicting the occurrence of MACE within one year compared to predictive models built using other machine learning algorithms.

Predictive models based on machine learning have emerged in recent years, but they are poorly interpreted and their models are like opaque “black boxes” that make it difficult to understand how the algorithms make decisions, even if you understand the mathematics of the algorithms [20,21]. We believe that providing physicians with detailed and understandable explanations is an important part of ensuring that they trust and use machine learning predictive models. In this study, two advanced interpretable analysis techniques, SHAP and LIME, were used to provide deeper insight into the RF machine learning predictive model, systematically exploring the global associations between all sample characteristic variables and MACE [22]. At the same time, we predicted several representative individuals in the model to ensure global understanding. This initiative not only ensured excellent model performance but also provided a new level of insight into clinical interpretation. To the best of our knowledge, we are the first to use LIME Interpretable Analytics to provide insight into a machine learning-based model for predicting the prognosis of patients with acute coronary syndrome (ACS). It has the advantage of showing, for the first time, the weighting of the predictive model by the range of feature variables, which is more clinically applicable and provides physicians with more explicit and intuitive model decision support. The subtle strategy we have constructed aims to provide physicians with more explicit and intuitive model decision support, thus promoting the efficient translation and application of predictive results into clinical practice. In addition, to effectively improve the targeting of clinical interventions, we paid particular attention to the range of threshold probabilities closely associated with DCA, i.e. between 15 and 70 percent.

In this study, the results of the SHAP analysis showed that LVEF had a significant impact on the generation of risk for patient outcomes. LVEF is strongly associated with cardiac function and is considered a core indicator in the current assessment of heart failure status [23]. Previous studies have reported the association between reduced LVEF and adverse cardiovascular events using traditional statistical analyses [24,25]. After an in-depth comparative analysis of the recovery of 1429 post-PCI patients, the team of experts led by Hanada et al. found that an LVEF of less than 40 % may be a strong predictor of long-term MACE [26]. The study also confirmed that by increasing LVEF, we were able to effectively prevent the phenomenon of ventricular remodeling, which in turn significantly reduced the likelihood of MACE [27]. Albumin also plays a very important role in our machine-learning model. Albumin is a protein that is primarily produced by the liver and circulates in the plasma. It performs a multitude of functions, including the maintenance of fluid balance, the transportation of hormones and drugs, and the regulation of the immune response. The most prevalent risk factor for the development of atherosclerosis is an elevated production of reactive oxygen species (ROS). Oxidative stress is a significant contributor to the pathophysiology of ACS [28]. Oxidative stress exerts a profound influence on circulating proteins, with albumin representing the most prevalent of these proteins [29].The prognostic value of low albumin in ACS and stable coronary artery disease, including previous myocardial infarction and heart failure, has been reported [30]. Another study concluded that low albumin in stable coronary artery disease (CAD) is caused by atherosclerotic systemic inflammation [31]. The present study revealed a significant reduction in albumin concentrations among patients with ACS who experienced MACE one year after PCI, in comparison to those who did not develop a MACE event. The elevated risk of MACE at albumin levels below 44.7 g/L is consistent with the findings of previous studies that have associated albumin levels below 45 g/L with an increased risk of coronary heart disease [32].

The GRACE score is a predictive model developed by Eagle et al. [33]. For the purpose of assessing the risk of death in patients with acute coronary syndrome (ACS) within six months of hospital discharge. Utilising the extensive data from the GRACE registry (encompassing 94 hospitals across 14 countries), Fox and colleagues refined the information from 32,037 cases to meticulously construct a novel GRACE scoring model, which was subsequently validated externally in a French registry of myocardial infarctions (both acute ST-segment-elevation and non-ST-segment-elevation types) [34]. In this study, the GRACE score was employed to forecast the incidence of MACE in ACS patients within a one-year timeframe. The AUC value obtained was 0.673, while the AUC value of the machine-learning prediction model constructed in this study reached 0.803. This suggests that the prediction model developed through the utilisation of a machine-learning algorithm is more effective than the traditional GRACE score. The concurrent expansion of the economy and the advancement of technology have resulted in a notable reduction in the costs associated with data collection and storage. Concurrently, data analysis techniques have become increasingly sophisticated. It is therefore appropriate that clinical prediction models should embrace larger and richer data resources (i.e. big data) and apply more sophisticated models and algorithms in order to provide more accurate results, thus better serving doctors, patients and healthcare decision makers.

Of course, there are limitations to this study; our cohort size was relatively small and the results should be considered preliminary. This study is a single-centre retrospective study, which is subject to some of the limitations inherent in retrospective studies. The predictive model developed in this study was not externally validated due to the lack of external data. Consequently, the applicability of the model to the wider population still requires validation with new data. A formal power analysis could not be performed for ML; therefore, it is unclear how many subjects are needed. As disease risk factors, unmeasured risk factors, therapeutic interventions, therapeutic context and patient focus, adherence, and psychological factors change over time, the performance of the prediction model may decrease.

5 Conclusion

The interpretable ML model constructed in this study can identify the risk of MACE within 1 year in ACS patients undergoing PCI and can identify risk markers associated with MACE. In particular, we represented the range of characteristic variables as weights in the model, which is more applicable to the clinic and easier for physicians to understand and thus make decisions. The interpretable ML model identified LVEF and albumin, among others, as important factors for MACE events after PCI in ACS patients. In addition, the model allows for individualized identification of clinically important factors that help predict risk for each patient.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.

Funding

This study was fully supported by the Key Project of the Air Force University of Military Medicine, China, Project No. KJ2022A000409 , and was also generously funded by the 2022 Chinese People's Liberation Army Air Force Medical Centre Science and Technology Boosting Programme, Grant No. 2022ZTYB14 .

For ethics approval

This study was reviewed and approved by Chinese People's Liberation Army Air Force Medical Centre with the approval number: 2023-73-PJ01, dated August 22, 2023.

CRediT authorship contribution statement

Xing-Yu Zhu: Writing – review & editing, Writing – original draft, Methodology. Kai-Jie Zhang: Investigation, Data curation. Xiao Li: Visualization, Validation. Fei-Fei Su: Writing – review & editing, Funding acquisition. Jian-Wei Tian: Writing – review & editing, Supervision, Resources, Funding acquisition, Data curation.

Declaration of competing interest

All authors disclosed no relevant relationships.
==== Refs
References

1 Interpretation of the annual report on cardiovascular health and diseases in China 2022 Cardiology Discovery 4 1 2024 58 80 10.1097/cd9.0000000000000119 DOl
2 Tsao C.W. Aday A.W. Almarzooq Z.I. Heart disease and stroke statistics-2023 update: a report from the American heart association Circulation 147 8 2023 e93 e621 10.1161/cir.0000000000001123 DOl 36695182
3 Bhatt D.L. Lopes R.D. Harrington R.A. Diagnosis and treatment of acute coronary syndromes: a review JAMA 327 7 2022 10.1001/jama.2022.0358 662-75. DOl
4 Tsao C.W. Aday A.W. Almarzooq Z.I. Heart disease and stroke statistics-2022 update: a report from the American heart association Circulation 145 8 2022 e153 e639 10.1161/cir.0000000000001052 DOl 35078371
5 Bergmark B.A. Mathenge N. Merlini P.A. Acute coronary syndromes Lancet (London, England) 399 10332 2022 10.1016/s0140-6736(21)02391-6 1347-58. DOl
6 Damluji A.A. Forman D.E. Wang T.Y. Management of acute coronary syndrome in the older adult population: a scientific statement from the American heart association Circulation 147 3 2023 e32 e62 10.1161/cir.0000000000001112 DOl 36503287
7 Hagström E. James S.K. Bertilsson M. Growth differentiation factor-15 level predicts major bleeding and cardiovascular events in patients with acute coronary syndromes: results from the PLATO study Eur. Heart J. 37 16 2016 1325 1333 10.1093/eurheartj/ehv491 DOl 26417057
8 Batra G. Lindbäck J. Becker R.C. Biomarker-based prediction of recurrent ischemic events in patients with acute coronary syndromes J. Am. Coll. Cardiol. 80 18 2022 1735 1747 10.1016/j.jacc.2022.08.767 DOl 36302586
9 Alaa A.M. Bolton T. Di Angelantonio E. Cardiovascular disease risk prediction using automated machine learning: a prospective study of 423,604 UK Biobank participants PLoS One 14 5 2019 e0213653 10.1371/journal.pone.0213653 DOl
10 D'ascenzo F. De Filippo O. Gallone G. Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): a modelling study of pooled datasets Lancet (London, England) 397 10270 2021 199 207 10.1016/s0140-6736(20)32519-8 DOl 33453782
11 Karim M.R. Islam T. Shajalal M. Explainable AI for bioinformatics: methods, tools and applications Briefings Bioinf. 24 5 2023 10.1093/bib/bbad236 DOl
12 Hilton C.B. Milinovich A. Felix C. Personalized predictions of patient outcomes during and after hospitalization using artificial intelligence NPJ digital medicine 3 51 2020 10.1038/s41746-020-0249-z DOl
13 Lundberg S.M. Erion G. Chen H. From local explanations to global understanding with explainable AI for trees Nat. Mach. Intell. 2 1 2020 56 67 10.1038/s42256-019-0138-9 DOl 32607472
14 Fahmy A.S. Csecs I. Arafati A. An explainable machine learning approach reveals prognostic significance of right ventricular dysfunction in nonischemic cardiomyopathy JACC Cardiovascular imaging 15 5 2022 10.1016/j.jcmg.2021.11.029 766-79. DOl
15 Li J. Liu S. Hu Y. Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study J. Med. Internet Res. 24 8 2022 e38082 10.2196/38082 DOl
16 Lundberg S.M. Lee S.-I. A unified approach to interpreting model predictions Proceedings of the 31st International Conference on Neural Information Processing Systems 2017 Curran Associates Inc. Long Beach, California, USA 4768 4777 [M]
17 Van Calster B. Wynants L. Verbeek J.F.M. Reporting and interpreting decision curve analysis: a guide for investigators Eur. Urol. 74 6 2018 796 804 10.1016/j.eururo.2018.08.038 DOl 30241973
18 Lee C. Light A. Alaa A. Application of a novel machine learning framework for predicting non-metastatic prostate cancer-specific mortality in men using the Surveillance, Epidemiology, and End Results (SEER) database The Lancet Digital health 3 3 2021 e158 e165 10.1016/s2589-7500(20)30314-9 DOl 33549512
19 Peng K. Menzies T. Documenting evidence of a reuse of ‘“why should I trust you?”: explaining the predictions of any classifier’ Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering 2021
20 Sánchez-Cabo F. Rossello X. Fuster V. Machine learning improves cardiovascular risk definition for young, asymptomatic individuals J. Am. Coll. Cardiol. 76 14 2020 1674 1685 10.1016/j.jacc.2020.08.017 DOl 33004133
21 Al-Zaiti S. Besomi L. Bouzid Z. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram Nat. Commun. 11 1 2020 3966 10.1038/s41467-020-17804-2 DOl 32769990
22 Azodi C.B. Tang J. Shiu S.H. Opening the black box: interpretable machine learning for geneticists Trends Genet. : TIG (Trends Genet.) 36 6 2020 442 455 10.1016/j.tig.2020.03.005 DOl 32396837
23 Correction to: 2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the American college of cardiology/American heart association joint committee on clinical practice guidelines Circulation 147 14 2023 36154623 10.1161/CIR.0000000000001097 DOl:
24 De Filippo O. D'ascenzo F. Wańha W. IncidenCe and predictOrs of heaRt fAiLure after acute coronarY Syndrome: the CORALYS registry Int. J. Cardiol. 370 2023 35 42 10.1016/j.ijcard.2022.10.146 DOl 36306949
25 Pavasini R. Cardelli L.S. Piredda A. Diastolic dysfunction, frailty and prognosis in elderly patients with acute coronary syndromes Int. J. Cardiol. 327 2021 31 35 10.1016/j.ijcard.2020.11.056 DOl 33271207
26 Gallone G. Kang J. Bruno F. Impact of left ventricular ejection fraction on procedural and long-term outcomes of bifurcation percutaneous coronary intervention Am. J. Cardiol. 172 2022 18 25 10.1016/j.amjcard.2022.02.015 DOl 35365291
27 Healthcare Engineering J O. Retracted: preventive effect observation of dapagliflozin on middle and later ventricular remodeling in patients with acute st segment elevation anterior wall myocardial infarction: a single-center, retrospective cohort study J Healthc Eng 2023 2023 9782190 10.1155/2023/9782190 DOl
28 Misra M.K. Sarwat M. Bhakuni P. Oxidative stress and ischemic myocardial syndromes Med. Sci. Mon. Int. Med. J. Exp. Clin. Res. : international medical journal of experimental and clinical research 15 10 2009 Ra209 R219
29 Sitar M.E. Aydin S. Cakatay U. Human serum albumin and its relation with oxidative stress Clin. Lab. 59 9–10 2013 945 952 24273915
30 Kurtul A. Murat S.N. Yarlioglues M. Usefulness of serum albumin concentration to predict high coronary SYNTAX score and in-hospital mortality in patients with acute coronary syndrome Angiology 67 1 2016 34 40 10.1177/0003319715575220 DOl 25783433
31 Suzuki S. Hashizume N. Kanzaki Y. Prognostic significance of serum albumin in patients with stable coronary artery disease treated by percutaneous coronary intervention PLoS One 14 7 2019 e0219044 10.1371/journal.pone.0219044 DOl
32 Binti N.N. Ferdausi N. Anik M.E.K. Association of albumin, fibrinogen, and modified proteins with acute coronary syndrome PLoS One 17 7 2022 e0271882 10.1371/journal.pone.0271882 DOl
33 Eagle K.A. Lim M.J. Dabbous O.H. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry JAMA 291 22 2004 10.1001/jama.291.22.2727 2727-33. DOl
34 Fox K.A. Fitzgerald G. Puymirat E. Should patients with acute coronary disease be stratified for management according to their risk? Derivation, external validation and outcomes using the updated GRACE risk score BMJ Open 4 2 2014 e004425 10.1136/bmjopen-2013-004425 DOl
