
==== Front
medRxiv
MEDRXIV
medRxiv
Cold Spring Harbor Laboratory

10.1101/2024.05.22.24307754
preprint
1
Article
Effect of Machine Learning on Anaesthesiology Clinician Prediction of Postoperative Complications: The Perioperative ORACLE Randomised Clinical Trial
Fritz Bradley A http://orcid.org/0000-0002-7239-8877

King Christopher R http://orcid.org/0000-0002-4574-8616

Abdelhack Mohamed http://orcid.org/0000-0002-6753-3237

Chen Yixin
Kronzer Alexander
Abraham Joanna
Tripathi Sandhya http://orcid.org/0000-0003-3992-2283

Ben Abdallah Arbi http://orcid.org/0000-0002-1287-0455

Kannampallil Thomas http://orcid.org/0000-0003-4119-4836

Budelier Thaddeus P http://orcid.org/0000-0002-8427-5583

Helsten Daniel
Montes de Oca Arianna
Mehta Divya
Sontha Pratyush
Higo Omokhaye http://orcid.org/0009-0007-7997-4551

Kerby Paul
Gregory Stephen H
Wildes Troy S http://orcid.org/0000-0002-9042-571X

Avidan Michael S http://orcid.org/0000-0001-6248-044X

23 5 2024
2024.05.22.24307754http://medrxiv.org/lookup/doi/10.1101/2024.05.22.24307754
nihpp-2024.05.22.24307754.pdf
Background: Anaesthesiology clinicians can implement risk mitigation strategies if they know which patients are at greatest risk for postoperative complications. Although machine learning models predicting complications exist, their impact on clinician risk assessment is unknown. Methods: This single-centre randomised clinical trial enrolled patients age ≥18 undergoing surgery with anaesthesiology services. Anaesthesiology clinicians providing remote intraoperative telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) also reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury within 7 days. Area under the receiver operating characteristic curve (AUROC) for the clinician predictions was determined. Results: Among 5,071 patient cases reviewed by 89 clinicians, the observed incidence was 2% for postoperative death and 11% for acute kidney injury. Clinician predictions agreed with the models more strongly in the assisted versus unassisted group (weighted kappa 0.75 versus 0.62 for death [difference 0.13, 95%CI 0.10-0.17] and 0.79 versus 0.54 for kidney injury [difference 0.25, 95%CI 0.21-0.29]). Clinicians predicted death with AUROC of 0.793 in the assisted group and 0.780 in the unassisted group (difference 0.013, 95%CI -0.070 to 0.097). Clinicians predicted kidney injury with AUROC of 0.734 in the assisted group and 0.688 in the unassisted group (difference 0.046, 95%CI -0.003 to 0.091). Conclusions: Although there was evidence that the models influenced clinician predictions, clinician performance was not statistically significantly different with and without machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. Trial Registration: ClinicalTrials.gov NCT05042804
==== Body
pmc
