
==== Front
JAMA Intern Med
JAMA Intern Med
JAMA Internal Medicine
2168-6106
2168-6114
American Medical Association

39037785
10.1001/jamainternmed.2024.3219
ild240016
Research
Letters
Research Letter
Online First
Accelerated Chest Pain Treatment With Artificial Intelligence–Informed, Risk-Driven Triage
Accelerated Chest Pain Treatment With Artificial Intelligence–Informed Triage
Letters
Hinson Jeremiah S. MD PhD 1 2 3
Taylor R. Andrew PhD 4 5
Venkatesh Arjun MD 4
Steinhart Benjamin D. MS 3
Chmura Christopher RN 4
Sangal Rohit B. MD MBA 4
Levin Scott R. PhD 1 2 3
1 Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
2 Malone Center for Engineering in Healthcare, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
3 Beckman Coulter Diagnostics, Brea, California
4 Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut
5 Department of Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, Connecticut
Article Information

Accepted for Publication: May 23, 2024.

Published Online: July 22, 2024. doi:10.1001/jamainternmed.2024.3219

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2024 Hinson JS et al. JAMA Internal Medicine.

Corresponding Author: Jeremiah S. Hinson, MD, PhD, Department of Emergency Medicine, Johns Hopkins University School of Medicine, 1830 E Monument St, Ste 6-100, Baltimore, MD 21287 (hinson@jhmi.edu).
Author Contributions: Dr Hinson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Hinson, Taylor, Venkatesh, Chmura, Levin.

Acquisition, analysis, or interpretation of data: Hinson, Taylor, Venkatesh, Steinhart, Sangal, Levin.

Drafting of the manuscript: Hinson, Taylor, Steinhart, Levin.

Critical review of the manuscript for important intellectual content: Hinson, Taylor, Venkatesh, Chmura, Sangal, Levin.

Statistical analysis: Hinson, Steinhart, Levin.

Administrative, technical, or material support: Hinson, Taylor, Venkatesh, Chmura.

Supervision: Hinson, Venkatesh, Levin.

Conflict of Interest Disclosures: Drs Hinson and Levin reported receiving royalties from Beckman Coulter for TriageGO (patent US20170242973A1 pending), which was developed at Johns Hopkins University and has been licensed to and made commercially available by Beckman Coulter Diagnostics. Dr Hinson reported receiving grants from the Agency for Healthcare Research and Quality, National Institutes of Health, and Centers for Disease Control and Prevention outside the submitted work. Dr Taylor reported receiving grants from Beckman Coulter during the conduct of the study and grants from the National Institutes of Health, Moore Foundation, Society to Improve Diagnosis in Medicine, and Agency for Healthcare Research and Quality outside the submitted work. Dr Venkatesh reported receiving grants from the Centers for Disease Control and Prevention and Society for Academic Medicine Foundation outside the submitted work. No other disclosures were reported.

Data Sharing Statement: See Supplement 2.

22 7 2024
9 2024
22 7 2024
184 9 11251127
5 2 2024
23 5 2024
Copyright 2024 Hinson JS et al. JAMA Internal Medicine.
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article distributed under the terms of the CC-BY-NC-ND License.
jamainternmed-e243219.pdf

This quality improvement study evaluates the use of artificial intelligence to accelerate triage of patients presenting to the emergency department with chest pain.
==== Body
pmcEight million US emergency department (ED) visits for chest pain occur annually.1 Approximately 6% are caused by life-threatening conditions.2 Interventions that address ED overcrowding, including improved triage, may speed time to identification and treatment of these conditions.

Methods

In this multisite quality improvement study, we compared treatment intervals for adult patients (aged ≥18 years) with chest pain before and after implementation of an artificial intelligence (AI)–informed, outcomes-driven decision support system for ED triage (TriageGO; Beckman Coulter).3,4 The Western-Copernicus Group institutional review board approved this study, with a waiver of informed consent after minimal risk determination. The study follows the SQUIRE reporting guideline.

At the time of ED arrival, TriageGO uses machine learning algorithms to estimate probabilities for critical care, emergency procedures, and hospital admission using demographics, arrival mode, vital signs, chief complaints, and active medical problems as predictors; it translates outcome probabilities to recommended acuity levels (1-5, with lower values indicating higher acuity).3 TriageGO implementation at 3 EDs within a university health system was staggered between May 19, 2021, and April 4, 2023. At all sites, TriageGO replaced the emergency severity index.5

Patients who visited a study site with a chief complaint of chest pain 180 days before or after implementation of TriageGO were included in this analysis. Downstream protocols for diagnosis and treatment of chest pain remained consistent before and after intervention. Patient characteristics, triage acuity distributions, ED length of stay (LOS) (arrival to departure), time to procedure (arrival to procedure start time), and rates of hospitalization, intensive care unit (ICU) admission, emergency surgery, and 30-day mortality were compared.

To limit confounding, we performed adjusted analyses using median regression models that controlled for patient (age, sex, hospitalization status, emergency surgery status, in-hospital mortality, ICU admission) and system (daily volume, time of day, ED boarding census) factors. The eMethods in Supplement 1 provide additional details.

Python, version 3.8.11 (Python Software Foundation) was used for all analyses. A 2-sided P < .05 by Wilcoxon rank sum or χ2 test was considered significant.

Results

A total of 12 147 ED visits were analyzed, with 6188 visits before implementation (median [IQR] age, 52.5 [36.6-65.5] years; 3283 females [53.1%] and 2905 males [46.9%]) and 5959 after (median [IQR] age, 51.9 [36.7-64.2] years; 3075 females [51.6%] and 2884 males [48.4%]). All patient factors were similar between study periods (Table 1). After implementation of TriageGO, vs before implementation, fewer patients were assigned to high acuity levels 1 or 2 (1317 [22.1%] vs 1708 [27.6%]) or mid acuity level 3 (3263 [54.8%] vs 4086 [66.0%]) and more to low acuity levels 4 or 5 (1379 [23.1%] vs 394 [6.4%]) (χ24 = 771.6; P < .001).

Table 1. Cohort Characteristics and Clinical Outcomes

	Patients, No. (%)	P valuea	
Total (N = 12 147)	Before implementation (n = 6188)	After implementation (n = 5959)	
Patient characteristics	
Age, median (IQR), y	52.2 (36.6-64.9)	52.5 (36.6-65.5)	51.9 (36.7-64.2)	.25	
Sex					
Female	6358 (52.3)	3283 (53.1)	3075 (51.6)	.11	
Male	5789 (47.7)	2905 (46.9)	2884 (48.4)	
Comorbidities					
Hypertension	4136 (34.0)	2112 (34.1)	2024 (34.0)	.86	
Diabetes	1993 (16.4)	989 (16.0)	1004 (16.8)	.21	
Hyperlipidemia	3385 (27.9)	1722 (27.8)	1663 (27.9)	.94	
Smoker	1372 (11.3)	641 (10.4)	731 (12.3)	.001	
Coronary artery disease	1679 (13.8)	842 (13.6)	837 (14.0)	.50	
Prior MI	101 (0.8)	51 (0.8)	50 (0.8)	>.99	
Prior CABG	13 (0.1)	4 (0.1)	9 (0.1)	.24	
Triage vital signs, median (IQR)					
Heart rate, beats/min	82.0 (71.0-95.0)	83.0 (71.0-95.0)	82.0 (71.0-94.0)	.11	
Systolic blood pressure, mm Hg	138.0 (124.0-154.0)	139.0 (124.0-156.0)	137.0 (123.0-153.0)	<.001	
Respiratory rate, breaths/min	18.0 (16.0-18.0)	18.0 (16.0-18.0)	18.0 (16.0-18.0)	.24	
Oxygen saturation, %	98.0 (97.0-100)	98.0 (97.0-100)	98.0 (97.0-100)	.20	
Temperature, °F	98.0 (97.5-98.3)	98.0 (97.6-98.4)	97.9 (97.4-98.3)	<.001	
Triage acuityb					
1 (Highest)	114 (0.9)	17 (0.3)	97 (1.6)	<.001	
2	2911 (24.0)	1691 (27.3)	1220 (20.5)	
3	7349 (60.5)	4086 (66.0)	3263 (54.8)	
4	1712 (14.1)	388 (6.3)	1324 (22.2)	
5 (Lowest)	61 (0.5)	6 (0.1)	55 (0.9)	
ED disposition					
Hospitalized	2245 (18.5)	1097 (17.7)	1148 (19.3)	.05	
Discharged	9504 (78.2)	4893 (79.1)	4611 (77.4)	
Left	331 (2.7)	168 (2.7)	163 (2.7)	
Clinical outcomes	
Patients hospitalized from the ED					
Any emergency procedurec	396 (3.3)	197 (3.2)	199 (3.3)	.67	
Cardiovascular interventiond	322 (2.7)	162 (2.6)	160 (2.7)	.86	
Cardiac catheterization	294 (2.4)	142 (2.3)	152 (2.6)	.39	
ICU admission	126 (1.0)	66 (1.1)	60 (1.0)	.81	
30-d Mortality	62 (0.5)	27 (0.4)	35 (0.6)	.30	
Patients discharged from the ED					
72-h Return with admission	466 (3.8)	221 (3.6)	245 (4.1)	.13	
72-h Return with emergency procedure	6 (<.01)	3 (<.01)	3 (0.1)	>.99	
30-d Mortality	19 (0.2)	12 (0.2)	7 (0.1)	.40	
Abbreviations: CABG, coronary artery bypass graft; ED, emergency department; ICU, intensive care unit; MI, myocardial infarction.

a Compared using χ2 test for categorical variables and Wilcoxon rank sum test for continuous variables.

b Triage acuities represent final assignments by triage nurses.

c Emergency procedures were defined as any procedure or surgery performed in a dedicated suite within 12 hours of ED departure.

d Cardiovascular interventions include any procedure or surgery performed on the heart or great vessels, with cardiac catheterization as a subset.

Median LOS was unchanged for discharged patients but was reduced from 657.5 (95% CI, 382.0-1413.5) to 502.0 (95% CI, 331.5-986.0) minutes for hospitalized patients. After adjustment for potential confounders, LOS was reduced by 76.4 minutes (95% CI, 28.6-124.1 minutes) for hospitalized patients and was most pronounced for emergency procedures (128.0 [95% CI, 3.9-286.0] minutes). Adjusted median time to emergency cardiovascular procedures was reduced by 205.4 minutes (95% CI, 23.0-387.8 minutes), including cardiac catheterization (by 243.2 minutes; 95% CI, 43.7-442.7 minutes) (Table 2). There were no changes in 30-day mortality or 72-hour ED returns requiring hospitalization or emergency procedures.

Table 2. Emergency Department Length of Stay and Time to Procedure Before and After Implementation of Artificial Intelligence–Informed Triage

Variable	Time, median (95% CI), min	
Total	Implementation period	Difference	
Before	After	Unadjusted	Adjusteda	
Length of stay b	
Discharged	288.0 (200.0 to 415.0)	288.0 (201.0 to 417.0)	288.0 (200.0 to 415.0)	0 (−8.0 to 8.0)	3.7 (−4.2 to 11.5)	
Hospitalized	560.0 (353.0 to 1232.5)	657.5 (382.0 to 1413.5)	502.0 (331.5 to 986.0)	155.5 (98.5 to 214.0)	76.4 (28.6 to 124.1)	
Any emergency procedurec	346.0 (79.5 to 692.5)	419.0 (120.0 to 889.0)	291.0 (57.0 to 546.5)	128.0 (3.9 to 286.0)	121.5 (10.4 to 232.7)	
Cardiovascular interventiond	256.5 (59.2 to 526.8)	318.5 (87.0 to 626.0)	166.0 (49.8 to 434.0)	152.5 (34.0 to 276.0)	159.5 (53.0 to 265.9)	
Cardiac catheterization	245.5 (56.0 to 537.0)	331.0 (77.2 to 690.2)	148.0 (47.8 to 434.0)	183.0 (36.5 to 325.5)	164.9 (47.7 to 282.1)	
Time to procedure e	
Any emergency procedurec	480.0 (60.0 to 1020.0)	600.0 (120.0 to 1140.0)	420.0 (60.0 to 915.0)	180.0 (0.0 to 480.0)	102.3 (−57.8 to 262.5)	
Cardiovascular interventiond	300.0 (60.0 to 900.0)	480.0 (60.0 to 960.0)	180.0 (30.0 to 840.0)	300.0 (119.3 to 480.0)	205.4 (23.0 to 387.8)	
Cardiac catheterization	360.0 (60.0 to 900.0)	480.0 (60.0 to 975.0)	120.0 (0.0 to 840.0)	360.0 (120.0 to 540.0)	243.2 (42.0 to 444.5)	
a Adjusted analyses were performed using median regression models that controlled for patient factors (age, sex, hospitalization status, emergency surgery status, in-hospital mortality, intensive care unit admission) and system factors (daily volume, time of day, emergency department [ED] boarding census).

b Emergency department length of stay represents the interval from ED arrival to ED departure.

c Emergency procedures were defined as any procedure or surgery performed in a dedicated suite within 12 hours of ED departure.

d Cardiovascular interventions include any procedure or surgery performed on the heart or great vessels, with cardiac catheterization as a subset.

e Time to procedure represents the interval from ED arrival to procedure start time.

Discussion

The findings show that implementation of TriageGO led to a wider distribution of patients with chest pain across ED triage levels. Alignment of triage decision-making with clinical risk allowed for safe diversion of more patients to lower acuity levels, driving timely attention to those at highest risk for adverse outcomes. This timely attention facilitated decreased ED LOS and faster transitions to inpatient care and cardiovascular interventions. Time savings associated with TriageGO were additive to any conferred by specific diagnostic approaches, which were unchanged between study periods. This study was limited by its performance within a single health system and nonrandomized study group allocation. Decision support informed by AI may improve patient stratification at triage and expedite chest pain management.

Supplement 1. eMethods

eReferences.

Supplement 2. Data Sharing Statement
==== Refs
References

1 Emergency department visits. National Center for Health Statistics. Published November 4, 2023. Accessed December 29, 2023. https://www.cdc.gov/nchs/fastats/emergency-department.htm
2 Hsia RY, Hale Z, Tabas JA. A national study of the prevalence of life-threatening diagnoses in patients with chest pain. JAMA Intern Med. 2016;176 (7 ):1029-1032. doi:10.1001/jamainternmed.2016.2498 27295579
3 Levin S, Toerper M, Hamrock E, . Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med. 2018;71 (5 ):565-574.e2. doi:10.1016/j.annemergmed.2017.08.005 28888332
4 Levin S, Toerper M, Hinson J, . Machine-learning-based electronic triage: a prospective evaluation. Ann Emerg Med. 2018;72 (4 ):S116. doi:10.1016/j.annemergmed.2018.08.299
5 Gilboy N, Tanabe P, Travers D, Rosenau AM, eds. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care. 4th ed. Agency for Healthcare Research and Quality; 2012.
