
==== Front
Arch Acad Emerg Med
Arch Acad Emerg Med
AAEM
Archives of Academic Emergency Medicine
2645-4904
Shahid Beheshti University of Medical Sciences Tehran, Iran

10.22037/aaem.v12i1.2269
Original Research
Development of a Clinical Score for Predicting 28-Day Mortality in Geriatric Sepsis Patients; a Cohort study
Sanguanwit Pitsucha 1
Yuksen Chaiyaporn 1
Khorana Jiraporn 234
Sutham Krongkarn 5
Phootothum Yuranun 1
Damdin Siriporn 1
1 Department of Emergency Medicine, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, 10400, Thailand
2 Division of Pediatric Surgery, Department of Surgery, Faculty of Medicine, Chiang Mai University, Chiang Mai 50200, Thailand
3 Center of Clinical Epidemiology and Clinical Statistic, Faculty of Medicine, Chiang Mai University, Chiang Mai 50200, Thailand
4 Clinical Surgical Research Center, Department of Surgery, Faculty of Medicine, Chiang Mai University, Chiang Mai 50200, Thailand
5 Department of Emergency Medicine, Faculty of Medicine Chiangmai University, Chiangmai, 50200, Thailand
* Corresponding author: Chaiyaporn Yuksen; Department of Emergency Medicine, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, 10400, Thailand.
2024
29 6 2024
12 1 e563 2024
5 2024
https://creativecommons.org/licenses/by-nc/3.0/ This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). (https://creativecommons.org/licenses/by-nc/3.0/)
Introduction:

Sepsis is a significant and common cause of death and burden among critically ill patients, which has increasing incidence and mortality in adults over 60 and advanced age. This study aimed to develop an easy-to-use clinical tool for assessing 28-day mortality risk in older sepsis patients upon their initial assessment in the emergency department (ED).

Method:

A retrospective cohort study was conducted using electronic medical records of older (≥60 years) ED patients with suspected sepsis from August 1, 2018, to December 31, 2018. A new prediction score was formulated based on the logistic coefficients of clinical predictors through multivariable regression analyses. Then, the score's screening performance was evaluated and compared to existing scoring systems; Systemic Inflammatory Response Syndrome (SIRS), quick Sequential Organ Failure Assessment (qSOFA), National early warning score (NEWS), and The Ramathibodi early warning score (REWS); using receiver operating characteristic curve analysis (AuROC).

Result:

The study included 599 patients with the mean age of 77.13 (range: 60-101) years (56.43% male) and an overall 28-day mortality rate of 7.01%. The newly developed prediction score had seven independent predictors of 28-day mortality: malignancy, dependent status, heart rate, respiratory rate, oxygen saturation, consciousness, and lactate, which demonstrated excellent discriminative ability (AuROC: 0.87, 95% confidence interval (CI): 0.82 - 0.92), significantly outperforming SIRS (AuROC: 0.62), qSOFA (AuROC: 0.72), NEWS (AuROC: 0.74), and REWS (AuROC: 0.71), all with p-values <0.01. The score allowed risk stratification into low-risk (positive likelihood ratio (LR+): 0.37, 95% CI: 0.24 - 0.58) and high-risk (LR+: 4.14, 95% CI: 3.14 - 5.44) groups with sensitivity of 69.0% and specificity of 83.3% at a cut-off point of 6.

Conclusion:

The novel prediction score demonstrates a remarkable ability to predict 28-day mortality risk in older sepsis patients during their initial ED assessment, offering potential for improved risk stratification and treatment guidance in older patients

Key Words

Clinical decision rules
Prognosis; Mortality
Older/Aged
Sepsis
Emergency service
hospital
==== Body
pmc1. Introduction

The Population Reference Bureau reported that the number of people with aging has increased worldwide (1). Sepsis was defined a syndrome of life-threatening organ dysfunction caused by dysregulated host response to infection (2). Sepsis is a significant and common cause of death and burden among critically ill patients, which has increasing incidence and mortality in adults over 60 years old and those with advanced age (3-5). Diagnosis and prognosis prediction of sepsis in older adults was difficult due to unusual immune and clinical presentation. This can delay diagnosis and allocation of appropriate sepsis treatment to the patients.

Several sepsis scoring systems were developed to assess the severity and mortality of sepsis including the Systemic Inflammatory Response Syndrome (SIRS), the Sequential Organ Failure Assessment (SOFA), the quick SOFA (qSOFA), Modified early waring score (MEWS), National early waring score (NEWS), Ramathibodi early warning score (REWS), which was developed in our institution, Mortality in ED Sepsis (MEDS), The Predisposition, Infection, Response, Organ dysfunction (PIRO), etc. However, all of them had poor to moderate accuracy for risk stratification in predicting mortality in older sepsis patients (6, 7).

Zelis N. et al developed “the rise up score” that included age, ≥2 abnormal vital signs, serum albumin, blood urea nitrogen (BUN), lactate dehydrogenase (LDH), and bilirubin for predicting 30-day mortality in older medical emergency department patients. They found that “the rise up score” was very good with an area under the curve (AUC) of 0.84. They aimed to make the score easy to apply using variables that are readily available, but getting the blood chemistry (serum albumin, BUN, LDH, and bilirubin) results take time (8).

On previous data, we explored readily available easy factors, which predicted 28-day mortality of suspicious older (≥60 years) sepsis patients in the emergency department (ED). We found that comorbid malignancy, Oxygen saturation ≤93%, and patient’s dependent status were significant prognostic indicators (9). In this study, we aimed to develop a clinical prediction tool that could be easily used for initial assessment of suspicious older (≥60 years) sepsis patients in emergency department, in order to predict 28-day mortality.

2. Methods

2.1 Study design and setting 

We performed a prognostic prediction research based on a single-center, retrospective, cohort study in the Emergency department of Ramathibodi Hospital, a tertiary university Hospital, Bangkok, Thailand. A new prediction score was formulated based on the logistic coefficients of clinical predictors through multivariable regression analyses. Then, the score's screening performance was evaluated and compared to existing scoring systems; Systemic Inflammatory Response Syndrome (SIRS), quick Sequential Organ Failure Assessment (qSOFA), National early warning score (NEWS), and the Ramathibodi early warning score (REWS); using area under the receiver operating characteristic AuROC curve.

Informed consent was waived as the data were retrospectively collected and were anonymous. This study was approved by the Committee on Human Rights Related to Research, Faculty of Medicine Ramathibodi Hospital, Mahidol University. (IRB COA. MURA2021/434 Date May 20, 2021.

2.2 Participants

We enrolled the older ED patients who were suspected of sepsis in emergency department during 1 August 2018 – 31 December 2018 (Figure 1). The older patients were aged equal to or more than 60 years (3), suspected sepsis case was defined as a patient who had the chief compliant of fever or had signs of infection (3, 7, 10-12). The diagnosis of sepsis was made according to the International Classification of Diseases, Tenth Revision (ICD 10) or on the results of blood or body fluid culture (7, 10-12). Sepsis-induced hypotension was defined as Mean Arterial Pressure (MAP) < 65 mm Hg in patients diagnosed with sepsis, which responded (≥65 mm Hg) to initial fluid therapy. The diagnosis of septic shock in patients who were diagnosed with sepsis was made based on requirement of vasopressor for maintaining MAP ≥ 65 mm Hg (13).

We excluded patients who had received treatment at outpatient unit or another hospital before transfer to Emergency Department, the patients who had previously received cardiopulmonary resuscitation in the same visit, and patients with missing data in the database of electronic medical record (EMR). The point of prediction was the time of patient’s visit at triage and performing point-of-care lactate test during initial assessment in ED.

2.3 Data collection

We selected baseline characteristics and potential predictive factors from our pilot unpublicized data. We collected them from electronic medical records including age, gender, comorbidities such as hypertension, diabetes mellitus, malignancy, pulmonary problems (defined as either chronic pulmonary disease or interstitial disease), dependent status (defined as at least one dependency to perform activities of daily living (ADL)), the initial vital signs at triage in emergency department, such as systolic blood pressure (SBP), MAP, heart rate (HR), temperature, respiratory rate (RR), oxygen saturation (SpO2), consciousness, and initial serum lactate, based on the performed point-of-care test (POCT). We calculated SIRS, qSOFA, NEWS and REWS scores.

2.4 Outcomes

All patients were followed for 28-day mortality, which is defined as all-cause mortality within 28 days of disease. We also followed other outcomes including sepsis treatment bundle in emergency department, ED disposition (intensive care unit (ICU), Ward, discharge, referral, death), and length of hospital stay.

2.5 Statistical analyses

Descriptive statistics were calculated for all clinical characteristics and relevant variables. Continuous variables are presented as mean and standard deviation (SD) in data with a normal distribution or median in non-parametric tests, and were analyzed using independent t-test or Mann–Whitney U test. Categorical data are presented as percentages and were analyzed using chi-square test or Fisher’s exact test, as appropriate. All tests were two-sided, and values were considered to be statistically significant with a P-value less than 0.05. We performed all data analyses using Stata version 16 (StataCorp LLC, College Station, TX, USA).

The sample size of this study was calculated based on our pilot data, 329 patients were included (9), on AUC = 0.86 with 8 parameters (comorbid malignancy, dependent status, body temperature, respiratory rate, altered mental status, oxygen saturation, heart rate, and lactate), assuming 0.05 acceptable difference in apparent & adjusted R-squared, assuming 0.05 margin of error in estimation of intercept. Events per Predictor Parameter (EPP) assumes prevalence of 28-day mortality = 0.11 in the STATA statistical program. Minimum sample size required for new model development was 357 with 40 events.

2.6 Model development

Initial predictors of sepsis mortality were chosen based on well-known sepsis scoring systems such as SIRS, qSOFA, NEWS and REWS scores, which include SBP <90 mmHg, MAP < 65 mmHg, HR ≤49, 50-119, ≥120 beat per minute (bpm), body temp ≤36.0, 36.1-38.4, ≥38.5 Celsius, RR≥24 bpm, oxygen saturation≤93% and altered consciousness. The cut point of each parameter was modified from 4 sepsis scores. We also included candidate predictors from our previous data and recent literature (14-17) in the developed model, including age of 80 years and older, being male, comorbidities such as hypertension, diabetes mellitus, malignancy, pulmonary, dependent status, and lactate≥4 mmol/L. We performed exploratory analysis for 15 potential predictors by using univariable logistic regression then reported odds ratio (OR); 95% confidence interval (CI) and P value, separately, for each variable. Multivariable logistic regression analysis was applied to identify an independent predictor of 28-day mortality. Each predictor was categorized at a generally accepted cutoff point, according to the previous literature. We used a backward elimination approach. The removal of non-contributing predictors was based on clinical relevancy and statistical significance. The reduced multivariable model was evaluated in terms of calibration and discrimination. Measurement of calibration was done with Hosmer-Lemeshow goodness of fit statistics.

2.7 Score derivation and validation

Each final predictor was assigned with a specific score derived from logistic regression coefficients of the multivariable model. The regression coefficient of each item was divided by the lowest coefficient, then rounded up to the nearest integer. The total score was then categorized into 2 risk groups (low and high risk) for applicability in clinical practice. Sensitivity, specificity, and positive likelihood ratio (LR+) were calculated to present predictive performance separately for each risk category. The measurement of calibration and discrimination was also performed via score-based multivariable logistic model. Predictive performance was validated and compared between newly-derived risk score and the other non-parametric receiver operating characteristic (ROC) regression with 1,000 bootstrapped replicates.

3. Results:

3.1 Baseline characteristics of studied cases

In this study, 605 older suspected sepsis patients were studied, 6 of whom were excluded because of receiving treatment at out-patient unit before transfer. The remaining 599 patients with the mean age of 77.13 (range: 60-101) years (56.43% male) met the eligibility criteria (7.01% 28-day mortality rate). 572 patients were diagnosed with sepsis, 27 patients were diagnosed with other causes. The sepsis group had 41 patients with non-survival on day 28. The non-sepsis group had 1 patient with non-survival on day 28, the patients’ flow chart is illustrated in Figure 1.

Candidate prognostic factors of older suspected sepsis patients in emergency department were shown in Table1, compared based on non-survivor and survivor status at day 28. Male gender (38.10% vs. 57.81%, p = 0.02) and hypertension (45.24% vs. 69.48%, p = 0.002) had significantly lower proportions among 28-day non-survivors. Malignancy (59.52% vs. 23.70%, p <0.01) and dependent status (78.57% vs. 38.60%, p <0.001) had significantly higher proportions in 28-day non-survivors. Heart rate ≤49 bpm (2.38% vs. 0.18%, p =0.14); 50-119 bpm (54.76% vs. 81.51%, p <0.001); and ≥120 bpm (42.86% vs. 18.31%, p <0.001) had higher proportions in 28-day non-survivors. RR≥24 bpm, oxygen saturation≤93%, alteration of consciousness, and lactate ≥4 mmol/l had significantly higher proportions in 28-day non-survivors than 28-day survivors (73.81% vs. 48.65%, <0.002), (59.52% vs. 27.29%, p <0.001), (50.00% vs. 20.83%, <0.00), (28.57% vs. 11.67%, p <0.01), respectively. Persons aged 80 years and older, comorbid with diabetes mellitus, pulmonary problems, SBP <90 mmHg, MAP < 65 mmHg, and temperature were similar between 28-day non-survivors and 28-day survivors.

3.2 Model development

Fifteen candidate predictors were simultaneously explored under univariable and multivariable logistic regression (Table 2). After backward elimination of non-contributive and non-significant predictors one by one, statistically significant predictors with P value of less than 0.100 were hypertension, malignancy, dependent status, HR ≤49 bpm; 50-119 bpm, ≥120 bpm, oxygen saturation≤93%, lactate≥4 mmol/l. We excluded hypertension from the analysis because there is insufficient literature supporting it as a protective factor (OR = 0.41). Including such variables might complicate the model by introducing negative associations with predicted mortality. We added respiratory rate ≥24 bpm and alteration of consciousness to the model due to clinical relevancy. Seven predictors (Table 2) were left in the final reduced model, which were malignancy comorbidity, dependent status, HR ≤49 bpm; 50-119 bpm, and ≥120 bpm, respiratory rate ≥24 beat per minute, oxygen saturation≤93%, alteration of consciousness, and lactate≥4 mmol/l. The logit coefficient of each predictor was used as a weight for score transformation. We had assigned a weighted score to the predictor as followes: 1 point for patients who had RR≥24 bpm, alteration of consciousness, and lactate≥4 mmol/l; 2 points for patients who had oxygen saturation≤93%, HR≥ 120 bpm, and dependent status; 3 points for patients who had comorbid malignancy; and 5 points for patients who had HR≤ 49 bpm. The newly-derived risk score was named Ramathibodi older sepsis score (ROSS). The ROSS ranged from a minimum of 0 point to a maximum of 15 points (Table 3). Sensitivity, specificity, and AUC of prediction of 28-day mortality for each score are reported in Table 4.

The ROSS could predict the 28-day mortality of older suspicious sepsis with good discriminative ability (AuROC: 0.87, 95% CI: 0.82-0.92- 0.90), which was significantly higher when compared with SIRS (AuROC: 0.62, 95% CI: 0.53 - 0.71; P <0.01), qSOFA (AuROC: 0.72, 95% CI: 0.66 - 0.79; P <0.01), NEWS (AuROC: 0.74, 95% CI: 0.67 - 0.82; P <0.01), and REWS (AuROC: 0.71, 95% CI: 0.62 - 0.80; P <0.01; Figure 2; supplementary table 1).

After using cut point of low-high risk, ROSS ≥6 had moderate discriminative ability (AuROC: 0.76, 95% CI: 0.69 - 0.83) but still higher than SIRS≥2 (AuROC: 0.55, 95% CI: 0.49 - 0.61; P <0.01), qSOFA≥2 (AuROC: 0.65, 95% CI: 0.57 - 0.72; P <0.01), NEWS≥5 (AuROC: 0.61, 95% CI: 0.57 - 0.66; P <0.01), and REWS≥4 (AuROC: 0.61, 95% CI: 0.55 - 0.68; P <0.01; Supplementary figure 1).

3.3 Measures of calibration

Measures of calibration were visualized through calibration plot, which showed that ROSS predicted 28-day mortality of older suspected sepsis patients and the observed risk of older suspected sepsis patients in the derivation cohort increased concomitantly (Supplementary figure 2). Hosmer-Lemeshow goodness of fit statistics also showed a non-significant P of 0.93. We performed internal validation of the score via nonparametric ROC with 1,000 bootstrap sampling, internal validation showed an apparent area under the ROC curve of 0.88 (95% CI: 0.83-0.92) with model optimism at 0.03 (range from 0.02 -0.11) C-statistic, Calibration in the large (CITL), and shrinkage factors indicated good calibration performance (Supplementary table 2).

The ROSS score predicted range of 28-day mortality probability from 7.01% to 66.67%. The score was categorized into low-risk and high-risk groups at the cut-off point of 6. Patients with ROSS score lower than 6 would be classified as low-risk patients, in whom 28-day mortality was 2.73% (13/477). Patients with higher ROSS score equal to or higher than 6 would be classified as high-risk patients, in whom 28-day mortality was 23.77% (29/122) with 69.0% (52.9-82.4%) sensitivity, 83.3% (79.9-86.3%) specificity, and 4.14 (95% CI: 3.14-5.44) positive likelihood ratio.

Table 1 Comparing the baseline characteristics of studied sepsis patients between cases with and without 28-day survival

Variable	Total
( n = 599 )	28-day survival	P value	
No ( n = 42 )	Yes ( n = 557 )	
Age (year)					
≥ 80	259 (43.24)	18 (42.86)	241(43.27)	1.00	
Gender					
Male	338 (56.43)	16 (38.10)	322 (57.81)	0.02	
Comorbidities					
Hypertension	406 (67.78)	19 (45.24)	387 (69.48)	0.002	
Diabetes mellitus	229 (38.23)	11 (26.19)	218 (39.14)	0.10	
Malignancy	157 (26.21)	25 (59.52)	132 (23.70)	<0.01	
Pulmonary disease	125 (20.87)	13 (30.95)	112 (20.11)	0.11	
Dependent status					
Yes	248 (41.40)	33 (78.57)	215 (38.60)	<0.001	
Blood pressure (mmHg)				
Systolic <90	50 (8.35)	6 (14.29)	44 (7.90)	0.15	
MAP < 6	45 (7.51)	4 (9.52)	41 (7.36)	0.55	
Heart rate (/minute)					
50-119	477 (79.63)	23 (54.76)	454 (81.51)	<0.001	
≤49	2 (0.33)	1 (2.38)	1 (0.18)	0.14	
≥120	120 (20.03)	18 (42.86)	102 (18.31)	<0.001	
Temperature (Celsius)					
36.1-38.4	371(61.94)	27 (64.29)	344 (61.76)	0.87	
≤36.0	25 (4.17)	2 (4.76)	23 (4.13)	0.69	
≥38.5	224 (37.40)	14 (33.33)	210 (37.70)	0.62	
Respiratory rate (/minute)			
≥24	302 (50.42)	31 (73.81)	271 (48.65)	<0.002	
Oxygen saturation (%)					
≤93	177 (29.55)	25 (59.52)	152 (27.29)	<0.001	
Alteration of consciousness			
Yes	137 (22.87)	21 (50.00)	116 (20.83)	<0.001	
Lactate (mmol/l)					
≥4	77 (12.85)	12 (28.57)	65 (11.67)	0.006	
Data are presented as frequency (%). MAP: Mean arterial pressure.

Table 2 Independent predictive factors of 28-day mortality among older sepsis patients in emergency department (n=599)

Variables	Crude OR
[95%CI]	P value	Multivariable OR
[95%CI]	P value	
Age (year)					
≥ 80	0.98 [0.52,1.85]	0.96	0.76 [0.31,1.80]	0.53	
Gender					
Male	0.45[0.23,0.86]	0.02	0.55[0.25,1.21]	0.14	
Comorbidities					
Hypertension	0.36 [0.19,0.68]	0.002	0.41 [0.17,0.98]	0.05	
Diabetes mellitus	0.55 [0.27,1.12]	0.10	1.10 [0.42,2.92]	0.84	
Malignancy	4.73 [2.48,9.04]	<0.001	4.90 [2.19,10.99]	<0.001	
Pulmonary disease	1.78 [0.90,3.54]	0.10	1.31 [0.55,3.13]	0.53	
Dependent status					
Yes	5.83 [2.73,12.43	<0.001	4.60 [1.67,12.68]	0.003	
Blood pressure (mmHg)			
Systolic <90	1.94 [0.78,4.87]	0.16	4.04 [0.37,44.59]	0.25	
MAP < 65	1.32 [0.45,3.89]	0.61	0.24 [0.02,2.13]	0.28	
Heart rate (/minute)			
50-119	0.27 [0.14,0.52]	<0.001	base		
≤49	13.56[0.83,220.76]	0.07	29.32[1.11,770.42]	0.04	
≥120	3.35 [1.75,6.39]	<0.001	2.58 [1.10-6.04]	0.03	
Temperature (Celsius)				
36.1-38.4	1.11 [0.58,2.14]	0.75	base		
≤36.0	1.16 [0.26,5.10]	0.84	1.49 [0.28,7.94]	0.64	
≥38.5	0.83 [0.43,1.60]	0.57	1.03 [0.44,2.44]	0.94	
Respiratory rate (/minute)				
≥24	2.97 [1.46,6.04]	0.003	2.04 [0.84,4.99]	0.12	
Oxygen saturation (%)				
≤93	3.92 [2.05,7.46]	<0.001	3.21 [1.42,7.24]	0.005	
Alteration of consciousness			
Yes	3.80 [2.01,7.20]	<0.001	2.15 [0.87,5.28]	0.09	
Lactate (mmol/l)				
≥4	3.02 [1.47,6.20]	0.002	2.09 [0.83,5.24]	0.12	
OR: Odds ratio; CI: confidence interval; MAP: Mean arterial pressure.

Table 3 Regression coefficient for 28-day mortality derived from generalized linear model and scoring scheme for predicting 28-day mortality

Indicator	Co*	OR	95%CI	P-value	T Co #	Score &	
Alteration of consciousness				
Yes	0.676	1.97	0.84-4.60	0.12	1.05	1	
O 2 Saturation							
≤93%	1.15	3.16	1.47-6.78	0.003	1.79	2	
Heart rate (/minute)		 	 		
50-119	base	base				0	
≤ 49	3.09	21.96	1.06-453.44	0.05	4.81	5	
≥ 120	1.15	3.14	1.44-6.86	0.004	1.78	2	
Lactate (mmol/l)							
≥4	0.75	2.12	0.91-4.96	0.08	1.17	1	
Respiratory rate (/minute)					
≥24	0.64	1.90	0.82-4.40	0.13	1	1	
Comorbid disease					
Malignancy	1.70	5.49	2.58-11.67	<0.001	2.65	3	
Dependent status					
Yes	1.45	4.27	1.67-10.92	0.002	2.26	2	
OR: Odds ratio; CI: confidence interval; *Co: Coefficients; #T Co: Transformed Coefficients; &: assigned score.

Table 4 Screening performance characteristics of Ramathibodi older sepsis score (ROSS) for 28-day mortality of older sepsis patients in different cut-off points [95%CI]

Score	Mortality rate	Sensitivity	Specificity	LR +	LR -	AUC	
≥0	42/599 (7.0)	100.00
[91.59-100]	0.00
[0-0.66]	NA	NA	 NA	
≥1	42/514 (8.2)	100.00
[91.59-100]	15.26
[12.37-18.52]	1.18
[1.14-1.22]	0.00	 0.58
[0.56-0.59]	
≥2	42/458 (9.2)	100.00
[91.59-100]	25.31
[21.75-29.14]	1.34
[1.28-1.41]	0.00	 0.63
[0.61-0.64]	
≥3	42/389 (10.8)	100.00
[91.59-100]	37.70
[33.66-41.87]	1.61
[1.50-1.71]	0.00	 0.50
[0.67-0.71]	
≥4	39/260 (15.0)	92.86
[80.52-98.50]	60.32
[56.12-64.41]	2.34
[2.05-2.67]	0.12
[0.04-0.35]	 0.77
[0.72-0.81]	
≥5	35/260 (17.9)	83.33
[68.64-93.03]	71.27
[67.32-75.00]	2.90
[2.40-3.50]	0.23
[0.12-0.46]	 0.77
[0.71-0.83]	
≥6	29/122 (23.8)	69.05
[52.91-82.38]	83.30
[79.94-86.31]	4.14
[3.14-5.44]	0.37
[0.24-0.58]	 0.76
[0.69-0.83]	
>7	23/64 (35.9)	54.76
[38.67-70.15]	92.64
[90.15-94.67]	7.44
[4.97-11.13]	0.49
[0.35-0.68]	 0.74
[0.66-0.81]	
≥8	19/43 (44.2)	45.24
[29.85-61.33]	95.69
[93.66-97.22]	10.50
[6.28-17.55]	0.57
[0.43-0.75]	 0.70
[0.63-0.78]	
≥9	15/25 (60.0)	35.71
[21.55-51.97]	98.20
[96.72-99.14]	19.89
[9.53-41.53]	0.65
[0.52-0.82]	0.67
[0.60-0.74]	
≥10	8/14 (57.1)	19.05
[8.60-34.12]	98.92
[97.67-99.60]	17.68
[6.43-48.60]	0.82
[0.71-0.95]	0.59
[0.53-0.65]	
≥11	2/3 (66.67)	4.76
[0.58-16.16]	99.82
[99.00-100.00]	26.52
[2.46-286.55]	0.95
[0.89-1.02]	0.52
[0.49-0.56]	
CI: confidence interval; LR+: positive likelihood ratio; LR-: negative likelihood ratio; AUC: area under the curve; NA: not available due to no value for some indicator.

Figure 1 Patients’ flow chart. OPD: outpatients’ unit; ED: emergency department; CPR: cardiopulmonary resuscitation.

Figure 2 Area under the ROC curves of different models in predicting 28-day mortality. ROC: receiver operating characteristic; ROSS: Ramathibodi older sepsis score; SIRS: Systemic Inflammatory Response Syndrome; qSOFA: quick Sequential Organ Failure Assessment; NEWS: National early waring score; REWS: Ramathibodi early warning score.

4. Discussion

Mortality in old and very old patients suffering from sepsis were 11-13%, while septic shock was 30-40% (18) . A sepsis severity score that helps in the stratification of high-risk patients in the ED may improve morbidity and mortality. Although outcome studies of older sepsis in emergency department were limited, several sepsis scores had been developed for use in general population, especially in the ICU setting (19). The well-known scores (SIRS, qSOFA, NEWS, REWS) were easy to calculate and readily available in the ED (20) but had poor ability for risk stratification in predicting mortality in older patients (6, 7). In our cohort study, the population from the emergency department may have had a lower risk profile compared to inpatients or those in the intensive care unit. The overall 28-day mortality rate among older patients with suspected sepsis was only 7%, which aligns with mortality rates observed in the general sepsis population (ranging from 4-12%)(21, 22).

Our study demonstrated the developed prediction score and compared it with the well-known scores. The ROSS had the best predictive ability for 28-day mortality in suspicious older sepsis with statistical significance followed by NEWS, qSOFA, REWS and SIRS. The ROSS is composed of 7 parameters “ABCDDDH”; A-alteration of consciousness, B-blood lactate ≥ 4 mmol/L, C-cancer, D-dyspnea (RR≥24), D-desaturation (oxygen ≤ 93%), D-dependent status, and H-HR while the NEWS is composed of 6 acute physiologic parameters including RR, oxygen saturation, temperature, SBP, HR, and level of consciousness with multiple-range score on 7 categories for each parameter, which was not easy to use, just like REWS. The qSOFA and SIRS are composed of less predictors; however, they were inferior to ROSS in prediction of 28-day mortality in older suspected sepsis patients.

Our data is in concordance with the study by de Groot et al. (6) that reported poor prognostic performance of predicted mortality for NEWS, qSOFA, modified early warning score (MEWS) (ROC: 0.57 (95%CI: 0.50–0.64), 0.60 (95%CI: 0.53–0.66), 0.56 (95%CI: 0.49–0.63)), and that by Boonmee P et al., which reported poor prognostic performance of qSOFA regarding predicted mortality (ROC: 0.55 (95%CI: 0.49–0.61)) (7) .

Heart rate ≤49 bpm was the strongest parameter in ROSS. Also, it was the parameter that had higher score in NEWS and REWS using the cut point HR ≤39 bpm for 3 points, 40-49 bpm for 1 point, 100-109 bpm for 1 point, 110-129 bpm for 2 points, and ≥130 bpm for 1 point. Tachycardia was associated with poor prognosis in sepsis. However, Beesley et al. (23) reported that relative bradycardia (approximate HR of 50-80 bpm) in patients with septic shock is associated with lower mortality. Bradycardia showed a stronger, poorer outcome than tachycardia.

Malignancy had the second highest score with 3 points in ROSS. Patients with malignancy had more than ten times higher risk of mortality than the general population in sepsis according to the cancer types (24). However, the population with malignancy is heterogeneous if we predicted mortality stratified by type of cancer prognosis could be more accurately predicted.

Dependent status was another parameter that was added to the physiologic warning score with 2 points. Dependent status was important in Geriatric-quick SOFA score (25) and other previous studies (15, 26) for predicting mortality in older patients. Dependent status might be more important than comorbid illness and was an independent predictor of outcome in older patients (15) .

Oxygen saturation≤93% had 2 point in ROSS. The data supported the association of oxygen saturation 94%-98% with favorable outcomes (27). Hypoxemia was associated with increased mortality, while hyperoxia may be associated with a higher mortality (28). Lactate cut point ≥4 mmol/l was a warning parameter (29). Point-of-care lactate was accurate, efficient, timeless in sepsis patients; additionally, it was available in ED and Emergency Medical Services (EMS) (30). Lactate≥4 mmol/l also increased prognostic performance in ROSS model compared with the well-known physiologic sepsis scores.

High respiratory rate and alteration of consciousness were included in SIRS, NEWS and REWS. Current data supported that they were associated with sepsis-related mortality (31-33). They were in concordance with the ROSS model. We modified the cut-point of Respiratory rate ≥24 bpm from cut-point of NEWS and REWS. Yang Y et al. reported that alteration of consciousness or sepsis associated encephalopathy had excellent performance in predicting risk of sepsis mortality (33). Preventing sepsis-associated encephalopathy, early detection, and appropriate specific treatments towards encephalopathy improved the patient’s prognosis.

The ROSS was categorized into low and high risk at the cut-off point of 6 to be stratified on risk of sepsis mortality for clinicians. Within 1 hour from presentation, bundle sepsis treatment, including fluid resuscitation, hemoculture, antimicrobial treatment, and lactate measurement, was applied to all patients with suspected sepsis in ED, then we monitored and observed the clinical response. In low-risk patients, if they had good response, and no abnormal physiologic parameter was present we would observe them in short stay, 48-72 hours, or discharge them from ED with early follow-up within 72 hours. In high-risk patients, they should be closely monitored, admitted to Indoor Patients Department (IPD) or ICU, and receive prompt resuscitation if they were deteriorating.

6. Conclusion:

Our cohort study developed the Ramathibodi older sepsis score (ROSS) based on seven independent predictors. The ROSS score was shown to have good discriminative ability, with easy to use and readily available components. The ROSS score could be practically applied for risk stratification in predicting 28-day mortality in older sepsis patients in ED.

Supplementary Materials

Supplementary Table Pairwise test comparing the area under the ROC curves of each two sepsis scores for predicting 28-day mortality in older suspected sepsis patients

	ROSS	SIRS	qSOFA	NEWS	REWS	
ROSS		P <0.01	P <0.01	P <0.01	P <0.01	
SIRS	P < 0.01		P = 0.03	P < 0.01	P = 0.01	
qSOFA	P < 0.01	P = 0.03		P = 0.58	P = 0.78	
NEWS	P < 0.01	P < 0.01	P = 0.58		P = 0.37	
REWS	P < 0.01	P = 0.01	P = 0.78	P = 0.37		
ROC: receiver operating characteristic; ROSS: Ramathibodi older sepsis score; SIRS: Systemic Inflammatory Response Syndrome; qSOFA: quick Sequential Organ Failure Assessment; NEWS: National early waring score; REWS: Ramathibodi early warning score.

Supplementary figure 1 Area under the ROC curves of different models for predicting 28-day mortality in their best cut-off points (ROSS≥6, SIRS≥2, qSOFA≥2, NEWS≥5, REWS≥4). ROC: receiver operating characteristic; ROSS: Ramathibodi older sepsis score; SIRS: Systemic Inflammatory Response Syndrome; qSOFA: quick Sequential Organ Failure Assessment; NEWS: National early waring score; REWS: Ramathibodi early warning score.

Supplementary figure 2 Score-predicted risk (line) and actual risk (circles) of 28-day mortality for each total score.

Supplementary table 2 Internal validation model calibration parameters with 995 bootstrapping replications

Performance	C-statistic	CITL	Shrinkage factor	
Apparent	0.88 (0.83-0.92)	0.00 (-0.36 - 0.36)	0.91 (0.75-1.25)	
Bootstrap	0.85 (0.81-0.90)	0.004 (-0.37 - 0.40)	0.89 (0.68-1.14)	
Data are presented with 95% confidence intervals. CITL: Calibration in the large.

5. Strengths and limitation

The ROSS was composed of patient’s history (malignancy, dependent status), physiologic parameters (HR, RR, oxygen saturation, consciousness), and point-of-care laboratory (lactate). The ROSS score was superior to qSOFA and other warning scores by added value of malignancy, dependent status, and lactate. The independent prognosis factors were simple and available in the initial phase of ED visit. Early categorization of sepsis prognosis in ED with appropriate monitoring and treatment may improve morbidity and survival outcome.

There are some limitations to this study. First, our study was retrospective and conducted in a single, university hospital. Thus, the results may not be generalizable. Second, we used the population of patients aged ≥60 for the older definition. Third, because we aimed to use simple and uncomplicated factors at ED presentation, we might have eliminated some comprehensive independent factors of older patients, which were used in IPD or ICU based on previous publications. Fourth, using different definitions for the suspected sepsis patients, may have led to selection or misclassification bias. Fifth, the ROSS model might be difficult to use in situations of over-crowning of ED or insufficient health care providers. We should develop an application for easy and flexible use of the ROSS.

We have planned further studies for external temporal validation of the ROSS in predicting 28-day mortality of older sepsis patients before applying it in clinical practice.

7. Declarations:

7.1 Acknowledgments

We wish to thank all the medical and nursing staff of the Emergency department of Ramathibodi Hospital for their collaborations. We thank Edanz Group (https://en-author-services.edanz.com/ac) for editing a draft of this manuscript. Part of this study was chosen to present as poster presentation in 42nd International symposium on intensive care & Emergency Medicine (ISICEM); meeting 21-24th March 2023. The abstract has been published in Meeting abstract 42nd International Symposium on Intensive Care & Emergency Medicine Critical Care (2023) 27:119 https://doi.org/10.1186/s13054-023-04377-x.

7.2 Authors' contributions

All of the authors significantly contributed to the idea, planning, implementation, data collection, analysis, and interpretation of the study. The article's creation, revision, or critical evaluation involved input from all authors. All authors agreed on the journal to which the manuscript was submitted, granted their final approval of the version to be published, and agreed to take responsibility for every part of the work.

7.3 Funding

No funding was obtained for this study.

7.4 Availability of supporting data

The data supporting this study’s findings are openly available in [Harvard Dataverse]:  "Clinical prediction score development for predicting 28-day mortality of elderly sepsis patients in emergency department", https://doi.org/10.7910/DVN/X3

7.5 Ethical approval and consent to participate.

This study was approved by The Committee on Human Rights Related to Research, Faculty of Medicine, Ramathibodi Hospital, Mahidol University (IRB COA. MURA2021/434 Date May 20, 2021). The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was waived as the data were retrospectively collected and were anonymous.

7.6 Consent for publication

Not applicable.

7.7 Conflict of interest

The authors declare that they have no competing interests.

7.8 Using artificial intelligence chatbots

During the preparation of this work the authors used ChatGPT/Open AI in order to check grammar and spelling. After using this service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.
==== Refs
References

1 bureau Pr 2012 World population data sheet 2012 Available from: https://www.prb.org/wp-content/uploads/2012/07/2012-population-data-sheet_eng.pdf
2 Singer M Deutschman CS Seymour CW Shankar-Hari M Annane D Bauer M The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA 2016 315 8 801 10 26903338
3 alliance.org gs WHAT IS SEPSIS? – DEFINITION OF SEPSIS 2021 [Available from: https://www.global-sepsis-alliance.org/sepsis
4 Ibarz M Haas LEM Ceccato A Artigas A The critically ill older patient with sepsis: a narrative review Ann Intensive Care 2024 14 1 6
5 Rudd KE Johnson SC Agesa KM Shackelford KA Tsoi D Kievlan DR Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study Lancet 2020 395 10219 200 11 31954465
6 de Groot B Stolwijk F Warmerdam M Lucke JA Singh GK Abbas M The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study Scand J Trauma Resusc Emerg Med 2017 25 1 91 28893325
7 Boonmee P Ruangsomboon O Limsuwat C Chakorn T Predictors of Mortality in Elderly and Very Elderly Emergency Patients with Sepsis: A Retrospective Study West J Emerg Med 2020 21 6 210 8 33207168
8 Zelis N Buijs J de Leeuw PW van Kuijk SMJ Stassen PM A new simplified model for predicting 30-day mortality in older medical emergency department patients: The rise up score Eur J Intern Med. 2020 77 36 43 32113943
9 Sanguanwit P Yuksen C Khorana J Phootothum Y Damdin S Sutham K Factors for predicting 28-day mortality in older patients with suspected of having sepsis in the emergency department Hong Kong Journal of Emergency Medicine 31 143 53
10 Bone RC Balk RA Cerra FB Dellinger RP Fein AM Knaus WA Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis The ACCP/SCCM Consensus Conference Committee Chest. 1992 101 6 1644 55 1303622
11 Levy MM Fink MP Marshall JC Abraham E Angus D Cook D 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference Crit Care Med 2003 31 4 1250 6 12682500
12 Lopansri BK Miller Iii RR Burke JP Levy M Opal S Rothman RE Physician agreement on the diagnosis of sepsis in the intensive care unit: estimation of concordance and analysis of underlying factors in a multicenter cohort J Intensive Care. 2019 7 13 30828456
13 Rhodes A Evans LE Alhazzani W Levy MM Antonelli M Ferrer R Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2016 45 3 486 552
14 Ittisanyakorn M Ruchichanantakul S Vanichkulbodee A Sri-On J Prevalence and factors associated with one-year mortality of infectious diseases among elderly emergency department patients in a middle-income country BMC Infect Dis 2019 19 1 662 31345168
15 Nasa P Juneja D Singh O Severe sepsis and septic shock in the elderly: An overview World J Crit Care Med 2012 1 1 23 30 24701398
16 Martin-Loeches I Guia MC Vallecoccia MS Suarez D Ibarz M Irazabal M Risk factors for mortality in elderly and very elderly critically ill patients with sepsis: a prospective, observational, multicenter cohort study Ann Intensive Care 2019 9 1 26
17 Evans L Rhodes A Alhazzani W Antonelli M Coopersmith CM French C Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021 Critical Care Medicine 2021 49 11
18 Bruno RR Wernly B Mamandipoor B Rezar R Binnebössel S Baldia PH ICU-Mortality in Old and Very Old Patients Suffering From Sepsis and Septic Shock Frontiers in Medicine 2021 8
19 Arabi Y Al Shirawi N Memish Z Venkatesh S Al-Shimemeri A Assessment of six mortality prediction models in patients admitted with severe sepsis and septic shock to the intensive care unit: a prospective cohort study Crit Care 2003 7 5 R116 22 12974979
20 Chen CC Chong CF Liu YL Chen KC Wang TL Risk stratification of severe sepsis patients in the emergency department Emerg Med J 2006 23 4 281 5 16549574
21 Prachanukool T Sanguanwit P Thodamrong F Suttapanit K The 28-Day Mortality Outcome of the Complete Hour-1 Sepsis Bundle in the Emergency Department Shock 2021 56 6 969 74 34779799
22 Suttapanit K Wisan M Sanguanwit P Prachanukool T Prognostic Accuracy of VqSOFA for Predicting 28-day Mortality in Patients with Suspected Sepsis in the Emergency Department Shock 2021 56 3 368 73 33577246
23 Beesley SJ Wilson EL Lanspa MJ Grissom CK Shahul S Talmor D Relative Bradycardia in Patients With Septic Shock Requiring Vasopressor Therapy Crit Care Med 2017 45 2 225 33 27618277
24 Taccone FS Artigas AA Sprung CL Moreno R Sakr Y Vincent JL Characteristics and outcomes of cancer patients in European ICUs Crit Care 2009 13 1 R15 19200368
25 Remelli F Castellucci F Vitali A Mattioli I Zurlo A Spadaro S Predictive value of geriatric-quickSOFA in hospitalized older people with sepsis BMC Geriatrics 2021 21 1 241 33849471
26 Jiang X Khan FA Ow MQ Poh HMN Sepsis in a Combined Medical and Surgical High Dependency/Intensive Care Unit in Singapore: A Cohort Study and Survival Analysis Int J Gen Med. 2022 15 4585 93 35535141
27 Girardis M Busani S Damiani E Donati A Rinaldi L Marudi A Effect of Conservative vs Conventional Oxygen Therapy on Mortality Among Patients in an Intensive Care Unit: The Oxygen-ICU Randomized Clinical Trial JAMA 2016 316 15 1583 9 27706466
28 Demiselle J Wepler M Hartmann C Radermacher P Schortgen F Meziani F Hyperoxia toxicity in septic shock patients according to the Sepsis-3 criteria: a post hoc analysis of the HYPER2S trial Ann Intensive Care 2018 8 1 90
29 Levy MM Evans LE Rhodes A The Surviving Sepsis Campaign Bundle: 2018 Update Critical Care Medicine 2018 46 6 997 1000 29767636
30 Ismail F Mackay WG Kerry A Staines H Rooney KD The accuracy and timeliness of a Point Of Care lactate measurement in patients with Sepsis Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 2015 23 1 68 26383239
31 Asiimwe SB Abdallah A Ssekitoleko R A simple prognostic index based on admission vital signs data among patients with sepsis in a resource-limited setting Critical Care 2015 19 1 86 25888322
32 Karlsson A Stassen W Loutfi A Wallgren U Larsson E Kurland L Predicting mortality among septic patients presenting to the emergency department-a cross sectional analysis using machine learning BMC Emerg Med 2021 21 1 84 34253184
33 Yang Y Liang S Geng J Wang Q Wang P Cao Y Development of a nomogram to predict 30-day mortality of patients with sepsis-associated encephalopathy: a retrospective cohort study Journal of Intensive Care 2020 8 1 45 32637121
