
==== Front
Kidney Int Rep
Kidney Int Rep
Kidney International Reports
2468-0249
Elsevier

S2468-0249(24)01811-4
10.1016/j.ekir.2024.06.034
Clinical Research
Validation of a Clinical Scale for Early Detection of Infections at the Exit Site of Central Venous Catheters for Hemodialysis
Cobo-Sánchez José Luis 123
Blanco-Mavillard Ian ian.blanco.sspa@juntadeandalucia.es
456∗
Pelayo-Alonso Raquel 12
Mancebo-Salas Noelia 7
Fernández-Fernández Ismael 4
Larrañeta-Inda Irene 8
Ulzurrun-García Ana 8
Sánchez-Villar Isidro 9
González-García Fernando 10
Hernando-García Julia 11
Rollán-de la Sota Ma Jesús 11
Vieira-Barbosa Lopes Luís Miguel 12
Prieto-Rebollo Ma del Rosario 13
Sesmero-Ramos Carolina 13
Jaume-Riutort Catalina 14
Casas-Cuesta Rafael 15
Alcántara-Crespo Mateo 15
Ernest de Pedro-Gómez Joan 56
1 Nursing Research Unit and Nephrology Department, Hospital Universitario Marqués de Valdecilla-IDIVAL, Santander, Spain
2 Immunopathology Research Group, Marqués de Valdecilla Institute for Health Research, Santander, Spain
3 University Nursing School Hospital Mompía, Universidad Católica de Ávila, Mompía, Spain
4 Implementation, Research and Innovation Unit, Hospital Regional Universitario de Málaga, Málaga, Spain
5 Department of Nursing and Physiotherapy, Universitat de les Illes Balears, Palma, Spain
6 Care, Chronicity and Evidence in Health Research Group, Health Research Institute of the Balearic Islands, Palma, Spain
7 Dirección General de Servicios Sociales, Consejería de Familia, Juventud y Política, Social, Comunidad de Madrid, Madrid, Spain
8 Nephrology Department, Complejo Hospitalario Universitario de Navarra, Pamplona, Spain
9 Nephrology Department, Hospital Universitario de Canarias, Santa Cruz de Tenerife, Spain
10 Nephrology Department, Hospital Universitario Gregorio Marañón, Madrid, Spain
11 Nephrology Department, Hospital Clínico Universitario de Valladolid, Valladolid, Spain
12 Haemodialysis Unit, Hospital Quirónsalud, A Coruña, A Coruña, Spain
13 Nephrology Department, Hospital Universitario Fundación Alcorcón, Alcorcón, Spain
14 Nephrology Department, Hospital de Manacor, Manacor, Spain
15 Nephrology Department, Hospital Universitario Reina Sofía de Córdoba, Córdoba, Spain
∗ Correspondence: Ian Blanco-Mavillard, Hospital Regional Universitario de Málaga, Implementation, Research, and Innovation Unit, Avda Carlos Haya s/n 29010 Málaga, Spain. ian.blanco.sspa@juntadeandalucia.es
28 6 2024
9 2024
28 6 2024
9 9 27392749
1 3 2024
12 6 2024
17 6 2024
© 2024 International Society of Nephrology. Published by Elsevier Inc.
2024
International Society of Nephrology
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Introduction

Exit-site infections (ESI) of central venous catheters for hemodialysis (CVC-HD) has been associated with early catheter removal and an increased risk of CVC-HD related bacteremia. No specific clinical scales to predict ESI have previously been validated.

Methods

A multicenter prospective cohort study was performed to validate the proposed scale, which is based on the following 5 signs and symptoms: (i) pain at exit site during interdialytic period; (ii) hyperemia or erythema ≥2 cm from exit site; (iii) inflammation, induration, or swelling at exit site; (iv) fever ≥38 °C not attributable to other causes, and (v) obvious abscess or purulent exudate at the exit site. Adult patients with a tunneled CVC-HD for at least 1 month after insertion has been included. During each hemodialysis session, the exit site was assessed with the proposed scale by nurses. If any item was present, a pericatheter skin swab culture was collected: positive results were gold standard. The scale was validated using receiver operating characteristic (ROC) curves and logistic regression analysis. For this purpose, the logit function was applied, and the ESI probability calculated, as elogit ESI/1 + elogit ESI.

Results

Three hundred thirty-seven CVC-HDs from 310 patients were analyzed, producing 515 cultures (117 infected and 398 healthy). The final version of the scale includes the following 3 signs and symptoms, which present the greatest predictive capacity: (i) pain at exit site during interdialytic period, (ii) hyperemia or erythema ≥2 cm from exit site, and (iii) abscess or purulent exudate at the exit site. The final version generated an area under the ROC curve (AUC) of 88.3% (95% confidence interval [CI]: 85.2%–91%; P < 0.001), Youden index 0.7557 ≈ 1, sensitivity 80.34% (95% CI: 71.36%–87.71%) and specificity 95.23% (95% CI: 92.73%–97%).

Conclusions

The validation shows that the scale has good predictive properties, detecting approximately 90% of ESI with very acceptable validity parameters.

Graphical abstract

Keywords

central venous catheter
early diagnosis
exit site infection
hemodialysis
reproducibility of results
validation study
==== Body
pmcWorldwide, more than 800 million people suffer from chronic kidney disease and 3 million are receiving kidney replacement therapy; the global prevalence of chronic kidney disease is estimated to be 7% to 12%,1,2 and the number of patients is expected to double by 2030.3 The treatment approach most commonly used is that of hemodialysis.4 In this context, the type of vascular access employed is closely related to patient morbidity and mortality. Thus, a patient receiving CVC-HD versus a fistula (whether autologous or prosthetic) is at 4 times greater risk of experiencing an infectious complication, and this risk is multiplied 10-fold when the sole vascular access is a CVC-HD.5, 6, 7, 8

The incidence of CVC-HD-related bacteremia is highly variable (ranging from 0.56 to 6.18/1000 catheter-days for tunneled catheters and from 1.4 to 8.3/1000 catheter-days for nontunneled catheters).9, 10, 11 Similarly, the incidence of ESI ranges from 0.31 to 8.3 per 1000 catheter-days for tunneled catheters and from 8.2 to 16.75 per 1000 catheter-days in nontunneled ones.9,10,12 The total cost of hospitalization for a catheter-related infection ranges from $17,000 to $32,000,1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 depending on the causative agent and the duration of treatment and hospitalization. The majority of this cost is related to catheter-associated bloodstream infections and their sequelae.

At present, clinical practice guidelines on vascular access for hemodialysis do not provide a universal definition regarding ESI4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and the 2019 update Kidney Disease Outcomes Quality Initiative of the vascular access guideline13 merely states that “Further validation studies of diagnostic criteria for exit site and tunnel infections in HD patients” are required. To our knowledge, no validated scales have yet been proposed to evaluate the exit site of CVC-HD. Therefore, we deem it necessary and timely to design and validate a scale for assessing the exit site of tunneled CVC-HD, which supports the recommendations of clinical practice guidelines. The aim of this study is to validate a clinical scale for the early detection of ESI during the assessment of tunneled CVC-HD.

Methods

The EXIT site Assessment (EXITA) study is designed to validate an instrument for the early detection of ESI in tunneled CVC-HD: the study protocol14 is based on the DeVellis recommendations.15 The scale was developed in the following 2 phases:1. Scale design. After conducting a scoping review of the literature16 to identify clinical signs and symptoms, they were categorized by an international panel of experts in CVC-HD exit site care, using the Delphi technique.17 The prioritization thus obtained was then used to develop the preliminary version of the proposed assessment scale.

2. Validation. A multicenter prospective cohort study was performed to determine the predictive properties of the ESI scale. The methodology of this validation phase and the results obtained are described below.

Study Design

In the validation phase, a multicenter prospective cohort study was carried out on a population of chronic patients receiving hemodialysis at the following 9 Spanish hospitals (the corresponding autonomous community or region is given in parentheses): Hospital Universitario Reina Sofía de Córdoba (Andalusia), Hospital Universitario de Canarias (Canary Isles), Hospital Universitario Marqués de Valdecilla (Cantabria), Hospital Clínico Universitario de Valladolid (Castilla y León), Hospital Quironsalud A Coruña (Galicia), Hospital de Manacor (Balearic Isles), Hospital General Universitario Gregorio Marañón and Hospital Universitario Fundación Alcorcón (Madrid), and Complejo Hospitalario de Navarra (Navarra). The participating centers are representative of the various types of hemodialysis units: peripheral centers, university hospitals, tertiary referral hospitals, and regional hospitals.

The Cantabria Ethics Committee for Research with Medical Products approved the study protocol14 in July 2019, under approval code 2019.146. A pilot study with version 1 of the scale was performed in May 2021. Patient recruitment took place from May 1, 2021, to June 30, 2022, during the validation period. The study data were analyzed, the predictive properties of the scale items reviewed, and the final version of the scale was decided upon from July to December 2022.

Participants

The following inclusion criteria were established: patient on kidney replacement therapy with hemodialysis via a tunneled CVC-HD for at least 1 month after insertion, aged ≥18 years, and provision of signed informed consent to participate in the study. Recruitment was consecutive and convenience-based, until the preestablished sample size was obtained, and each patient was followed-up with during the study period.

The sample size was calculated taking as a reference the mean 20% incidence of ESI in Spain.18, 19, 20 The expected sensitivity and specificity of the scale to be validated were 95% and 90%, respectively. Because no validation studies have been conducted of similar scales in vascular catheters, these expected properties are based on the parameters described by Eriguchi et al.21 following their validation of the exit site scoring system for peritoneal dialysis catheters, as recommended in the 2005 guidelines of the International Peritoneal Dialysis Society. We calculated that 92 patients should be included in the infection group to estimate the presence of ESI (92 exit sites with a positive culture), at a confidence level of 95% and a precision of ± 10 percentage units. We also calculated that 365 patients should be included in the noninfection group (365 exit sites with a negative culture), in order to estimate a healthy exit site (without infection), at a confidence level of 95% and a precision of ± 5 percentage units. The necessary sample size was calculated using Epidat 4.2 software.

Variables

Version 1 of the scale was constructed from the 9 signs and symptoms prioritized by the panel of experts in the first phase of the EXITA study,17 which our research team then grouped into the following 5 items:– Pain at exit site during interdialytic period.

– Hyperemia or erythema ≥2 cm from exit site.

– Inflammation, induration, or swelling at exit site.

– Fever ≥38 °C not attributable to other causes.

– Obvious abscess or purulent exudate at the exit site.

The first version of the scale was piloted with 10 patients at each participating hospital, evaluated by 3 nurses in every case, in May 2021. This initial approach was intended to obtain linguistic validation of the instrument and thus evaluate its understandability and clarity. This piloting showed that no change in the items was needed.

ESI determined by microbiological culture of skin smears or pericatheter exudate was taken as the gold standard. In addition, the patients’ clinical variables, catheter type, local CVC-HD maintenance policies and the outcomes of microbiological studies were recorded (Supplementary Table S1).

Data Collection

Version 1 was taken as the exit site assessment method, which was applied before each hemodialysis session (in which the exit site treatment protocol remained unchanged). To evaluate the exit site, the nurses were provided with a millimeter ruler magnifying glass, the items to be assessed, and a QR code with access to the hospital’s data collection record (Supplementary Figure S1). The nurses at the participating hospitals received training to ensure the unequivocal identification of the signs and symptoms to be validated, thus eliminating observer bias. For each CVC-HD, an initial control culture was obtained, upon the recruitment of each patient, based on a pericatheter skin smear from the healthy exit site (i.e., with no signs or symptoms of infection), as a control. When a relevant item was identified, a microbiological study of the pericatheter skin smear and/or exudate from the exit orifice (as appropriate) was performed. A pericatheter skin swab culture was repeated when the signs and symptoms disappeared, as a control. During the study period, each patient might present more than 1 positive or negative culture. If the patient had more than 1 CVC-HD during the study period, data corresponding to each catheter type were collected. Therefore, each catheter could have more than 1 culture during the follow-up period, always including a control culture (negative culture in the absence of signs and symptoms of infection). The cultures were collected by the nurse performing the hemodialysis session, before the session began. If patients were hospitalized due to an ESI or developed an ESI during hospitalization, the culture was collected in the same manner. A common protocol for culture collection was established across all centers. To obtain the samples, a dry cotton swab was rubbed over an area of 2 cm2 around the exit orifice, immediately after removing the dressing, without disinfecting the skin.22,23 If a local allergic reaction and/or bleeding were observed, no pericatheter skin smears were taken. ESI was assumed to be present when the culture results were positive, that is, ≥ 15 CFU/ml by the semiquantitative Maki technique or ≥ 1000 CFU/ml by the Cleri technique.22,23 Skin contamination (microbiological growth in the culture ≤15 CFU/ml by the semiquantitative Maki technique or ≤ 1000 CFU/ml by the Cleri technique22,23) was not considered infection and was not included in the statistical analysis.

Statistical Analysis

For each item present, 1 point was scored; its absence was scored as 0 points. The total score of the scale was calculated by adding the points for the presence or absence of the signs and symptoms to be validated (maximum 5 points in version 1). The total score was then subjected to exploratory descriptive analysis (central tendency, dispersion, skewness, and kurtosis) and univariate and multivariate tests of normality. For the decision validity analysis (predictive validity), the total score obtained was compared with the gold standard (microbiological culture of skin smears from the pericatheter or exit site exudate). The validity of the scale was assessed by calculating the specificity, the sensitivity, the positive and negative likelihood ratios (LH+, LH−), ROC curves, the AUC, and the 95% CI. An AUC value > 0.5 and close to 1 indicated a good level of predictability of the scale. In addition, the Youden index was calculated to optimize the cut-off point of the scale (Youden = sensitivity + specificity − 1).

The scale was validated by logistic regression analysis. The signs and symptoms of the scale in the multivariate analysis were selected according to a univariate analysis performed on each sign and symptom, following the selection criteria proposed by Hosmer, Lemeshow, and Sturdivant,24 in which the variables with a significance level < 0.25 were considered. In addition, the clinical relevance of each sign or symptom was considered, regardless of the statistical significance obtained. The β coefficients for each item on the scale to be validated were estimated by logistic regression.

The formula

Logit ESI = constant + β1 × Xi +βn × Xn

was then applied.

To transform the logit ESI into the probability of ESI being present, the following formula was used:

Probability of ESI = elogit ESI/1 + elogit ESI

The above procedure enabled us to determine the probability, expressed as a percentage, of a patient with CVC-HD presenting ESI, according to the presence or absence of the items to be validated.24,25 With this formula and the treatment approaches prioritized by the international panel of experts in phase 2 of the EXITA17study, an ESI risk calculator was generated (in Excel format), with the final version of the scale.

The statistical analyses were performed using IBM SPSS Statistics software (version 20.0) and MedCalc software (version 19.6).

Results

The study included 337 CVC-HDs, corresponding to 310 hemodialysis patients, of whom 39.4% were female, with a mean age of 67.78 ± 14.66 years (range 22–91 years) and a mean score on the modified Charlson comorbidity index for kidney disease of 5.1 ± 2.23 points (range 2–14 points). The characteristics of these CVC-HDs are detailed in Table 1. Five swabs had contamination and were excluded. A total of 515 exit site cultures were included; of these, 117 were positive and 398 were negative. Figure 1 shows a summary of the participants and cultures flow chart. Figure 2 shows individual patient participation in the study flowchart. Table 2 shows the characteristics of swabs collected.Table 1 Baseline characteristics of patients and catheters included in the cohort

Characteristic	n (%)	
Patients (n)	310	
Age (yr)	67.78 ± 14.66	
Charlson Index for patients with kidney disease (score)	5.1 ± 2.29	
Women,	132 (39.4)	
Etiology of kidney disease according to ERA/EDTA code		
Systemic	166 (49.6)	
Glomerular	62 (18.5)	
Other	55 (16.4)	
Tubulointerstitial	20 (6.0)	
Hereditary-family	17 (5.1)	
Various kidney disorders (nephrectomies. tumors, etc.)	15 (4.5)	
Number of catheters for patient (n)	337	
 1 catheter	314 (93.4)	
 2 catheters	14 (4.2)	
 3 catheters	3 (0.9)	
 4 catheters	4 (0.6)	
 5 catheters	1 (0.45)	
 6 catheters	1 (0.45)	
Catheters design		
 1 exit site	323 (96.4)	
 2 exit sites	12 (3.6)	
Insertion vein		
 Right jugular	279 (83.3)	
 Left jugular	34 (10.1)	
 Right subclavian	8 (2.4)	
 Left subclavian	7 (2.1)	
 Other	7 (2.1)	
Previous infections		
 Infection of the exit site	47 (14)	
 Bacteremia	23 (6.9)	
 Tunnel infection	7 (2.1)	
ERA/EDTA European Renal Association-European Dialysis and Transplant Association.

Figure 1 Summary participants and cultures flowchart. SS, Signs and symptoms.

Figure 2 Individual patient participation in the study flowchart.

Table 2 Characteristics of swabs collected

Characteristic	n (%)	
Swabs (n)	515	
Catheter’s swabs frequency		
1 swab	238 (70.6)	
2 swabs	54 (16)	
3 swabs	26 (7.7)	
4 swabs	10 (3)	
5 swabs	5 (1.5)	
6 swabs	3 (0.9)	
8 swabs	1 (0.3)	
Exit sites’ swabs frequency		
Healthy exit sites (without signs or symptoms)	391 (76)	
Clinically suspicious exit sites	124 (24)	

Supplementary Table S2 shows the variables related to exit site local policy of the CVC-HDs studied. In 63.3% of cases, the orifice was treated weekly, usually by washing with 0.9% physiological saline solution and followed (in 34.6% of cases) by the application of otic ciprofloxacin and covering it with a gauze dressing.

Of the 515 cultures performed, 75.9% presented no signs or symptoms and were taken as controls. The most common sign was hyperemia or erythema ≥ 2 cm from the exit site (15%), followed by obvious abscess or purulent exudate at the exit site (Supplementary Table S3). Supplementary Table S4 shows the frequency of combinations of signs and symptoms in the sessions in which cultures were collected. Among the 117 positive cultures, 18 different microorganisms were identified, the most common of which were Staphylococcus epidermidis (45.86%) and Staphylococcus lugdunensis (12.03%) (Supplementary Table S5).

Supplementary Table S6 shows the validity properties of each item on the initial version of the scale. The total score for this version, including the 5 items, presented the following characteristics: AUC 0.885 (88.5%; 95% CI: 85.4%–91.1%; standard error: 0.0191; bootstrap 95% CI: 84%–91.6%; z statistic: 22.953), Youden index 0.7557 ≈ 1, sensitivity 80.34% (95% CI: 72.0%–87.1%), specificity 95.23% (95% CI: 92.6%–97.1%), LR +: 16.83 (95% CI: 10.75–26.34), LR−: 0.21 (95% CI: 0.14 LR−:0.30). The ROC curve for the total score of version 1 of the scale is shown in Supplementary Figure S2, and its characteristics in Tables 3 and 4.Table 3 Sensitivity, specificity, and likelihood ratios of versions 1, 2A, and 2B of scale for predicting exit site infection of central venous catheters for hemodialysis

Total score	Sensitivity (95% CI)	Specificity (95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio (95% CI)	Positive predictive value (95% CI)	Negative predictive value (95% CI)	
Version 1	
≥ 0.053822444	100.00 (96.9–100)	0.00 (0.0–0.9)	1.00 (1.00–1.00)		22.7 (22.7–22.7)	–	
> 0.053822444	100.00 (96.9–100)	1.26 (0.4–2.9)	1.01 (1.00–1.02)	0.00 (-)	22.9 (22.7–23.1)	100.0	
> 0.065018712a	80.34 (72.0–87.1)	95.23 (92.6–97.1)	16.83 (10.75–26.34)	0.21 (0.14–0.30)	83.2 (76.0–88.6)	94.3 (91.9–96.0)	
> 0.202270873	77.78 (69.2–84.9)	95.48 (92.9–97.3)	17.20 (10.84–27.29)	0.23 (0.17–0.33)	83.5 (76.1–88.9)	93.6 (91.2–95.4)	
> 0.705746946	76.07 (67.3–83.5)	95.73 (93.2–97.5)	17.81 (11.06–28.67)	0.25 (0.18–0.35)	84.0 (76.5–89.4)	93.2 (90.8–95.0)	
> 0.710928773	58.97 (49.5–68.0)	97.49 (95.4–98.8)	23.47 (12.50–44.09)	0.42 (0.34–0.52)	87.3 (78.6–92.8)	89.0 (86.7–91.0)	
> 0.746177556	55.56 (46.1–64.7)	97.74 (95.8–99.0)	24.57 (12.62–47.82)	0.45 (0.37–0.56)	87.8 (78.8–93.4)	88.2 (85.9–90.2)	
> 0.782316442	28.21 (20.3–37.3)	98.99 (97.4–99.7)	28.06 (10.15–77.60)	0.73 (0.65–0.81)	89.2 (74.9–95.8)	82.4 (80.7–84.0)	
> 0.897385536	27.35 (19.5–36.4)	98.99 (97.4–99.7)	27.21 (9.82–75.38)	0.73 (0.66–0.82)	88.9 (74.3–95.7)	82.3 (80.6–83.8)	
> 0.899672339	26.50 (18.8–35.5)	98.99 (97.4–99.7)	26.36 (9.50–73.16)	0.74 (0.67–0.83)	88.6 (73.6–95.6)	82.1 (80.4–83.6)	
> 0.929097431	22.22 (15.1–30.8)	99.75 (98.6–100.0)	88.44 (12.13–644.87)	0.78 (0.71–0.86)	96.3	81.4 (79.8–82.8)	
> 0.991996851	21.37 (14.3–29.9)	99.75 (98.6–100.0)	85.04 (11.65–620.99)	0.79 (0.72–0.87)	96.2	81.2 (79.7–82.6)	
> 0.992193546	15.38 (9.4–23.2)	100.00 (99.1–100.0)	–	0.85 (0.78–0.91)	100.0	80.1 (78.8–81.3)	
> 0.997792258	12.82 (7.4–20.3)	100.00 (99.1–100.0)	–	0.87 (0.81–0.93)	100.0	79.6 (78.4–80.7)	
> 0.997846828	0.00 (0.0–3.1)	100.00 (99.1–100.0)	–	1.00 (1.00–1.00)	–	77.3 (77.3–77.3)	
Version 2A	
≥ 0.06606743	100.00 (96.9–100)	0.00 (0.0–0.9)	1.00 (1.00–1.00)	–	1 (1.0–1.0)	–	
> 0.06606743a	77.78 (69.2–84.9)	95.48 (92.9–97.3)	17.20 (10.84–27.29)	0.23 (0.17–0.33)	17.20 (10.8–27.3)	0.23	
> 0.724269589	57.26 (47.8–66.4)	97.49 (95.4–98.8)	22.79 (12.12–42.87)	0.44 (0.36–0.54)	22.79 (12.1–42.9)	0.44	
> 0.803532537	22.22 (15.1–30.8)	99.75 (98.6–100)	88.44 (12.13–644.87)	0.78 (0.71–0.86)	88.44 (12.1–644.9)	0.78	
> 0.993458263	0 (0.0–3.1)	100.00 (99.1–100)	–	1.00 (1.00–1.00)	–	1	
Version 2B	
≥ 0.064863578	100.00 (96.9–100.0)	0.00 (0.0–0.9)	1.00 (1.00–1.00)	–	22.7 (22.7–22.7)	–	
> 0.064863578a	80.34 (72.0–87.1)	95.23 (92.6–97.1)	16.83 (10.75–26.34)	0.21 (0.14–0.30)	83.2 (76.0–88.6)	94.3 (91.9–96.0)	
> 0.203391964	77.78 (69.2–84.9)	95.48 (92.9–97.3)	17.20 (10.84–27.29)	0.23 (0.17–0.33)	83.5 (76.1–88.9)	93.6 (91.2–95.4)	
> 0.710357369	58.97 (49.5–68.0)	97.49 (95.4–98.8)	23.47 (12.50–44.09)	0.42 (0.34–0.52)	87.3 (78.6–92.8)	89.0 (86.7–91.0)	
> 0.778239804	28.21 (20.3–37.3)	98.99 (97.4–99.7)	28.06 (10.15–77.60)	0.73 (0.65–0.81)	89.2 (74.9–95.8)	82.4 (80.7–84.0)	
> 0.900276331	26.50 (18.8–35.5)	98.99 (97.4–99.7)	26.36 (9.50–73.16)	0.74 (0.67–0.83)	88.6 (73.6–95.6)	82.1 (80.4–83.6)	
> 0.928150249	22.22 (15.1–30.8)	99.75 (98.6–100.0)	88.44 (12.13–644.87)	0.78 (0.71–0.86)	96.3 (78.1–99.5)	81.4 (79.8–82.8)	
> 0.992005419	15.38 (9.4–23.2)	100.00 (99.1–100.0)	–	0.85 (0.78–0.91)	100.0	80.1 (78.8–81.3)	
> 0.997815416	0.00 (0.0–3.1)	100.00 (99.1–100.0)	–	1.00 (1.00–1.00)	–	77.3 (77.3–77.3)	
CI, confidence interval.

Version 1: Pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site; inflammation, induration or swelling at exit site; fever ≥ 38 °C not attributable to other causes; obvious abscess or purulent exudate at the exit site.

Version 2A: Hyperemia or erythema ≥ 2 cm from exit site and obvious abscess or purulent exudate at the exit site.

Version 2B (Final version): Pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site and obvious abscess or purulent exudate at the exit site.

a Associated criterion.

Table 4 Area under the ROC curve and Youden index values of version 2A and 2B of scale for predicting exit site infection of central venous catheters for hemodialysis

EXITA scale	Version 1	Version 2A	Version 2B	
Area under the ROC curve (AUC)	0.885	0.872	0.883	
Standard errora	0.0191	0.0201	0.0193	
95% Confidence intervalb	0.854–0.911	0.840–0.899	0.852–0.910	
95% Bootstrap CIc	0.840–0.916	0.726–0.819	0.821–0.904	
z statistic	20.156	18.511	19.865	
Significance level P (Area = 0.5)	< 0.0001	< 0.0001	< 0.0001	
Youden index J	0.7557	0.7326	0.7557	
Associated criterion	> 0.0650187212	> 0.06606743	> 0.064863578	
Sensitivity	80.34	77.78	80.34	
Specificity	95.23	95.48	95.23	
AUC, area under the ROC curve; ROC, receiver operating characteristic curve.

Version 1: Pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site; inflammation, induration or swelling at exit site; fever ≥ 38 °C not attributable to other causes; obvious abscess or purulent exudate at the exit site.

Version 2A: Hyperemia or erythema ≥ 2 cm from exit site and obvious abscess or purulent exudate at the exit site.

Version 2B (Final version): Pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site and obvious abscess or purulent exudate at the exit site.

a DeLong et al.,26 1988.

b Binomial exact.

c BCa bootstrap confidence interval (1000 iterations; random number seed: 978).

Table 5 shows the weights of each item considered to predict the presence of ESI in a univariate analysis, by calculating the β coefficients. These calculations show that the signs that presented the greatest weight and which were statistically significant were the items “hyperemia or erythema ≥ 2 cm from exit site” (β = 51.68, P < 0.001) and “obvious abscess or purulent exudate at the exit site” (β = 35.366, P < 0.001). The item “pain at exit site during interdialytic period” presented β >1 but lacked statistical significance (β = 3.646, P = 0.138). In the logistic regression analysis, the same 2 signs (“hyperemia or erythema ≥ 2 cm from exit site” and “obvious abscess or purulent exudate at the exit site”), which remained until the last step, were the main predictors of ESI (Table 6). The symptom “pain at exit site during interdialytic period” was eliminated in the penultimate step, with β >1 (β =1.303, P = 0.132).Table 5 Univariate analysis of the β coefficients for each of the items of version 1 of the scale

Item	β	95% CI	P-value	
Lower limit	Upper limit	
Pain at exit site during interdialytic period	3.646	0.661	20.113	0.138	
Hyperemia or erythema ≥2 cm from exit site	51.680	22.557	118.404	0.000	
Inflammation, induration or swelling at exit site	0.818	0.124	5.389	0.835	
Fever ≥ 38 °C not attributable to other causes	0.975	0.085	11.233	0.984	
Obvious abscess or purulent exudate at the exit site	35.366	14.310	87.406	0.000	
(Constant)	0.070			0.000	
CI, confidence interval.

Table 6 Logistic regression with exit site infection as dependent variable and signs and symptoms from version 1 of the scale as predictor variables

Variables in the equation	
Recommendation	β	Standard Error	Wald	df	P	
Step 1	Pain at exit site during interdialytic period	1.294	0.871	2.205	1	0.138	
Hyperemia or erythema ≥2 cm from exit site	3.945	0.423	86.988	1	0.000	
Inflammation, induration, or swelling at exit site	−0.201	0.962	0.044	1	0.835	
Fever ≥ 38 °C not attributable to other causes	−0.025	1.247	0.000	1	0.984	
Obvious abscess or purulent exudate at the exit site	3.566	0.462	59.660	1	0.000	
(Constant)	−2.666	0.201	176.046	1	0.000	
Step 2	Pain at exit site during interdialytic period	1.292	0.869	2.212	1	0.137	
Hyperemia or erythema ≥ 2 cm from exit site	3.945	0.423	86.993	1	0.000	
Inflammation, induration, or swelling at exit site	−0.201	0.962	0.044	1	0.834	
Obvious abscess or purulent exudate at the exit site	3.563	0.440	65.626	1	0.000	
(Constant)	−2.666	0.201	176.060	1	0.000	
Step 3	Pain at exit site during interdialytic period	1.303	0.865	2.270	1	0.132	
Hyperemia or erythema ≥ 2 cm from exit site	3.924	0.409	92.124	1	0.000	
Obvious abscess or purulent exudate at the exit site	3.566	0.440	65.778	1	0.000	
(Constant)	−2.668	0.201	176.973	1	0.000	
Step 4	Hyperemia or erythema ≥ 2 cm from exit site	4.057	0.402	101.615	1	0.000	
Obvious abscess or purulent exudate at the exit site	3.614	0.436	68.762	1	0.000	
(Constant)	−2.649	0.199	176.926	1	0.000	
df, degree of freedom.

Supplementary Figure S3 shows the ROC curve and Table 4, the AUC characteristics for version 2A, with the sum of “hyperemia or erythema ≥ 2 cm from exit site” and “obvious abscess or purulent exudate at the exit site.” These 2 signs jointly predict 87.2% of ESI, with a sensitivity of 77.78% and a specificity of 95.48%, given the presence of at least 1 of these signs (Youden index 0.7326 ≈1, P < 0.001).

Figure 3 shows the ROC curve and Table 4 shows the AUC characteristics for version 2B of the scale, with the sum of “pain at exit site during interdialytic period”, “hyperemia or erythema ≥ 2 cm from exit site” and “obvious abscess or purulent exudate at the exit site.” The combination of these 3 signs and symptoms predict 88.3% of ESI, with a sensitivity of 80.34% and a specificity of 95.23%, based on the presence of 1 of these signs (Youden index 0.7557 ≈ 1, P < 0.001). In view of the properties of the combination of these 3 signs and symptoms, this was taken as the optimum version of the scale (final version, version 2B).Figure 3 ROC curve and AUC characteristics of the scale, for final version (2B version) (pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site and obvious abscess or purulent exudate at the exit site). AUC, area under the ROC curve; ROC, receiver operating characteristic curve.

According to the corresponding formula, the values of final version (version 2B) of the scale were calculated as:

Logit ESI= −2.668 + Pain at exit site during interdialytic period × 1.303 + Hyperemia or Erythema ≥ 2 cm from exit site × 3.924 + Obvious abscess or purulent exudate at the exit site × 3.566

By applying this formula in the presence of signs and symptoms, we obtain a numerical value that can be transformed into probability using the equation:

elogit IOS/1 + elogit IOS

This procedure obtains the probability, expressed as a percentage, of a patient with CVC-HD presenting ESI, according to the presence or absence of these 3 signs and symptoms.

EXITA scale calculator is available in this link: https://goo.su/HU64JM, which can be used to calculate the probability of a patient presenting ESI according to the presence or absence of each of the items in final version, together with the best treatment approach to be taken, as established in phase 2 of the study.17 This probability ranges from 6.49% (when all the items are absent, and corresponds to the mere existence of a CVC-HD) to 99.78% (when all the items are present).

Figure 4 shows final version of the scale (version 2B), together with the treatment approach determined in phase 2 of the study,17 upon the appearance of each sign and symptom.Figure 4 Final version of the EXITA scale, with the recommended treatment approach. EXITA, EXIT site Assessment.

Discussion

To our knowledge, the multicenter cohort study we describe is the first to develop a predictive model of ESI by CVC-HD, validating the EXITA scale. With 3 very simple items to assess, it also guides the treatment approach to be adopted, according to the presence or absence of each item. It is the first instrument of its kind, it has been robustly validated, and predicts almost 90% of ESI, with high sensitivity and specificity.

In the present study, version 1 of the instrument (with 5 items) was subjected to clinical validation in a predictive capacity study. The very specific prioritization of these 5 signs and symptoms by the expert panel was endorsed during the clinical validation phase, which highlighted the excellent predictive properties of this first version of the scale. With the presence of just 1 of the items, the scale had a sensitivity of 85.47% and a specificity of 93.97%.

The calculation of the β coefficients in the univariate analysis showed that the items with the greatest predictive capacity were “hyperemia or erythema ≥ 2 cm from exit site” (β = 51.68, P < 0.001) and “obvious abscess or “purulent exudate at the exit site.” This conclusion was confirmed by logistic regression. For our research team, however, the presence of pain at the exit site during the interdialytic period is sufficiently significant to merit attention. In this respect, Harwood et al.27 observed an association between the presence of pain and ESI. For this reason, pain was included in the final version of the scale. This improved the predictive characteristics of the instrument, raising the AUC from 87.2% to 88.3%, and the sensitivity from 77.8% to 80.34%, while maintaining good specificity (which fell from 95.5% to 95.23%). In all cases, the Youden index was close to 1, and so the mere presence of 1 of the items allows the early identification of ESI. The precision obtained with these items reflects the good work done by the panel of experts and their expertise in the phenomenon under study. These values are similar to those presented by the initial 5 items, improving efficiency in the application of the scale. The fact that the final version of the scale has only 3 items, all of which can be straightforwardly determined in each hemodialysis session, facilitates its immediate transfer to clinical practice. Moreover, the development of the ESI risk calculator will facilitate its implementation at the point of care, enabling its integration into clinical information systems, as part of the daily assessment recommended by the corresponding clinical practice guidelines.13,23,28

To our knowledge, the only scale of this type currently available is the REDS scale (published in 2021, after the start of the present research). However, it has not been subjected to validation or a study of its predictive capacity.29 The REDS scale evaluates the presence of 4 items: redness, edema, and exudate at the exit site; and systemic symptoms of infection that might indicate catheter-related bacteremia.

In another study, Porazko et al.29 reported that with the implementation of the REDS scale, the 2-year cumulative incidence of ESI was significantly reduced (log rank P < 0.001) from 0.89 episodes/1000 catheter days (53.5%, 95% CI: 35.9%–66.2%) in the period before the use of the REDS scale, to 0.26 episodes/1000 catheter days (18.6%, 95% CI: 6.1%–29.4%) with its application. These authors also observed a significant decrease in complicated ESI with catheter-related bacteremia that required removal of the tunneled CVC-HD (0.6 episodes/1000 catheter days; 18.6%, 95% CI: 6.1%–29.4%; vs. 0.3 episodes/1000 catheter days; 4.7%, 95% CI: 0.0%–10.7%; log-rank P = 0.04, in the periods before and after implementation of the REDS scale, respectively.29 In our opinion, the use of such a scale could reduce the rates of local and systemic infection, by enabling the early detection of signs and symptoms indicative of ESI, and the prompt application of measures to inhibit its progression and prevent the intraluminal contamination of the CVC-HD, with the risk of bacteremia. Nevertheless, we do not believe this improvement would be of the magnitude indicated by Porazko et al.29 Accordingly, an implementation study of the EXITA scale should be conducted to determine the real impact (both clinical and economic) of its use in clinical practice.

The limitations and strengths of our study should be considered in order to properly interpret the results obtained. For example, a significant consideration is that the ethnicity of the patients was not considered in our analysis, although most patients were of Caucasian origin. However, Rigo et al.30 observed differences in the results obtained when the Twardowski scale was used to assess the exit site of peritoneal dialysis catheters, according to the race of the patient, in the sense that ESI was underdiagnosed in the case of African Americans. Accordingly, we suggest the scale we propose should be validated for use with African-American and sub-Saharan populations in order to verify its behavior in this respect. Nevertheless, in general terms we believe the presentation of the phenomenon under study is universal and that our scale is generally applicable in any health care context. Positive exit site cultures in the absence of exudate may represent contamination rather than ESI.

The implementation of the EXITA scale in the clinical environment may reduce early ESI and catheter-related bacteremia and therefore reduce morbidity and mortality among patients with chronic kidney disease with CVC-HD, because colonization by microorganisms of the catheter surface and of the exit orifice is the main source of contamination of this type of vascular device.13

In conclusion, we have successfully developed the first clinically-validated ESI prediction scale for CVC-HD. This instrument has good predictive properties, and only requires the straightforward evaluation of 3 items before each hemodialysis session. According to the predictive model developed, the presence of just 1 of these items has a high predictive power for ESI. The use of this scale facilitates the early detection and treatment of ESI, before its microbiological confirmation. Furthermore, its use would improve observational objectivity by ensuring uniformity of classification, thus facilitating the comparison of results, the continuity of care, cost savings, patient education, and the comparison of research studies on measures for prevention and/or treatment.

Disclosure

All the authors declared no competing interests.

Supplementary Material

Supplementary File (PDF)

Figure S1. A millimeter ruler with magnifying glass showing the signs and symptoms to be evaluated together with the QR code to access the data collection notebook used by nurses to assess the exit site(s) in each hemodialysis session. Figure S2. ROC curve and AUC characteristics of version 1 of the scale (pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site; inflammation. induration or swelling at exit site; fever ≥ 38 °C not attributable to other causes and obvious abscess or purulent exudate at the exit site). Figure S3. ROC curve and AUC characteristics of version 2A of the scale (hyperemia or erythema ≥2 cm from exit site and obvious abscess or purulent exudate at the exit site). Table S1. Variables included in the study of predictive capacity. Table S2. Variables related to exit site local policy for CVC-HD maintenance (n = 335). Table S3. Frequency of each sign and symptom in the sessions in which cultures were collected (n = 515). Table S4. Frequency of combinations of signs and symptoms in the sessions in which cultures were collected (n = 515). Table S5. Frequency of occurrence of microorganisms responsible for culture-positive exit site infections. Table S6. Validity properties of each item of the scale. STROBE Statement.

Acknowledgments

This work is supported by 10.13039/501100018812 IDIVAL - Valdecilla Biomedical Research Institute (grant number NVAL18/05 ) and General Council of Official Nursing Colleges of Spain in the 2022 Doctoral Research Support Programme. We thank the Íñigo Álvarez de Toledo Renal Foundation (FRIAT) for granting this study the 22nd Research Award in Nephrological Nursing and Humanisation in the Treatment of Patients, in 2022. We also thank Pedro Muñoz Cacho for his statistical advice in phase 2 of the study. The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Supplementary File (PDF)

Figure S1. A millimeter ruler with magnifying glass showing the signs and symptoms to be evaluated together with the QR code to access the data collection notebook used by nurses to assess the exit site(s) in each hemodialysis session.

Figure S2. ROC curve and AUC characteristics of version 1 of the scale (pain at exit site during interdialytic period; hyperemia or erythema ≥ 2 cm from exit site; inflammation. induration or swelling at exit site; fever ≥ 38 °C not attributable to other causes and obvious abscess or purulent exudate at the exit site).

Figure S3. ROC curve and AUC characteristics of version 2A of the scale (hyperemia or erythema ≥2 cm from exit site and obvious abscess or purulent exudate at the exit site).

Table S1. Variables included in the study of predictive capacity.

Table S2. Variables related to exit site local policy for CVC-HD maintenance (n = 335).

Table S3. Frequency of each sign and symptom in the sessions in which cultures were collected (n = 515).

Table S4. Frequency of combinations of signs and symptoms in the sessions in which cultures were collected (n = 515).

Table S5. Frequency of occurrence of microorganisms responsible for culture-positive exit site infections.

Table S6. Validity properties of each item of the scale.

STROBE Statement.
==== Refs
References

1 Romagnani P. Remuzzi G. Glassock R. Chronic kidney disease Nat Rev Dis Primers 3 2017 17088 10.1038/nrdp.2017.88
2 Bikbov B. Purcell C.A. Levey A.S. Global, regional, and national burden of chronic kidney disease, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017 Lancet 395 2020 709 733 10.1016/S0140-6736(20)30045-3 32061315
3 Liyanage T. Ninomiya T. Jha V. Worldwide access to treatment for end-stage kidney disease: a systematic review Lancet (London, England) 385 2015 1975 1982 10.1016/S0140-6736(14)61601-9 25777665
4 Himmelfarb J. Vanholder R. Mehrotra R. Tonelli M. The current and future landscape of dialysis Nat Rev Nephrol 16 2020 573 585 10.1038/s41581-020-0315-4 32733095
5 Taylor G. Gravel D. Johnston L. Incidence of bloodstream infection in multicenter inception cohorts of hemodialysis patients Am J Infect Control 32 2004 155 160 10.1016/j.ajic.2003.05.007 15153927
6 Wang K. Wang P. Liang X. Lu X. Liu Z. Epidemiology of haemodialysis catheter complications: a survey of 865 dialysis patients from 14 haemodialysis centres in Henan Province in China BMJ Open 5 2015 e007136 10.1136/bmjopen-2014-007136
7 Klevens R.M. Edwards J.R. Andrus M.L. Special report: dialysis surveillance report: National Healthcare Safety Network (NHSN)-data summary for 2006 Semin Dial 21 2007 24 28 10.1111/j.1525-139X.2007.00379.x
8 Patel P. Kallen A. Arduino M. Epidemiology, surveillance, and prevention of bloodstream infections in hemodialysis patients Am J Kidney Dis 56 2010 566 577 10.1053/j.ajkd.2010.02.352 20554361
9 Weijmer M.C. Vervloet M.G. ter Wee P.M. Compared to tunnelled cuffed haemodialysis catheters, temporary untunnelled catheters are associated with more complications already within 2 weeks of use Nephrol Dial Transplant 19 2004 670 677 10.1093/ndt/gfg581 14767025
10 Develter W. De Cubber A. Van Biesen W. Vanholder R. Lameire N. Survival and complications of indwelling venous catheters for permanent use in hemodialysis patients Artif Organs 29 2005 399 405 10.1111/j.1525-1594.2005.29067.x 15854216
11 Lazarus B. Bongetti E. Ling J. Gallagher M. Kotwal S. Polkinghorne K.R. Multifaceted quality improvement interventions to prevent hemodialysis catheter-related bloodstream infections: a systematic review Am J Kidney Dis 82 2023 429 442.e1 10.1053/J.AJKD.2023.02.006 37178814
12 Kotwal S. Cass A. Coggan S. Multifaceted intervention to reduce haemodialysis catheter related bloodstream infections: REDUCCTION stepped wedge, cluster randomised trial BMJ 377 2022 e069634 10.1136/BMJ-2021-069634
13 Lok C.E. Huber T.S. Lee T. KDOQI clinical practice guideline for vascular access: 2019 update Am J Kidney Dis 75 2020 S1 S164 10.1053/j.ajkd.2019.12.001 32778223
14 Cobo-Sánchez J.L. Blanco-Mavillard I. Pelayo-Alonso R. Mancebo-Salas N. Fernandez-Fernandez I. De Pedro-Gomez J.E. Validity of a catheter exit site clinical assessment scale for the early detection of exit site infections in patients on haemodialysis with a central venous catheter: protocol for a multicentre validation study in Spain (EXITA Study) BMJ Open 12 2022 e065724 10.1136/BMJOPEN-2022-065724
15 DeVellis R.F. Scale development: theory and applications 4th Edition 2017 SAGE Publications
16 Cobo-Sánchez J.L. de Pedro-Gómez J.E. Mancebo-Salas N. Pelayo-Alonso R. Blanco-Mavillard I. Defining the relevance of signs and symptoms to identify exit site infection of a hemodialysis central venous catheter: a Delphi study protocol Int J Qual Methods 20 2021 1 8 10.1177/16094069211002749
17 Cobo-Sánchez J.L. Blanco-Mavillard I. Mancebo-Salas N. Pelayo Alonso R. Gancedo-González Z. De Pedro-Gómez J.E. Definition and clinical management of hemodialysis central venous catheter local infections (exit site and tunnel infection): an international consensus assessment J Clin Nurs 33 2023 559 571 10.1111/jocn.16941 38093579
18 Cobo Sánchez J.L. Pelayo Alonso R. Alonso Nates R. Hemodialysis catheter exit site healing with chlorhexidine gluconate discs: a case study. Libro de comunicaciones XXXV congreso de la sociedad española de enfermería Nefrológica SEDEN 2011 SEDEN Madrid 110 115
19 Martín-Peña A. Márquez R.L. Marco Guerrero M.J. Tunneled hemodialysis catheter-related bloodstream infections: a prospective multicenter cohort study from Spain J Vasc Access 13 2012 239 245 10.5301/JVA.5000034 22266591
20 Bouza E. Rojas L. Guembe M. Predictive value of superficial cultures to anticipate tunneled hemodialysis catheter-related bloodstream infection Diagn Microbiol Infect Dis 78 2014 316 319 10.1016/j.diagmicrobio.2013.12.008 24428979
21 Eriguchi M. Tsuruya K. Yoshida H. Validation of the exit-site scoring system recommended by the 2005 guidelines of the International Society for peritoneal dialysis Perit Dial Int 31 2011 698 700 10.3747/PDI.2010.00287 22123857
22 Mermel L.A. Allon M. Bouza E. Clinical practice guidelines for the diagnosis and management of intravascular catheter-related infection: 2009 update by the Infectious Diseases Society of America Clin Infect Dis 49 2009 1 45 10.1086/599376 19489710
23 Ibeas J. Roca-Tey R. Vallespín J. Spanish clinical guidelines on vascular access for haemodialysis Enferm Nefrol 21 2018 S1 S256 10.4321/S2254-28842018000500001
24 Hosmer D.W. Lemeshow S. Sturdivant R.X. Applied logistic regression 3rd Edition 2013 John Wiley & Sons, Inc 10.1002/9781118548387
25 Hilbe J.M. Logistic regression models 1st Edition 2009 CRC Press 10.1201/9781420075779
26 DeLong E.R. DeLong D.M. Clarke-Pearson D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach Biometrics 44 1988 837 845 3203132
27 Harwood L. Wilson B. Thompson B. Brown E. Young D. Predictors of hemodialysis central venous catheter exit-site infections CANNT J 18 2008 26 35
28 Vascular Access 2006 Work Group Clinical practice guidelines for vascular access Am J Kidney Dis 48 2006 S176 S247 10.1053/j.ajkd.2006.04.029 16813989
29 Porazko T. Stasiak E. Klinger M. The impact of the introduction of innovative REDS scale for the evaluation of central tunnelled catheter (CTC) exit site on infection prevention in long-term haemodialyzed patients Front Surg 8 2021 629367 10.3389/FSURG.2021.629367
30 Rigo M. Pecoits-Filho R. Lambie M. Tuon F.F. Barretti P. de Moraes T.P. Clinical utility of a traditional score system for the evaluation of the peritoneal dialysis exit-site infection in a national multicentric cohort study Perit Dial Int 41 2021 292 297 10.1177/0896860820949032 32856533
