
==== Front
Medicine (Baltimore)
Medicine (Baltimore)
MD
Medicine
0025-7974
1536-5964
Lippincott Williams & Wilkins Hagerstown, MD

MD-D-23-11893
00074
10.1097/MD.0000000000039516
3
3400
Research Article
Observational Study
The relationship between the technical performance score (TPS) and outcomes and its discriminative ability in congenital heart surgery
https://orcid.org/0000-0002-9833-1363
TAN-RECEP Berra Zümrüt MD a*
Ozturk Erkut PhD erkut_ozturk@yahoo.com
b
a Department of Pediatric Cardiovascular Surgery, Istanbul Başakşehir Çam and Sakura City Hospital, University of Science Turkey, Turkey
b Department of Pediatric Cardiology, Istanbul Başakşehir Çam and Sakura City Hospital, University of Science Turkey, Turkey.
* Correspondence: Berra Zümrüt TAN-RECEP, Department of Pediatric Cardiovascular Surgery, Istanbul Başakşehir Çam and Sakura City Hospital, University of Science Turkey, Başakşehir, Istanbul, Turkey (e-mail: bzumrut.tan@gmail.com).
06 9 2024
06 9 2024
103 36 e3951603 1 2024
08 8 2024
09 8 2024
Copyright © 2024 the Author(s). Published by Wolters Kluwer Health, Inc.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial License 4.0 (CCBY-NC), where it is permissible to download, share, remix, transform, and buildup the work provided it is properly cited. The work cannot be used commercially without permission from the journal.

Various scoring systems have been used to predict mortality and morbidity after congenital heart surgery. While the ideal system is still controversial, the technical performance score (TPS) has recently gained popularity. In this study, was investigated the effect of the TPS in predicting mortality and morbidity in pediatric patients who underwent congenital heart surgery in our clinic. Patients aged < 18 years who underwent congenital heart surgery between 2020 and 2023, were retrospectively analyzed. The patients’ TPS categorizations were assigned according to their echocardiographic results at discharge and whether they required reintervention. The primary endpoints of the study were mortality (death within 30 days postoperatively) and morbidity. The secondary endpoint was a comparison of the effectiveness of TPS with that of the widely used Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery score. Included in this study were 1075 patients. The median patient age was 3 months (interquartile range, 1–5 months). The mortality and morbidity rates were 11% and 24%, respectively. Of the patients, 60% were categorized as TPS I (optimal), 25% as TPS 2 (adequate, minimal residual defect), and 15% as TPS 3 (inadequate, hemodynamically significant residual defect). Being categorized as TPS 3 was associated with mortality, prolonged ICU stay, and major adverse events. The predictive power of TPS for mortality and morbidity was an area under the receiver operating characteristic curve (AUC) of 0.810 (95% CI: 0.79–0.839, P < .001) and 0.78 (95% CI: 0.76–0.80, P < .001), respectively. These values were similar to those of the Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery score (0.81 vs 0.83 and 0.78 vs 0.80 for mortality and morbidity, respectively). In patients with highly heterogeneous congenital heart disease, the use of intraoperative TPS may be helpful in predicting mortality and morbidity.

congenital heart disease
mortality
STAT score
technical performance score
OPEN-ACCESSTRUE
==== Body
pmc1. Introduction

Congenital heart disease (CHD) occurs in approximately 1% of newborns. Of these cases, 1 in 4 are severe defects and require surgery or other procedures in the first year of life.[1]

Early outcomes of congenital heart surgery depend on several factors. These include the complexity of the defect, preoperative status of the patient, duration of cardiopulmonary bypass (CPB), whether the surgery is palliative or curative, intensive care stay, and duration of hospitalization.[2,3] Various scoring systems are used to objectively evaluate the risk of morbidity and mortality after congenital heart surgery, which aim to create a common and, reliable database, reduce costs in intensive care units (ICUs), lead to a more effective use of resources, guide clinical decisions and practices, and allow for an objective evaluation. However, the ideal scoring system is still controversial.[4]

The technical performance score (TPS) is new scoring system developed to measure operative adequacy.[3] Although it was first introduced for selected surgeries such as tetralogy of Fallot and the Norwood procedure, it has become available in most surgeries.[5] Studies have shown that TPS is directly related to surgical outcomes, and outcomes, such as mortality, hospital stay, and major adverse events, can be improved with optimal TPS.[6–8] Although there are a few studies, their success in showing late mortality and reoperation is limited.[9] TPS is based on echocardiographic and clinical measurements. This makes the method simple and easy to apply. However, the possible subjectivity of echocardiographic measurements, the fact that they do not involve information about the patient’s preoperative clinical status, and the complexity of the procedures are disadvantages.

In this study, we investigated the effect of TPS in predicting mortality and morbidity in pediatric patients who underwent congenital heart surgery in our clinic.

2. Materials and methods

Patients aged < 18 years who underwent surgery for congenital heart defects between August 1, 2020 and January 1, 2023, were retrospectively analyzed. The TPSs of all patients were measured under operating room conditions, through preoperative and postoperative echocardiography, by the same pediatric cardiology team with >10 years of echocardiography experience.

Premature patients, those with permanent pacemakers, and those transferred to other units were excluded from the study. This retrospective study was approved by our institutional ethics committee and conducted in accordance with the principles of the Declaration of Helsinki.

The preoperative demographic data (sex and weight), anatomical diagnoses, surgical data (CPB time, cross-clamp time), clinical follow-up results (extracorporeal membrane oxygenation [ECMO]), major adverse events, and postoperative ICU reports (ventilation time/hour, vasoactive inotrope score, re-intubation, delayed closure, chylothorax, infection, and acute kidney injury) of the study group were evaluated.

Major adverse events were defined as the presence of at least one of the following: cardiac arrest, unplanned reoperation (as a result of an unexpected surgical procedure or significant postoperative residual lesion), emergency thoracotomy (for sternal bleeding control or mediastinal pressure relief), or low cardiac output syndrome (LCOS). LCOS was defined by clinical changes, such as altered consciousness, altered skin appearance, cold extremities, weak pulse, and a capillary circulation time of >2 seconds.[10]

Kidney disease improving global outcomes staging was used to standardize acute kidney injury (AKI) diagnoses. AKI was defined as kidney disease improving global outcomes stage ≥ 2 or requiring renal replacement therapy due to progressive elevation of serum lactate, urea, and creatinine levels despite medical treatment.

Infection was defined as the presence of microorganisms in the cultures of samples taken from patients with clinical symptoms such as fever or pneumonia, or wound site septic discharge evident by Gram staining, and the presence of sepsis findings in the laboratory examinations.

In this study, the 2 primary outcomes were operative mortality and pediatric cardiac ICU length of stay (PCILOS). Operative mortality included all deaths that occurred during the hospital stay immediately after the operation and deaths that occurred within 30 days postoperatively. For further analysis of PCILOS, data were dichotomized as the 25th percentile (best) versus the 75th percentile (worst). The PCILOS of the 75th percentile was considered to be prolonged. In addition, the effectiveness of TPS was compared with that of the Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery (STAT) score.[10]

The TPS categorizations were assigned according to echocardiographic results and need for reintervention.

2.1. Technical performance score

All postoperative echocardiograms were performed epicardially by the same pediatric cardiology team. In the case of a TPS of 2 and above, the opinions of other professionals were sought to ensure that the decision was objective.

Patients were classified according to their last echocardiographic results before ICU discharge or total discharge. Each component of the surgical procedure was evaluated separately. Patients were divided 2 groups according to echocardiographic findings: TPS 1 (optimal, no residual defect), TPS 2 (adequate, minimal residual defect), and TPS 3 (inadequate, hemodynamically significant residual defect). Patients who required any catheters or surgical intervention due to defects repaired during the initial surgery and who required an indwelling catheter were placed in the TPS 3 group.[11]

Patients with mortality and those who had no previous indication for intervention but underwent reoperation for reasons such as ventricular septal defect opening or valve failure requiring intervention were considered as TPS 3.

2.2. Statistical analysis

Data were analyzed using SPSS Statistics for Windows (version 15.0: SPSS Inc., Chicago). The median with the interquartile range (IQR) was used to describe continuous data, whereas the absolute count with percentages was used for categorical data. Pearson chi-squared test and one-way analysis of variance (ANOVA) were used to compare variables between the groups. In logistic regression models, discrimination of factors predicting mortality and morbidity was assessed using the area under the receiver operating characteristic (ROC) curve (AUC).[8] The results showed an estimated AUC and a 95% confidence interval. Statistical significance was set at P < .05.

3. Results

A total of 1075 patients were included in the study, of whom 50% were male, and the median age was 2 months (IQR: 1–3 months). The demographic characteristics of the patients are presented in Table 1.

Table 1 Demographics and patient characteristics.

Variables	Median (IQR) or n, %	
n	1075	
Age (mo)	2(1–3)	
Weight (kg)	4(3.2–5)	
Body surface area (m2)	0.24(0.20–0.28)	
Male	537(50)	
STAT category
 1
 2
 3
 4
 5	102(10)
125(12)
95(10)
495(46)
258(24)	
Cardiopulmonary bypass	967(90)	
Emergent procedure	172(16)	
Physiology
 Single ventricle
 Biventricular	301(28)
774(70)	
Main procedure
 ASD closure
 Arch reconstruction
 Arterial switch operation
 AVSD repair
 Bidirectional cava pulmonary anastomosis
 DORV repair
 Fontan operation
 Norwood
 Pulmonary artery banding
 Shunt
 TAPVC repair
 Tetralogy of Fallot total repair
 Truncus repair
 Valve repair
 VSD closure
 Other	65(6%)
108(10%)
87(8%)
58(5.3%)
34(3.1%)
10(0.9%)
3(0.2%)
25(2.3%)
62(5.7%)
26(2.4%)
42(3.9%)
84(7.8%)
8(0.7%)
34(3.1%)
212(19.7%)
217(20.1%)	
ASD = atrial septal defect, AVSD = atrioventricular septal defect, DORV = double outlet right ventricle, STAT = Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery, TAPVC = total anomalous pulmonary venous connection, VSD =  = ventricular septal defect.

Regarding to the TPS categorization, 60% of the patients had TPS I, 25% had TPS 2, and 15% had TPS 3. The mortality rate was 11%. Most deaths occurred after prolonged intensive care in patients who underwent palliative surgery for a single ventricle. The death was due to multi-organ failure and sepsis. The morbidity rate was 24%. An ICU stay of 13 days was considered prolonged.

The predictive power of TPS for mortality and morbidity was an AUC of 0.810 (95% CI: 0.79–0.839, P < .001) and 0.78 (95% CI: 0.76–0.80, P < .001), respectively. The ROC curve graphs related to the predictive power of TPS for mortality and morbidity are shown in Figures 1 and 2. Accordingly, the TPS 3 categorization significantly predicted mortality and morbidity.

Figure 1. Prediction of mortality according to the Technical Performance Score of the cases. ROC = receiver operating characteristic.

Figure 2. Prediction of morbidity according to the Technical Performance Score of the cases. ROC = receiver operating characteristic.

The predictive power of the STAT score for mortality and morbidity was an AUC of 0.810 and 0.800, respectively. This was not significantly different than the predictive power of the TPS (P > .05).

4. Discussion

In this study, were investigated the effect of TPS in predicting mortality and morbidity in pediatric patients who underwent congenital heart surgery in our clinic. Significant residual shunting detected by intraoperative post-CPB echocardiography was associated with poor early outcomes. This was true even for patients with different demographics and preoperative clinical statuses undergoing surgeries of varying complexity (Table 2). While TPS 1 patients had the lowest risk, TPS 3 patients had the highest. In accordance with the associated high risk, categorization of TPS 3 was able to predict morbidity and mortality.

Table 2 Technical performance score, surgical outcomes and perioperative data.

Outcome	TPS-1 (n = 645)	TPS-2 (n = 269)	TPS-3 (n = 161)	P	
CPB time (min)	90(70–110)	120(100–145)	160(115–180)	<.001	
Cross clamp time (min)	50(30–70)	80(60–100)	110(70–145)	<.001	
Ventilation time/hour	12(8–16)	20(15–25)	32(26–40)	.002	
ECMO	14(2.1%)	9(3.3)	15(9.3)	.120	
Major adverse event	58(9.1%)	54(20.1%)	51(31.6%)	<.001	
Vasoactive Inotrope score <10	516(80)	188(70)	96(60)	.080	
Re-intubation	13(2)	8(2.9)	8(4.9)	.460	
Delayed closure	45(7%)	48(17.8%)	47(29.2%)	<.001	
Arrhythmias	38(6)	19(7)	14(8.5)	.280	
Chylothorax	19(3%)	6(2.2%)	4(2.4%)	.780	
Infection	22(3.4%)	11(4%)	10(6.2%)	.180	
Acute kidney injury	30(4.6%)	35(13%)	34(21.1%)	<.001	
ICU stay	3(2–4)	4(3–5)	7(6–8)	.02	
Post-operative hospital stay (d)	7(5–10)	9(7–13)	14(11–16)	.03	
Mortality	23(3.5%)	17(6.1%)	78(48.4%)	<.001	
Morbidity	68(10.5%)	70(26%)	120(74%)	<.001	
CPB = cardiopulmonary bypass, ECMO = extracorporeal membrane oxygenation, ICU = intensive care unit.

As reported by Pasquali et al, TPS 3 is also associated with high complication rates and a prolonged ICU stay.[12] In another series of 1926 patients published in 2014, TPS was found to be effective in determining early mortality and morbidity. In the aforementioned series, the overall early mortality rate was 2.6%, whereas it was 16.9% in patients with TPS 3.[13] In the present study, the mortality rate was 11%. Similarly, the mortality rate of patients with a TPS 3 of was higher. This higher mortality rate than reported in other studies was attributed to the high number of neonates and patients with STAT categories of 4 and above who underwent complex surgery. Most mortalities occurred after a prolonged ICU period in patients who underwent single-ventricle palliation in the neonatal period. Currently, the Norwood procedure, one of the most complex neonatal surgical procedures, still has a high mortality rate, and TPS has been shown to be a predictor of both early and late outcomes.[14] TPS has also been reported to be effective in predicting reintervention and the use of hospital resources after arterial switch surgery, which is another complex neonatal surgical procedure. The ability of the previously unstandardized TPS to determine the severity of residual defects has been shown to be remarkable.[15]

In their study, Shuhaiber et al showed that neonatal patients with mortality had no significant differences regarding their Aristotle scores, risk adjusted classification for congenital heart surgery (RACHS)-1 categories, and single or biventricular operation. Their multivariable analysis results suggested that inadequate TPS, surgical palliation, and the need for postoperative ECMO were associated with in-hospital mortality.[16] Other studies on neonatal and infant patients have also reported that a categorization of TPS 3 effectively indicates high mortality and morbidity, especially in high-risk patients. The current study also emphasized that it is important for patients to leave the operating room with optimal outcomes.[17] An inadequate TPS was found to be directly associated with poor survival, even in complicated neonatal surgeries such as the Norwood 1 procedure, regardless of the patient’s anatomical and preoperative physiological status.[18] In our clinical practice, perioperative epicardial echocardiography is performed after CPB to evaluate whether the patients have hemodynamically significant residual defects. If necessary, surgical revision is performed, and the operation is terminated under optimal conditions. Our clinical opinion is that immediate intervention in patients with a TPS 3 would significantly reduce mortality and morbidity. In the current study, patients who died immediately were admitted with cardiogenic shock, while other deaths usually occurred because of multi-organ failure and sepsis within 30 days after palliative surgery for a single ventricle. Some studies have shown that the outcomes of patients requiring intraoperative revision are not significantly worse than those of patients who do not undergo postoperative reintervention. On the other hand, terminating surgery in patients with a TPS 3 was associated with both early mortality, prolonged ICU stay, and early reintervention, as well as mid- and late-term outcomes. Recent studies have shown that early intervention for significant residual shunting, especially in single-ventricle patients, may be effective in reducing postoperative complications and length of hospitalization.[19]

The heterogeneous nature of congenital heart surgery hinders the development of a standardized system for assessing TPSs. The criteria used for each technical category are different and may be partly subjective because they are based on echocardiographic findings. The main purpose of a scoring system is to evaluate the technical performance and benefits of surgery in terms of survival.[17] The TPS was based solely on the echocardiographic results of the patient at discharge, and all results were scored equally. Evaluating not only postoperative but also intraoperative TPS and dividing surgical procedures into subgroups may be beneficial.[20]

In this study, TPS was used together with scoring systems that consider case complexity, such as the STAT, to prevent subjective results. No significant differences were observed between these systems in terms of mortality or morbidity. Studies comparing TPS with RACHS have reached similar results. TPS was shown to be prospectively effective in children younger than 6 months with a RACHS-1 categorization of 2 to 6.[21] While a TPS 3 was strongly associated with higher STAT categorization and RACHS-1 scores, it was reported to be more effective in predicting late mortality.[19]

The main limitation of this study is that it was performed retrospectively on a limited number of patients in a single center. Another limitation is the heterogeneous nature of these cases. Additionally, the TPS relies on postoperative results, limiting its ability to predict preoperative mortality. In our clinic, early postoperative echocardiography is performed epicardially after completion the surgical procedure. However, assessment of TPS may be difficult and misleading in clinics with different strategies if echocardiography cannot be performed before death. This is one of the disadvantages of the TPS.

In conclusion, the use of intraoperative TPSs may be helpful in predicting mortality and morbidity in patients with highly heterogeneous heart patients. Especially after the surgical procedure, it may be useful to determine TPS by transepicardial echocardiography. In the presence of a correctable surgical defect, early reoperation in patients with TPS 2 and above improves surgical outcomes. Further multi-center prospective studies are required.

Author contributions

Conceptualization: Berra Zümrüt TAN-RECEP, Erkut Ozturk.

Data curation: Berra Zümrüt TAN-RECEP.

Formal analysis: Erkut Ozturk.

Funding acquisition: Berra Zümrüt TAN-RECEP.

Investigation: Berra Zümrüt TAN-RECEP.

Methodology: Berra Zümrüt TAN-RECEP.

Project administration: Berra Zümrüt TAN-RECEP.

Resources: Berra Zümrüt TAN-RECEP.

Software: Berra Zümrüt TAN-RECEP.

Supervision: Erkut Ozturk.

Validation: Berra Zümrüt TAN-RECEP

Visualization: Berra Zümrüt TAN-RECEP.

Writing – original draft: Berra Zümrüt TAN-RECEP.

Writing – review & editing: Berra Zümrüt TAN-RECEP.

Abbreviations:

AKI acute kidney injury

AUC area under the receiver operating characteristic curve

CPB cardiopulmonary bypass

ECMO extracorporeal membrane oxygenation

ICU intensive care unit

IQR interquartile range

LCOS low cardiac output syndrome

PCILOS pediatric cardiac ICU length of stay

RACHS risk adjusted classification for congenital heart surgery

ROC receiver operating characteristic

STAT Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery

TPS technical performance score.

The authors have no funding and conflicts of interest to disclose.

The datasets generated during and/or analyzed during the current study are publicly available.

How to cite this article: TAN-RECEP BZ, Ozturk E. The relationship between the technical performance score (TPS) and outcomes and its discriminative ability in congenital heart surgery. Medicine 2024;103:36(e39516).
==== Refs
References

[1] Oster ME Lee KA Honein MA Riehle-Colarusso T Shin M Correa A . Temporal trends in survival among infants with critical congenital heart defects. Pediatrics. 2013;131 :e1502-8.23610203
[2] Barach P Johnson JK Ahmad A . A prospective observational study of human factors, adverse events and patient outcomes in surgery for pediatric cardiac disease. J Thorac Cardiovasc Surg. 2008;136 :1422–8.19114184
[3] Benavidez OJ Gauvreau K del Nido P Bacha E Jenkins KJ . Complications and risk factors for mortality during congenital heart surgery admissions. Ann Thorac Surg. 2007;84 :147–55.17588402
[4] Yildiz O Kasar T Ozturk E . Analysis of congenital heart surgery results: A comparison of four risk scoring systems. Turk gogus kalp damar cerrahisi dergisi. 2018;26 :200–6.32082735
[5] STS.org. STS. Congenital Heart Surgery Data Summary. 2016
[6] Jenkins KJ Gauvreau K Newburger JW Spray TL Moller JH Iezzoni LI . Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg. 2002;123 :110–8.11782764
[7] Welke KF Jacobs JP Jenkins KJ . Evaluation of quality of car efor congenital heart disease. Senin Thorac Cardiovasc Surg Pediatr Card Surg Annu. 2005;8 :157–67.
[8] de Leval MR Carthey J Wright DJ Farewell VT Reason JT . Human factors and cardiac surgery: a multicenter study. J Thorac Cardiovasc Surg. 2000;119 :661–72.10733754
[9] Nathan M Karamichalis JM Liu H . Surgical technical performance scores are predictors of late mortality and unplanned reinterventions in infants after cardiac surgery. J Thorac Cardiovasc Surg. 2012;144 :1095–101.e7.22939862
[10] Oztürk DY Oztürk E Ozcanoglu HD Tanidir IC Çetinkaya M Hatemi AC . Factors predicting early major adverse events in the intensive care unit after successful cardiac surgery for congenital heart disease in full-term neonates. Braz J Cardiovasc Surg. 2023;38 :e20220442.37540728
[11] Larrazabal LA del Nido PJ Jenkins KJ . Measurement of technical performance in congenital heart surgery: a pilot study. Ann Thorac Surg. 2007;83 :179–84.17184656
[12] Pasquali SK Jacobs ML He X . Variation in congenital heart surgery costs across hospitals. Pediatrics. 2014;133 :e553–60.24567024
[13] Nathan M Karamichalis J Liu H . Technical Performance Scores are strongly associated with early mortality, postoperative adverse events, and intensive care unit length of stay—analysis of consecutive discharges for 2 years. J Thorac Cardiovasc Surg. 2014;147 :389–94, 396.e1.24035318
[14] Sengupta A Gauvrea K Kohlsaat K . Intraoperative technical performance score predicts outcomes after congenital cardiac surgery. Ann Thorac Surg. 2023;115 :471–8.35595087
[15] Michalowski AK Gauvreau K Kaza A . Technical performance score: a predictor of outcomes after the norwood procedure. Ann Thorac Surg. 2021;112 :1290–7.32987019
[16] Shuhaiber J Gauvreau K Thiagarjan R . Congenital heart surgeon’s technical proficiency affects neonatal hospital survival. J Thorac Cardiovasc Surg. 2012;144 :1119–24.22421402
[17] Karamichalis JM Colan SD Nathan M . Technical performance scores in congenital cardiac operations: a quality assessment initiative. Ann Thorac Surg. 2012;94 :1317–23; discussion 1323.22795058
[18] Karamichalis JM Thiagarajan RR Liu H Mamic P Gauvreau K Bacha EA . Stage I Norwood: Optimal technical performance improves outcomes irrespective of preoperative physiologic status or case complexity. J Thorac Cardiovasc Surg. 2010;139 :962–8.20074754
[19] Nathan M Pigula FA Liu H . Inadequate technical performance scores are associated with late mortality and late reintervention. Ann Thorac Surg. 2013;96 :664–9.23782646
[20] Nathan M Gauvreau K Samnaliev M . Technical performance score predicts resource utilization in congenital cardiac procedures. J Am Coll Cardiol. 2016;67 :2696–8.27256837
[21] Nathan M Karamichalis JM Liu H . Intraoperative adverse events can be compensated by technical performance in neonates and infants after cardiac surgery:A prospective study. J Thorac Cardiovasc Surg. 2011;142 :1098–107, 1107.e1.21840545
