
==== Front
Europace
Europace
europace
Europace
1099-5129
1532-2092
Oxford University Press UK

39257213
10.1093/europace/euae229
euae229
Translational Research
AcademicSubjects/MED00200
Eurheartj/1
Eurheartj/4
Surgical skill simulation training to proficiency reduces procedural errors among novice cardiac device implanters: a randomized study
https://orcid.org/0000-0002-1618-3169
Mascheroni Jorio Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
Department of Cardiac Rhythm Management Training & Education, Medtronic International Trading Sàrl, Route du Molliau 31, 1131 Tolochenaz, Switzerland

https://orcid.org/0000-0002-6927-9420
Stockburger Martin Department of Cardiology and Internal Medicine, Havelland Kliniken, Ketziner Straße 21, 14641 Nauen, Germany
Institute of Medical Sociology and Rehabilitation Science, Charité—Universitaetsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany

https://orcid.org/0000-0002-3166-0816
Patwala Ashish Department of Cardiology, University Hospital of North Midlands, Newcastle Road, ST4 6QG Stoke-on-Trent, UK

https://orcid.org/0000-0002-8115-5906
Mont Lluís Department of Cardiology, Hospital Clinic, Universitat de Barcelona, C/Villarroel 170, 08036 Barcelona, Catalonia, Spain
Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), c/Rosselló 149, 08036 Barcelona, Catalonia, Spain
Centro de Investigación Biomédica en Red Cardiovascular (CIBERCV), Av. Monforte de Lemos 3-5, 28029 Madrid, Spain

https://orcid.org/0000-0002-3893-3944
Rao Archana Department of Cardiology, Liverpool Heart and Chest Hospital, Thomas Drive, L14 3PE Liverpool, UK

Retzlaff Hartwig Training Concept Consulting, Alpenstrasse 14a, D-82194 Groebenzell, Germany

https://orcid.org/0000-0002-2173-1035
Garweg Christophe Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
Department of Cardiology, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium

https://orcid.org/0000-0003-0887-5613
Gallagher Anthony G Orsi Academy, Proefhoevestraat 12, 9090 Melle, Belgium
Faculty of Medicine, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
School of Medicine, Faculty of Life and Health Sciences, Ulster University, Magee Campus, Northland Rd, BT48 7JL Londonderry, UK

https://orcid.org/0000-0003-1461-3986
Verbelen Tom Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
Department of Cardiac Surgery, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium

Corresponding author. Tel: +41(0)218027727. E-mail address: jorio.mascheroni@yahoo.com
Conflict of interest: J.M. conducted the present research project as part of his PhD studies at KU Leuven, Belgium. J.M. is also an employee of Medtronic. M.S. reports personal fees (honoraria for educational activity) from Medtronic during the conduct of the project and personal fees from Biotronik outside the submitted work; A.P. reports personal fees (honoraria for educational activity) from Medtronic during the conduct of the project; L.M. reports grants and personal fees (support for Fellowship programme, research grants, consulting honoraria) from Medtronic, Abbott, and Boston Scientific during the conduct of the project; A.R. reports teaching honoraria from Medtronic outside the submitted work; H.R. reports personal fees (honoraria for educational activity) from Medtronic during the conduct of the project and outside the submitted work; C.G. reports research funding from Abbott, Biotronik, and Medtronic, and speaker/consultancy fees from Medtronic, Biotronik, and Abbott outside the submitted work; A.G.G. reports personal fees (consulting) from Medtronic during the conduct of the project; T.V. reports personal fees (lectures honoraria) from Medtronic outside the submitted work.

9 2024
11 9 2024
11 9 2024
26 9 euae22911 7 2024
01 9 2024
19 9 2024
© The Author(s) 2024. Published by Oxford University Press on behalf of the European Society of Cardiology.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.

Abstract

Aims

In cardiac device implantation, having both surgical skills and ability to manipulate catheter/lead/wire is crucial. Few cardiologists, however, receive formal surgical training prior to implanting. Skills are mostly acquired directly on-the-job and surgical technique varies across institutions; suboptimal approaches may increase complications. We investigated how novel proficiency-based progression (PBP) simulation training impacts the surgical quality of implantations, compared to traditional simulation (SIM) training.

Methods and results

In this international prospective study, novice implanters were randomized (blinded) 1:1 to participate in a simulation-based procedure training curriculum, with proficiency demonstration requirements for advancing (PBP approach) or without (SIM). Ultimately, trainees performed the surgical tasks of an implant on a porcine tissue that was video-recorded and then scored by two independent assessors (blinded to group), using previously validated performance metrics. Primary outcomes were the number of procedural Steps Completed, Critical Errors, Errors (non-critical), and All Errors Combined. Thirty novice implanters from 10 countries participated. Baseline experiences were similar between groups. Compared to SIM-trained, the PBP-trained group completed on average 11% more procedural Steps (P < 0.001) and made 61.2% fewer Critical Errors (P < 0.001), 57.1% fewer Errors (P = 0.140), and 60.7% fewer All Errors Combined (P = 0.001); 11/15 (73%) PBP trainees demonstrated the predefined target performance level vs. 3/15 SIM trainees (20%) in the video-recorded performance.

Conclusion

Proficiency-based progression training produces superior objectively assessed novice operators’ surgical performance in device implantation compared with traditional (simulation) training. Systematic PBP incorporation into formal academic surgical skills training is recommended before in vivo device practice. Future studies will quantify PBP training’s effect on surgery-related device complications.

Graphical Abstract

Graphical Abstract

Training
Surgical skill
Proficiency-based progression
CIED implant
Pacing
Cardiac resynchronization therapy
Medtronic Europe 10.13039/100020192
==== Body
pmcWhat’s new?

Few cardiologists receive formal surgical training before implanting cardiac devices in vivo. In this prospective randomized study, two alternative simulation training approaches were compared and differed only in the training progression criteria.

Surgical skills required for cardiac device implantation were assessed using explicit objective intraoperative performance metrics validated previously, derived from expert consensus specifically for teaching purposes.

The proficiency-based progression (PBP) simulation training method generated superior, objectively assessed, operator surgical performance compared to traditional simulation (SIM) training, resulting on average in 61% fewer surgical errors.

The number of trainees demonstrating the target performance level at the end was 73% in the PBP study group vs. 20% in the SIM group.

Formal surgical skills training and assessment using PBP would strengthen academic/institutional curricula for novice implanters and ensure skill acquisition prior to in vivo practice.

Introduction

Every year, ∼1.7 million patients affected by cardiac rhythm disorders receive a new cardiac implantable electronic device (CIED) worldwide.1 While technologies have rapidly evolved and therapy benefits expanded over time,2 severe complications still affect a number of implanted patients resulting in increased morbidity, mortality, and healthcare costs.3–10 In CIED implantation, having both surgical skills and the ability to manipulate catheter/lead/wire is crucial. Suboptimal surgical techniques may lead to various complications,11 including pneumothorax (incidence range: 0.2–3.7%12), haematoma formation (1.0–16.6%13), system lead dislodgement (1–8%14), and infections (1–4%8). Practices differ across CIED implanting centres and operators15,16 regarding procedure phases such as incision and pocket location, venous access, lead fixation, and wound closure.

For cardiology trainees implanting CIEDs, the significance of training in surgical techniques is often overlooked compared to the focus on handling implant tools guided by fluoroscopy. While level III competence in CIED implant is required by Heart Rhythm specialty training curricula across the world,17,18 standards for surgical techniques in CIEDs are not established, and very few cardiologists receive formal surgical training.19 Current training is still largely based on the apprenticeship model established by William Halsted20 more than a century ago, although labour regulations, hospital infrastructure organization, and clinical practice have markedly changed over the last few decades and challenged such model. Institutional curricula vary and often reflect local senior operators’ practice; novice implanters learn surgical skills in vivo, resulting in increased patient risks at the beginning of an operator’s learning curve.21–23

To ensure patients receive optimal and consistent quality of care, irrespective of physicians’ expertise and training institution, surgical skills training for CIED implantation (and early-stage training in general) should take place in a risk-free (simulated) environment first, to continue and augment learning at the bedside when the fundamental skills are acquired and honed. Objective target performance levels should be established up-front and demonstrated by each trainee prior to starting in vivo practice. A task-specific system of performance measurement, or ‘Metrics’, is necessary and should be adopted as a standard for training in a simulated environment first. Trainees should advance only upon demonstration of the proficiency requirements. This approach is known as proficiency-based progression (PBP)24–26 training. Over the last two decades, PBP training has consistently generated superior trainees’ performance outcomes (≈60%) compared to conventional (simulation) training in other areas of procedural medicine,27 and may represent a viable solution to help reducing surgical errors and associated complications in CIEDs.28 Hence, intraoperative PBP Metrics were recently developed for a ‘reference’ CIED implant procedure and its surgical aspects.29 Metrics are of three types: ‘Step’ (a small functional element of performance of the procedure), ‘Error’ (indicates that the action is considered incorrect), and ‘Critical Error’ (an error that may compromise the safety/success of the therapy). Proficiency-based progression Metrics were constructed using unambiguous operational definitions to be scored binarily (occurring/not occurring) with a high degree of reliability.30 To investigate the value of the PBP approach in surgical skills training for novice CIED implanters, we hypothesized that a novel PBP simulation training programme would be superior to a traditional simulation programme. We sought to evaluate the training effect on CIED implanters’ surgical performance in a prospective, randomized, and controlled study.

Methods

Study design

This was an international, prospective, randomized (1:1), double-blinded (trainees and assessors), controlled study (ClinicalTrials.gov ID: NCT05952908) conducted by KU Leuven, Belgium, from March to December 2022. The study was approved by the local ethics committee, and written informed consent was obtained from all participants. Data were collected at a skill centre in Tolochenaz, Switzerland.

Participants

Participants were physicians engaged in CIED implantation from Europe and Israel, navigating the early stages of their careers. Subjects voluntarily enrolled in a peer-to-peer simulation-based training curriculum that specifically emphasized CIED implantation techniques. This curriculum served as an optional supplement to their academic and institutional education. Candidates had to strictly fulfil the following inclusion criteria: (1) actively practice CIED implantation at time of enrolment; (2) have previously implanted minimum 20 pacemakers/defibrillators as first operator, and minimum three cardiac resynchronization therapy (CRT) systems as second operator; and (3) be familiar with English language. They were excluded if they had implanted ≥200 CRT systems as first operator.

Interventions

The methods employed in this study are similar to those used in the investigation of skills for fluoroscopy-guided lead positioning,31 but here focusing on the surgical tasks involved in a CIED procedure. Participants were blinded and randomly assigned in a 1:1 ratio to undergo one of two alternative training curricula for ‘skin-to-skin’ implantation of pacing and CRT systems: a traditional simulation programme (referred to as SIM) or a PBP simulation programme (referred to as PBP). Both curricula consisted of the same eLearning training component (4–6 h), summative knowledge check, and peer-to-peer simulation-based component (2 days). The key distinction was in how subjects progressed through the stages of training. Proficiency-based progression trainees had to demonstrate a predefined level of performance (proficiency benchmark) verified by the instructors before advancing: the proficiency ‘checks’ were formative and continuous throughout each training module (to acquire the necessary knowledge/skill) and summative at the end of each module (to verify the proficiency benchmark demonstration). Trainees were required to demonstrate the benchmark in the online module (the number of test attempts was unlimited) to be able to proceed to the simulation-based module. The optimal performance of the procedure had been earlier characterized and verified through international expert consensus for training purposes.29 The benchmarks were quantitatively defined using objective, procedure-specific performance Metrics that had been previously developed and its application validated for the same purpose, context, and learner profile.29,30 Proficiency-based progression benchmarks were calculated as the average scores obtained by experienced implanters performing on the exact same procedural tasks. For surgical skills training, the proficiency benchmark was defined as a maximum of two errors allowed over the entire surgical procedure. In contrast, SIM trainees were not provided with quantitative pass/fail thresholds to advance. The study/training flowchart is illustrated in Figure 1.

Figure 1 Study/training flowchart.

Senior high-volume implanters, all at consultant level, with over 10 years of device implant experience and accustomed to coaching junior operators in clinical practice and simulation, delivered the skills training. Prior to the study, the PBP group’s faculty underwent training on how to apply the PBP method, while the SIM group’s faculty were not exposed to it. The skills training concluded when a trainee was deemed ‘ready’ (within the maximum available time for practice scheduled on the training agenda, the same for both groups ∼10 h): ‘readiness’ was determined by the instructor’s personal and professional judgment in the SIM arm and the demonstration of the quantitatively defined proficiency benchmark in the PBP arm. The groups were allowed to use the hands-on time allocated in the agenda entirely; individual time-to-readiness was not tracked.

The CIED surgical skills training and assessment encompassed the following tasks, simulated for a device system procedure comprising right ventricular and atrial leads: injection of local anaesthetic, skin incision, device pocket creation, lead fixation with sutures, device-lead connection, system insertion into pocket, and wound closure. Such tasks had been demonstrated earlier in the online module. An outline of the surgical steps covered in the eLearning, as well as an example of a performance Metrics scoresheet utilized in the skill training, is illustrated in Supplementary material online, Appendix S1. The surgical simulation training was conducted in a skills lab on a piece of porcine tissue (food quality, procured from large-scale food retailer), using the same surgical instruments, disposable materials (i.e. syringes, needles, and sutures), pacing leads, and devices as in a real-world procedure (Figure 2). The porcine tissue was prepared and presented at room temperature wrapped in a surgical drape opened in the region of intervention. Trainees were equipped with surgical gloves and gowns. One tissue model per trainee was utilized during the coaching sessions and a second identical piece for skill assessment.

Figure 2 Skills training for the surgical CIED procedure on porcine tissue. Tasks illustrated: (A) lead fixation; (B) leads-device connection; (C) system insertion into pocket; (D) wound closure. CIED, cardiac implantable electronic device.

The summative skills assessment consisted of a ‘solo’ performance of the same surgical tasks practiced under expert coaching. Simulation performances were video-recorded in an anonymous fashion (subjects were not recognizable) using a video camera positioned over the trainee’s workspace, filming operator’s hands working on the tissue.

Participation in the study concluded upon trainee departure from the skill centre. Before leaving, trainees were requested to complete a satisfaction survey about the training curriculum.

Outcomes

The primary outcome of the study encompassed four types of performance Metrics, objectively assessed from each trainee’s video recording of the surgical performance: number of procedural Steps Completed, Critical Errors, and Errors (uncritical) were the dependent variables and scored separately, while All Errors Combined cumulated all deviations from optimal performance. The primary analysis involved comparing the performances of SIM and PBP groups for each of the four variables separately. The pre-specified secondary outcomes and comparisons between the two groups included: (1) procedure duration; (2) trainees’ scores (expressed as percentage of correct answers) obtained in the surgical section of the online test; (3) number of trainees per study arm demonstrating the proficiency benchmark in the video-recorded performance; and (4) participants’ satisfaction derived from the training evaluation forms.

Outcomes were assessed after all participants completed the study. All videos were reviewed and scored by two independent assessors (who were not teaching faculty but had >20 years of experience in CIED therapy) blinded to the group. For each video, the assessors’ scores were compared, and the inter-rater reliability (IRR) was subsequently calculated using the formulae: IRR = agreements/(agreements + disagreements) ∗ 100.

Sample size, randomization, and blinding

The sample size calculations were derived from the Metrics construct validity30 and the Transfer of Training32 studies: for the PBP group, a hypothesized reduction of 42% in errors was considered.32 Thus, with α = 0.05 and β = 0.20, it was determined that a minimum of 14 subjects were required in each group to achieve statistical significance.

At enrolment, participants were randomly assigned to follow one of the two alternative training curricula (SIM or PBP). The training curriculum manager generated a set of random numbers using online software (Research Randomizer, Social Psychology Network, https://www.randomizer.org), and sequentially revealed them as training applications arrived.

To ensure participant blinding to their training group, the structure, content, and agenda of the two alternative training curricula were identical.

Statistics

Differences between groups were assessed for each primary and secondary outcome variable separately with SPSS Statistics software (IBM, Armonk, NY, USA). The significance level was set at 0.05 for each comparison, utilizing a two-sided test with the Mann–Whitney U test for continuous variables; a χ2 test of independence was performed to assess the relationship between study group (PBP/SIM) and proficiency benchmark demonstration (YES/NO).

Results

Participant flow and baseline data

The study included 16 novice CIED implanters per group who were randomly assigned. One subject per group was unable to participate, resulting in 15 trainees in each group who received training, completed the study, and were analysed for primary outcomes based on an intention-to-treat approach. The study flow diagram is depicted in Figure 3. Trainees came from 10 different countries in Western Europe and Israel. Baseline implant experience of SIM vs. PBP groups was similar: a mean (SD) of 1.9 (2.1) vs. 2.1 (1.6) years spent in device implant practice after specialty training (P = 0.428), a median [interquartile range (IQR)] of 50 (35–125) vs. 45 (30–110) pacemakers/defibrillators (P = 0.588), and 3 (1–5) vs. 5 (2–15) CRT systems (P = 0.127) implanted as first operator.

Figure 3 Study flow diagram.

Outcomes

The evaluation comprised 30 videos (mean length = 32.0 min) of the trainees performing the surgical phases of the implant procedure, independently scored by the two assessors. Each performance was scored using 27 intraoperative metrics. The IRR between assessors, with a mean of 0.972 (95% CI: 0.959, 0.984), demonstrated high agreement.

The performances of SIM- and PBP-trained groups were analysed for each measured variable, and primary outcomes are presented in Figure 4. Objective assessment using validated performance metrics showed that PBP trainees consistently outperformed SIM trainees in the surgical phases of the implant procedure. In comparison to the SIM group, the PBP group completed, on average, 11.0% more procedural Steps and made 57.1% fewer Errors, 61.2% fewer Critical Errors, and 60.7% fewer All Errors Combined (total Errors + Critical Errors). Apart from Errors, all differences were statistically significant, and the effect size was large33 (see Table 1). Critical Errors more likely to be made by SIM trainees were for example non-compliance with lead anchoring sleeve suturing method, cleaning lead connector before insertion into device, confirming lead fixation in device header by pull-test.

Figure 4 Primary study outcomes from the surgical CIED performance videos. Median, 25th, and 75th rank scores of the objectively assessed performance Metrics in the surgical CIED procedure videos: number of Steps Completed, Critical Errors, Errors (non-critical), All Errors Combined (Errors + Critical Errors). The boxes indicate medians (central line) and 75% and 25% quartiles (upper and lower ends); whiskers represent maximum and minimum values (without outliers); and circles represent outliers. CIED, cardiac implantable electronic device; PBP, proficiency-based progression simulation training group; SIM, traditional simulation training group.

Table 1 Descriptive statistic measures and statistical analysis results of study groups’ surgical CIED performances

Dependent variable	Median (IQR)	Mann–Whitney U	P value	Effect size r (rank-biserial correlation)	
SIM (n = 15)	PBP (n = 15)	
Performance metrics						
 No. Steps Completed	21 (20–22)	24 (23–24)	23.5	<0.001	0.79	
 No. Critical Errors	3 (3–4)	1 (0–2)	32.0	<0.001	0.72	
 No. Errors	0 (0–1)	0 (0–0)	84.5	0.140	0.25	
 No. All Errors Combined	4 (3–5)	1 (0–3)	35.0	0.001	0.69	
Conventional measures						
 Procedure duration (min)	32.6 (29.9–34.3)	32.3 (28.5–34.5)	105	0.772	0.07	
CIED, cardiac implantable electronic device; PBP, proficiency-based progression simulation training group; SIM, traditional simulation training group.

The data on procedure duration, presented in Table 1, did not show a statistically significant difference between groups, and the observed effect size was small.

In the online summative knowledge assessment, all PBP trainees demonstrated the proficiency benchmark and all SIM trainees scored below that benchmark (no threshold was specified for the SIM arm). In the subgroup of questions related to surgical tasks, the average score of the PBP group was 23% higher than that of the SIM group (median [IQR] 91.7 [91.7–100] vs. 75.0 [66.7–87.5], P < 0.001).

In the objectively assessed video-recorded implant performance, 11/15 (73%) trainees in the PBP group and 3/15 (20%) in the SIM group demonstrated the skills proficiency benchmark. The relationship between training group and benchmark demonstration was significant, χ2 (1, n = 30) = 8.57, P = 0.003.

Regarding the anonymous training evaluation forms, all 30 participants rated the quality of the programme ‘excellent’ and were extremely satisfied with their teaching faculty; 13/15 (87%) trainees in each group considered the surgical part of the training interesting and relevant to their clinical practice.

Discussion

In this study, we evaluated the impact of a PBP simulation training approach on the surgical performances of novice CIED implanters in comparison to a traditional simulation training approach. Participants were randomized to follow one of two alternative training curricula, which differed only for the metrics-based progression criteria. In their final video-recorded surgical performance (scored using binary intraoperative Metrics) the PBP-trained group completed significantly more procedure Steps and made significantly fewer objectively assessed performance Errors + Critical Errors overall, as compared to the SIM-trained group. Moreover, 73% of the PBP trainees demonstrated the quantitatively defined proficiency benchmark as opposed to 20% of the SIM. The high IRR level underlined the score reliability of the Metrics used for assessment.

These results indicate that a PBP approach to training surgical skills for CIED implantation improves the quality of the execution and is more effective in achieving a predetermined performance standard than a traditional (simulation-based) training method. This conclusion is supported by the comparable implant experience of the two groups at baseline, and the similarity of two curricula, with the only difference lying in the PBP training requirement. Compared to the traditional approach, PBP seems to accelerate skills acquisition towards the performance goal primarily through a conspicuous reduction in total number of procedural errors. This is likely driven by the forced attention to detail in the metrics-based instruction and the strictly monitored and mentored progression. In fact, during deliberate practice,34 PBP trainees receive immediate, explicit, transparent, and formative objective feedback on each deviation from the ‘reference’ procedure. These ‘deviations’ had been agreed by international experts as a standard for teaching. Furthermore, PBP trainees are invited to repeat, under supervision, any incorrect element, sequence, or phase of the implant, until the target performance level—quantified by the metrics—is demonstrated, while SIM trainees are asked to redo their tasks when the instructors feel unsatisfied with the performance based on their (subjective) expert judgement.

Our results confirm that simulation is a useful tool supporting skills development, but it’s the quality of the (PBP) training curriculum that drives skills acquisition, not the simulator.

The observed reduction in total errors was mainly driven by the Critical Errors made by the two groups, while the number of Errors was low in both arms. This finding is not surprising because most of the deviations from the optimal performance in the surgical aspects of the procedure were classified and agreed by international expert consensus as ‘Critical Errors’ metrics rather than ‘Errors’, due to the risk they involve. Thus, the final results reflected this type of metrics distribution.

In our study, we also observed a modest but significant increase in number of Steps Completed by the PBP group vs. the SIM group. There was no difference in procedure duration. Steps and time are not very good predictors of skill because Steps may be completed but performed badly, and procedures may be done quickly but unsafely, or procedure phases may be omitted. Finally, the training satisfaction survey confirmed that individual trainee’s perception of training quality is a poor indicator of actual skill acquisition.

The contribution of the eLearning module to the final performance results should not be underestimated. Proficiency-based progression trainees were required to demonstrate the proficiency benchmark in the online knowledge test before proceeding to the skills training; a clear objective that most likely increased their focus during the eLearning. Their knowledge scores were significantly higher than those of the SIM group, and their effort resulted into better understanding and retention of the recommended ‘reference’ procedure. Consequently, they probably used their time in the skills lab more efficiently, resulting in more efficient skills development.

It is important to highlight that simulators are rarely utilized in contemporary cardiology training programmes.35 Hence, the ‘control’ group in this study inherently had the advantage of supplemental simulation-enhanced training over typical trainees in the field. The performance disparity would almost certainly have been even more pronounced if PBP had been compared to the traditional apprenticeship model, as demonstrated in previous research.36,37 This study chose to test PBP against conventional simulation to underscore that simulation by itself doesn’t guarantee a specific level of proficiency. Thus, simulators should not be regarded as the panacea for current and future educational frameworks.

To our knowledge, this is the first study investigating the effect of metrics-based training on the surgical tasks of a CIED implantation. Surgical performance involves a different skillset from fluoroscopy-guided catheter/lead/wire manipulation and contributes significantly to determining procedure outcomes. It needs therefore, to be developed through dedicated training and verified against a proficiency benchmark before being applied to real implants. The present findings align with previous research on PBP conducted in various domains of procedural medicine. A virtual reality simulation study focusing on the placement of the three leads for a CRT device reported a 77% reduction in overall errors with PBP vs. traditional simulation training.31 A systematic review and meta-analysis from 12 studies (in various domains such as laparoscopic surgery, ultrasound-guided peripheral nerve block, arthroscopic Bankart procedure, clinical communication, carotid and coronary artery angiography, and knot tying) showed that PBP reduced the number of performance errors by 60% in comparison to quality-assured, conventional but non-PBP training.27

Current surgical practice in CIED implantation varies greatly across countries and institutions: an international survey of the European Heart Rhythm Association among 62 centres showed differences in device implant side, pocket incision tools, vein accessed for lead implantation, and generator placement site.15 Similar variations were reported across 11 hospitals of the same region in England regarding types of suture used for securing leads, suturing technique for pocket closure, skin closure, and pressure dressing.11 Sometimes adaptations even violate product manufacturers’ instruction for use.38 Ideally any alternative technique should be proven safe before applying it to patients. Considering the high incidence of surgery-related complications in CIEDs, it is surprising that surgical techniques for CIED procedures are not prioritized and formally integrated in training curricula for cardiologists.19 Complications not only affect patients’ morbidity and mortality, but add significant costs to the healthcare system; for instance, pocket complication, thoracic trauma, and infection increased treatment cost, respectively, by $35 543, $67 573, and $80 247 in a US database of healthcare claims.4 The association between technical skill and clinical outcome has already been demonstrated in various domains of procedural medicine,39,40 and future studies are likely to confirm such relationship in CIED implantation. Among the surgical skills developed in this PBP curriculum, for instance, proper device pocket creation and wound closure in layers may reduce risks of bleeding and infection; adequate lead securing may prevent lead dislodgment or lead damage (i.e. risk of incorrect therapy delivery). It should also be noted that surgical technique alone is not expected to eliminate all complications, because patient specific factors such as body habitus, nutritional status, and antiplatelet agents play a role too.

In summary, surgical techniques applied to cardiac device implantation vary across centres and operators due to lack of standards and limited formal surgical training opportunities, especially for cardiologists. The present study has demonstrated that, among novice CIED implanters, a PBP approach to surgical skills training significantly reduces objectively assessed procedural Errors + Critical Errors overall compared to traditional training. Ideally, a PBP (simulation) training paradigm should be implemented from the early stages of an electrophysiologist’s career (i.e. at the beginning of the learning curve) and continue throughout their working life whenever a new skill needs to be mastered.

Study limitations

One limitation concerns the finite rather than unlimited duration of the PBP skills training in the present study, due to pre-arranged travel logistics. Proficiency-based progression instruction is by default not time- but performance-based and adapts to different learning speeds. This constraint resulted in 4/15 PBP trainees missing the proficiency benchmark. Had they practiced with metrics-based feedback for (a little) longer, they could have further reduced their error rate and met the benchmark too. Still, over the same training duration, 11 PBP but only three SIM trainees achieved the proficiency benchmark, indicating that PBP is also a more efficient approach.

Another limitation concerns the fact that the faculty was different in the two study groups, although they had an equivalent senior operative and teaching experience profile. The reason is that PBP instructors had to be trained up-front on the implementation of metrics-based training; had they taught the SIM group too, such new skill would have probably affected their traditional teaching approach, so it was chosen to avoid that bias. On the positive side, they were all excellently and equally evaluated by trainees.

Conclusions

A PBP training for surgical skills in cardiac device implantation generates superior objectively assessed novice operators’ performance compared with a traditional (simulation) training approach. The findings of the present study suggest that formal surgical training for CIEDs should be systematically implemented in academic/institutional curricula using a PBP approach. Proficiency-based progression will ensure that trainees demonstrate a predetermined, objectively assessed performance level before starting with (supervised) implantations of real patients. Future studies will quantify the effect of PBP training on the reduction of surgical-related CIED complications and therapy outcomes.

Supplementary Material

euae229_Supplementary_Data

Acknowledgements

No artificial intelligence programs contributed to the compilation of the present manuscript.

Supplementary material

Supplementary material is available at Europace online.

Funding

The present research project has been conducted by J.M. as part of his PhD studies in Biomedical Sciences at KU Leuven, Belgium. The training was supported by Medtronic Europe that provided the training infrastructure but did not fund/influence: design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data availability

The experimental data used to support the findings of this study are available from the corresponding author upon reasonable request.
==== Refs
References

1 Ferrick  AM, Raj  SR, Deneke  T, Kojodjojo  P, Lopez-Cabanillas  N, Abe  H  et al  2023 HRS/EHRA/APHRS/LAHRS expert consensus statement on practical management of the remote device clinic. Heart Rhythm  2023;20 :e92–144.37211145
2 Defaye  P, Biffi  M, El-Chami  M, Boveda  S, Glikson  M, Piccini  J  et al  Cardiac pacing and lead devices management: 25 years of research at EP Europace journal. Europace  2023;25 :euad202.37421338
3 Ascoeta  MS, Marijon  E, Defaye  P, Klug  D, Beganton  F, Perier  M-C  et al  Impact of early complications on outcomes in patients with implantable cardioverter-defibrillator for primary prevention. Heart Rhythm  2016;13 :1045–51.26749313
4 Cantillon  DJ, Exner  DV, Badie  N, Davis  K, Gu  NY, Nabutovsky  Y  et al  Complications and health care costs associated with transvenous cardiac pacemakers in a nationwide assessment. JACC Clin Electrophysiol  2017;3 :1296–305.29759627
5 Haugaa  KH, Potpara  TS, Boveda  S, Deharo  J-C, Chen  J, Dobreanu  D  et al  Patients’ knowledge and attitudes regarding living with implantable electronic devices: results of a multicentre, multinational patient survey conducted by the European Heart Rhythm Association. Europace  2018;20 :386–91.29300970
6 Kipp  R, Hsu  JC, Freeman  J, Curtis  J, Bao  H, Hoffmayer  KS. Long-term morbidity and mortality after implantable cardioverter-defibrillator implantation with procedural complication: a report from the National Cardiovascular Data Registry. Heart Rhythm  2018;15 :847–54.28987459
7 Clémenty  N, Fernandes  J, Carion  PL, de Léotoing  L, Lamarsalle  L, Wilquin-Bequet  F  et al  Pacemaker complications and costs: a nationwide economic study. J Med Econ  2019;22 :1171–8.31373521
8 Wilkoff  BL, Boriani  G, Mittal  S, Poole  JE, Kennergren  C, Corey  GR  et al  Impact of cardiac implantable electronic device infection. Circ Arrhythm Electrophysiol  2020;13 :e008280.32281393
9 Pietro  P, Federico  G, Gabriele  D, Ernesto  A, Matteo  Z, Mattia  L  et al  Impact on all-cause and cardiovascular mortality of cardiac implantable electronic device complications. JACC Clin Electrophysiol  2020;6 :382–92.32327071
10 van Barreveld  M, Verstraelen  TE, Buskens  E, van Dessel  PFHM, Boersma  LVA, Delnoy  PPHM  et al  Hospital utilisation and the costs associated with complications of ICD implantation in a contemporary primary prevention cohort. Neth Heart J  2023;31 :244–53.36434382
11 Naneishvili  T, Khalil  A, Prasad  N, Glancy  J. Variations in surgical technique for permanent pacemaker implantation in West Midlands. Br J Cardiol  2021;28 :6.35747483
12 Kotter  J, Lolay  G, Charnigo  R, Leung  S, McKibbin  C, Sousa  M  et al  Predictors, morbidity, and costs associated with pneumothorax during electronic cardiac device implantation. Pacing Clin Electrophysiol  2016;39 :985–91.27230623
13 Kewcharoen  J, Kanitsoraphan  C, Thangjui  S, Leesutipornchai  T, Saowapa  S, Pokawattana  A  et al  Postimplantation pocket hematoma increases risk of cardiac implantable electronic device infection: a meta-analysis. J Arrhythm  2021;37 :635–44.34141016
14 Qin  D, Filippaios  A, Murphy  J, Berg  M, Lampert  R, Schloss  EJ  et al  Short- and long-term risk of lead dislodgement events: real-world experience from product surveillance registry. Circ Arrhythm Electrophysiol  2022;15 :e011029.35925831
15 Bongiorni  MG, Proclemer  A, Dobreanu  D, Marinskis  G, Pison  L, Blomstrom-Lundqvist  C  et al  Preferred tools and techniques for implantation of cardiac electronic devices in Europe: results of the European Heart Rhythm Association survey. Europace  2013;15 :1664–8.24170423
16 Traykov  V, Bongiorni  MG, Boriani  G, Burri  H, Costa  R, Dagres  N  et al  Clinical practice and implementation of guidelines for the prevention, diagnosis and management of cardiac implantable electronic device infections: results of a worldwide survey under the auspices of the European Heart Rhythm Association. Europace  2019;21 :1270–9.31209483
17 Merino  JL, Arribas  F, Botto  GL, Huikuri  H, Kraemer  LI, Linde  C  et al  Core curriculum for the heart rhythm specialist. Europace  2009;11 :iii1–26.19710176
18 Zipes  DP, Calkins  H, Daubert  JP, Ellenbogen  KA, Field  ME, Fisher  JD  et al  2015 ACC/AHA/HRS advanced training statement on clinical cardiac electrophysiology (a revision of the ACC/AHA 2006 update of the clinical competence statement on invasive electrophysiology studies, catheter ablation, and cardioversion). Heart Rhythm  2016;13 :e3–37.26392325
19 Khavandi  A, Hamilton  S, Fitzpatrick  A, Wright  DJ, Lewis  M, Harcombe  A  et al  Cardiologists should have basic surgical skills training. Heart  2010;96 :741–2.20448124
20 Cameron  JL . William Stewart Halsted: our surgical heritage. Ann Surg  1997;225 :445–58.9193173
21 Eberhardt  F, Bode  F, Bonnemeier  H, Boguschewski  F, Schlei  M, Peters  W  et al  Long term complications in single and dual chamber pacing are influenced by surgical experience and patient morbidity. Heart  2005;91 :500–6.15772212
22 Haug  B, Kjelsberg  K, Lappegård  KT. Pacemaker implantation in small hospitals: complication rates comparable to larger centres. Europace  2011;13 :1580–6.21712283
23 Leong  KMW, Pollard  C, Cooke  CJ. Cardiology registrars and permanent pacemaker complication rates in a district general hospital—safety and service implications. Clin Med (Lond)  2014;14 :34–7.24532741
24 Gallagher  AG, Ritter  EM, Champion  H, Higgins  G, Fried  MP, Moses  G  et al  Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg  2005;241 :364–72.15650649
25 Gallagher  AG . Metric-based simulation training to proficiency in medical education: what it is and how to do it. Ulster Med J  2012;81 :107–13.23620606
26 Gallagher  AG, O’Sullivan  GC. Fundamentals of surgical simulation: principles and practice. London: Springer-Verlag; 2012.
27 Mazzone  E, Puliatti  S, Amato  M, Bunting  B, Rocco  B, Montorsi  F  et al  A systematic review and meta-analysis on the impact of proficiency-based progression simulation training on performance outcomes. Ann Surg  2021;274 :281–9.33630473
28 Mascheroni  J, Mont  L, Stockburger  M, Patwala  A, Retzlaff  H, Gallagher  AG. The imperative of consistency and proficiency in cardiac devices implant skills training. Open Heart  2021;8 :e001629.33972405
29 Mascheroni  J, Mont  L, Stockburger  M, Patwala  A, Retzlaff  H, Gallagher  AG  et al  International expert consensus on a scientific approach to training novice cardiac resynchronization therapy implanters using performance quality metrics. Int J Cardiol  2019;289 :63–9.31088758
30 Mascheroni  J, Mont  L, Stockburger  M, Patwala  A, Retzlaff  H, Gallagher  AG. A validation study of intraoperative performance metrics for training novice cardiac resynchronization therapy implanters. Int J Cardiol  2020;307 :48–54.32081470
31 Mascheroni  J, Stockburger  M, Patwala  A, Mont  L, Rao  A, Retzlaff  H  et al  Effect of metrics-based simulation training to proficiency on procedure quality and errors among novice cardiac device implanters: the IMPROF randomized trial. JAMA Netw Open  2023;6 :e2322750.37651144
32 Gallagher  AG, Seymour  NE, Jordan-Black  J-A, Bunting  BP, McGlade  K, Satava  RM. Prospective, randomized assessment of transfer of training (ToT) and transfer effectiveness ratio (TER) of virtual reality simulation training for laparoscopic skill acquisition. Ann Surg  2013;257 :1025–31.23426342
33 Cohen  J . Statistical power analysis for the behavioral sciences. 2nd ed. New York: L. Erlbaum Associates; 1988.
34 Ericsson  KA . Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med  2015;90 :1471–86.26375267
35 Tjong  FVY, Perrotta  L, Goette  A, Duncker  D, Vernooy  K, Boveda  S  et al  Utilization of and perceived need for simulators in clinical electrophysiology: results from an EHRA physician survey. Europace  2024;26 :euae037.38293821
36 Angelo  RL, Ryu  RKN, Pedowitz  RA, Beach  W, Burns  J, Dodds  J  et al  A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set. Arthroscopy  2015;31 :1854–71.26341047
37 Breen  D, O’Brien  S, McCarthy  N, Gallagher  A, Walshe  N. Effect of a proficiency-based progression simulation programme on clinical communication for the deteriorating patient: a randomised controlled trial. BMJ Open  2019;9 :e025992.
38 Baroni  M, Gigli  L, Posca  F, Carbonaro  M, Leidi  F, Fortuna  M  et al  Direct anchoring technique for pacing and defibrillation leads inserted through cephalic vein: insight for a single high-volume center. Minerva Cardiol Angiol  2023;71 :342–8.36305778
39 Birkmeyer  JD, Finks  JF, O’Reilly  A, Oerline  M, Carlin  AM, Nunn  AR  et al  Surgical skill and complication rates after bariatric surgery. N Engl J Med  2013;369 :1434–42.24106936
40 Curtis  NJ, Foster  JD, Miskovic  D, Brown  CSB, Hewett  PJ, Abbott  S  et al  Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg  2020;155 :590–8.32374371
