
==== Front
Med Educ Online
Med Educ Online
Medical Education Online
1087-2981
Taylor & Francis

39244775
10.1080/10872981.2024.2396163
2396163
Version of Record
Research Article
Refocusing on the “E” in Continuing Medical Education (CME)
Benefits and limitations of the transfer online of Irish College of General Practitioners continuing medical education small group learning during the COVID pandemic: a national Delphi study
S. DOWLING ET AL.
MEDICAL EDUCATION ONLINE
Dowling Stephanie a b
Minihan Finola a
Duffy Ilona a
McNicholas Claire a
Doran Gillian a
Harrold Pat a
Burke John a
Cullen Walter b
a CME Small Group Tutor Network, Irish College of General Practitioners , Dublin 2, Ireland
b UCD School of Medicine, University College Dublin , Belfield, Dublin 4, Ireland
CONTACT Stephanie Dowling stephanie.dowling@icgp.ie CME Small Group Tutor Network, Irish College of General Practitioners, 4-5 Lincoln Place, Dublin 2 D02 XR68 Ireland
8 9 2024
2024
8 9 2024
29 1 2396163Integra07 9 2024
Integra07 9 2024
12 10 2023
16 3 2024
20 8 2024
© 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
2024
The Author(s)
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.

ABSTRACT

Background

In Ireland and internationally, small-group learning (SGL) has been shown to be an effective way of delivering continuing medical education (CME) and changing clinical practice.

Research Question

This study sought to determine the benefits and limitations, as reported by Irish GPs, of the change of CME-SGL from face-to-face to online learning during COVID.

Methods

GPs were invited to participate via email through their respective CME tutors. The first of three rounds of a survey using the Delphi method gathered demographic information and asked GPs about the benefits and/or limitations of learning online in their established small groups. Subsequent rounds obtained a consensus opinion.

Results

Eighty-eight GPs across Ireland agreed to participate. Response rates varied from 62.5% to 72% in different rounds. These GPs reported that attending their established CME-SGL groups allowed them to discuss the practical implications of applying guidelines in COVID care into practice (92.7% consensus), reviewing new local services and comparing their practice with others (94% consensus); helping them feel less isolated (98% consensus). They reported that online meetings were less social (60% consensus), and informal learning that occurs before and after meetings did not take place (70% consensus). GPs would not like online learning to replace face-to face-CME-SGL after COVID (89% consensus).

Conclusion

GPs in established CME-SGL groups benefited from online learning as they could discuss how to adapt to rapidly changing guidelines while feeling supported and less isolated. They report that face-to-face meetings offer more opportunities for informal learning.

KEYWORDS

CME online learning
other learning
continuing professional development
The author(s) reported that there is no funding associated with the work featured in this article.
==== Body
pmcIntroduction

Continuing medical education (CME) is part of the process of lifelong learning that all doctors undertake from medical school until retirement in order to keep their clinical knowledge and practice up to date [1]. In Ireland, CME for general practitioners (GPs) is delivered by a national network of 40 tutors based locally who coordinate ‘small group learning’ (SGL) sessions for 2–5 groups of GPs. Under the direction of the tutors, group leaders (who are group members) facilitate teaching which involves small groups of 8–12 GPs meeting in the evenings after work for approximately 2 hours to discuss cases, reflect on evidence presented in the meeting and consider what changes they will make to their own practice. Tutors not only co-ordinate this teaching but are responsible for teaching within their own small groups within a local area. There are typically 7–8 meetings per group per year and around 1400 CME-SGL meetings annually across the country. First established in 1983, the tutor network is funded by the Health Service Executive (HSE), Ireland’s national health authority, and is governed by the Irish College of General Practitioners (ICGP), which also administers the professional competence scheme for GPs.

There is a large body of literature which examines the effectiveness of CME for physicians and for their patients [2–4]. The effectiveness of any medical education intervention is influenced by the format through which it is delivered. Structured small group work has been shown to contribute to an individual’s increase in and ongoing use of knowledge in Ireland [5]. CME-SGL provides opportunities for doctors to meet with colleagues and have time away from practice obligations, incorporating personal, social, and professional experiences into the learning process [6,7]. The core skills applied in SGL are questioning, listening, responding and explaining [8]. These skills form a platform to facilitate reflection and discussion; however, the success of this approach is dependent in part upon the roles and responsibilities taken by students (participants) and tutors (including group leaders) and the consequent group dynamic.

A recent paper assessed, analysed, and compared CME programs for general practice in the United Kingdom, Norway, the Netherlands, Belgium, Germany, Switzerland, and France and concluded that future CME programs for GPs should develop curriculum objectives, promote innovative comprehensive learning platforms, establish clear rules for sponsorship, develop new financing models, and scientifically evaluate CME training [9]. The authors noted the importance of cooperation between primary care providers and health authorities during the Corona pandemic, and that the forecasts of an increase in pandemics emphasise the importance of appropriate training content in CME.

In line with changes to medical education that occurred worldwide during the COVID-pandemic, the mode of CME-SGL in Ireland changed from face-to-face meetings to online interaction. Such changes have been found to significantly affect medical education as well as clinical training; moreover, the COVID pandemic altered the way people interacted with each other [10]. This study sought to determine the benefits and limitations, as reported by GPs, of the transfer of Irish CME-SGL to online meetings.

Methods

This study was carried out from March 2021 over a 6-month period following ethical approval obtained from the ICGP in 2020, and involved the application of the conventional Delphi survey technique in which a series of questionnaires are sent to a target cohort (experts) over a set period with the aim of achieving an expert consensus [11,12]. Three sequential surveys were sent in electronic form to a national sample of GPs attending CME-SGL in the Republic of Ireland. The Delphi method was considered an optimal means of obtaining consensus on the target group as the survey could be carried out online and in an anonymous manner during the time of COVID restrictions. The iterative approach ensured participants could reflect on group consensus over the three rounds of the survey.

An initial email was sent by the principal investigator to all 40 Irish CME tutors nationally explaining the nature and purpose of the study and asking for help to recruit volunteers from their CME groups (comprising just over 2800 CME-SGL attendees across Ireland). All CME tutors replied by email to the principal investigator agreeing to email their group members directly, thus ensuring a national sample. After this, tutors had no further involvement; in particular, they were not made aware of who volunteered to participate, or what their responses were regarding online learning. CME tutors could therefore not influence participation or responses in any way.

Participants volunteered to be involved in the study by emailing the principal researcher and by giving their signed consent. As all those who agreed to participate were recruited, no randomization was applied. Participants were advised that they could opt out at any stage of the study and that all responses were anonymous. They were then sent questionnaires (3 rounds) via Survey Monkey. With each survey round, a single reminder email was sent to all participants 2 weeks after the original email. Apart from these emails, there was no direct contact between the principal investigator and study participants.

The first round of the survey collected descriptive information on the expert group including age, gender, number of years in practice, and location of GP practice (rural or urban). In addition, the first questionnaire asked two open-ended questions to be answered in free text: What were the benefits (if any) of CME-SGL being transferred online during the COVID-19 pandemic?

What were the limitations (if any) of CME-SGL being transferred online during the COVID-19 pandemic?

Responses received from the first round were reviewed by the research team. Where two or more GPs gave similar responses to a question, these were collated into a single statement. All statements, without exclusion, were then divided into several key themes (Table 2). Subsequently, in the round two questionnaire, GPs were asked to indicate on a Likert scale their level of agreement with each of the statements as ‘strongly agreed’ (1), ‘agreed’ (2), ‘neither agreed nor disagreed’ (3), ‘disagreed’ (4), or ‘strongly disagreed’ (5).

The results of the second round of the Delphi process were reviewed by the research team and those statements with which more than 60% of GPs had either strongly agreed or agreed were considered to represent the consensus view and thus the study outcome (Table 2). Statements which received less than 40% agreement were excluded from the final results.

The second-round responses had indicated satisfaction with certain aspects of the move to online teaching for CME-SGL, and dissatisfaction with other aspects of this educational format. Accordingly, in the third round of the Delphi survey the research team included just two specific questions, focusing on whether a permanent switch to online teaching might be desirable, and on informal aspects of face-to-face meetings. In this round, GPs could also provide free text comments.

Results

Eighty-eight GPs from 10 different tutor groups consented to participate in the study. The response rate was 72.7% in round one, 62.5% in round two and 64% in round three. Participant demographics and practice settings (Table 1) were consistent with those previously published for GPs attending CME-SGL [5].Table 1. Demographics and experience of GPs (N = 88) in the Delphi study group.

 	Participating GPs
N [%]	
Response rate round one	64 [72.7%]	
Male GPs	25 [39%]	
Female GPs	63 [61%]	
Practice setting	Rural 14 [22%]	
Mixed 12 [19%]	
Urban 38 [59%]	
Years working in GP	<5 years: 5 [7.8%]	
5-14 years: 14 [21.8%]	
15-29 years: 32 [50%]	
>29 years: 13 [20.3%]	
Regular attender of CME (≥ 4 meetings per annum)	63 [98.4%]	
Response rate round two	55 [62.5%]	
Response rate round three	56 [63.6%]	

Collation and review of round one responses generated statements comprising five main themes, including educational benefits of online CME-SGL, helping GPs to cope at a time of significant change in practice, importance of prior knowledge of the group for online learning, challenges associated with online learning and convenience of online learning (Table 2).Table 2. Level of agreement with statements resulting from the Delphi survey round one [N = 55].

Statement	(Agree/
Strongly agree)	
 	N [%]	
Theme 1: Educational benefits of online CME-SGL	 	
Online CME-SGL helped us to keep up to date with new evidence or guidelines regarding the management of COVID illness in GP as well as the practical application of these guidelines into clinical practice.	51 [92.7%]	
Online CME-SGL facilitated exchange of knowledge among GPs with respect to different local services that were available for our patients during COVID.	49 [89.1%]	
Online CME-SGL helped us to keep up to date with new evidence or guidelines regarding the management of non-COVID illness in GP as well as the practical application of these guidelines into clinical practice.	50 [90.9%]	
Theme 2: Helping GPs to cope at time of significant change in practice	 	
Online CME-SGL facilitated interaction with colleagues for advice and reassurance during a time when GPs were practicing medicine in a completely different way.	52 [94%]	
Online CME-SGL facilitated sharing of knowledge regarding adapting to the changes we needed to make in our practices because of COVID (e.g., appointment systems, telephone consultations, prescribing systems, etc.).	52 [94%]	
Online CME-SGL was very good for problem solving during this period of adjustment and uncertainty.	45 [81.8%]	
It was great to have contact with peers during times of constantly changing information on COVID and vaccines.	53 [96.4%]	
It was very supportive knowing we weren’t alone; participation in the group helped us to cope.	54 [98%]	
Theme 3: Importance of prior knowledge of the group for online learning	 	
In other Zoom meetings where I don’t know the other participants well, talking and sharing ideas can sometimes be awkward.	48 [87.3%]	
There was comradery in sharing common experiences online with familiar faces where trust had been established prior to learning online.	51 [92.8%]	
The CME-SGL group is established (i.e., I know the other participants well), so I feel comfortable talking and sharing ideas over Zoom.	42 [76.3%]	
Theme 4: Challenges associated with online learning	 	
A key benefit of attending CME-SGL is finding out what other GPs do in particular situations, and it is harder for people to share these anecdotes online.	37 [67.2%]	
It is much harder for discussions on certain topics to develop during online CME-SGL, as you cannot see the whole group or react to their social cues (body language, facial cues).	33 [60%]	
Theme 5: Convenience of online learning	 	
It is very convenient to attend online CME-SGL from home (e.g., less travel, easier childcare).	50 [90%]	
It was great being able to log in from home after a long tiring day during COVID.	45 [82%]	

Consensus was obtained on 15 statements in round two (Table 2); two statements on lack of enjoyment for and difficulty with concentration during online CME were rejected (less than 40% agreement). These GPs agreed that online CME-SGL facilitated interaction with their colleagues which they found supportive at a time of uncertainty and constant change. The online sharing of knowledge on new clinical problems, treatment guidelines and services helped them to cope during then pandemic. They agreed that prior knowledge of the group created trust and a level of comfort, but that certain topics were more difficult to discuss in an online setting. The convenience of attending online CME-SGL from home was generally agreed.

The round three consensus feedback (level of agreement) from the group on two key statements, one about attitudes to a permanent switch to online CME-SGL, and another on specific reservations GPs had raised with respect to the third and fourth main themes identified in round one, are presented in Table 3, along with associated comments from the participants. The majority of GPs did not favour a permanent switch to online CME-SGL and emphasised the importance of direct contact for social clues, body language and full engagement of all participants. These GPs also felt that online meetings result in a loss of informal spontaneous educational interactions that often happen in a face-to-face setting.Table 3. GP responses to two key statements in round three of the Delphi survey.

Statement	Disagree/Strongly disagree	
I would like all meetings to be online in the future (i.e., after COVID).	89.29% [50/56]	
Comments made in round three	
There are so many missing cues online instead of face to face	
Zoom has been necessary, but it is remote contact. Personal contact has far more depth at every level.	
People get distracted during zoom. They look at their mobile phones, they play with their iPad.	
Zoom CME was great when we needed it but it’s time to go back to face-to-face.	
It can never be as good as a face-to-face meeting.	
It is much more difficult to pick up on social cues online	
Body language is a huge part of normal discussion. Subtle body language is absent. You cannot see the whole group, and quieter members are less likely to participate.	
Some people don’t’ speak at all during Zoom CME. I’ve had enough! PLEASE -back to face to face….	
Sometimes it was hard to see all the faces at the same time and on several occasions, there were a few people who did not turn on their camera/fell asleep during the meeting and did not engage. This was quite disconcerting. Also, the natural flow of conversation that occurs at the real-life group was not apparent	
Statement	Agree/Strongly agree	
The opportunity for easy informal discussions about both clinical and non-clinical matters is lost because the Zoom format intrudes on the usual spontaneity of discussion.	70% (39/56)	
Comments made in round three	
Much of the meaningful contact within a meeting setting takes place either before or after the formal meeting. This is lost in Zoom.	
Again, more difficult to have spontaneous interactive discussion. In traditional small groups topics could come up that were not always directly related to the proposed topic on the night which often provided a learning opportunity. However, I found that with zoom people stick directly with topic and meetings are completed quicker with much less diversion into other areas of interest.	
There was a tendency for everyone to want to get off zoom after the formal clinical meeting was ending, we did have some discussion of cases or more social stuff, but normally in F2F meetings we would have a lot of time and space, and enthusiasm for this. on zoom, people just want to log off	

Discussion

Eighty-eight GPs from 10 different geographical areas in Ireland agreed to participate in this national Delphi study. Response rates were high in all three rounds. Demographics and practice setting were consistent with those previously published for GPs attending CME in Ireland; most were experienced practitioners who were regular participants in SGL. These GPs reported with high levels of consensus (≥89%) that online attendance at their already established CME-SGL groups during the pandemic allowed them to discuss the practical implications of applying rapidly changing guidelines on COVID care into clinical practice, to share knowledge about new local services, and compare their practice (including adaptations made due to the pandemic) with others. There was high consensus regarding the benefits of interaction with an established group of peers that is both familiar and trusted. Participation in online CME-SGL allowed these GPs to feel both reassured and less isolated, helping them to cope at a challenging time for general practice. Despite this very positive feedback, the majority (89% consensus) reported that they would not like all CME-SGL meetings to continue online after the pandemic. Limitations reported for online SGL included that it was less social (60% consensus), and that the informal learning that occurs before and after meetings was lost (70% consensus).

Implementation of practice guidelines is a complex activity that requires attention to the task of clinicians, the constraints they face, and the social practice of medicine. A scoping review by Fischer found that the central elements of successful strategies for guideline implementation include dissemination, education and training, and social interaction. Furthermore, the available evidence indicates that structured implementation can improve adherence to guidelines [13,14]. Irish CME-SGL provides a structure that incorporates both education and social interaction, and so is an ideal forum for both implementation of guidelines, and adherence to them. A qualitative study conducted in 2018 among GPs (n = 43) in Ireland 6 months after delivery of an educational module through CME-SGL found that care was delivered more consistently as a consequence of the teaching implemented. Study participants reported that through discussion with their colleagues during CME-SGL, they can find practical ways to apply guidelines in the care of their patients [7]. These authors concluded that participation in CME-SGL helps with the application of clinical guidelines into daily practice [15]. In keeping with the above, the current Delphi study found that at a time of significant and rapid change with respect to COVID and non-COVID care GPs were facilitated to implement new treatment guidelines through the discussions they had with their colleagues at online CME-SGL meetings.

Small group learning provides a safe and supportive environment where doctors can openly discuss the pressures and emotional challenges of work; this, in turn, provides GPs with the support they need while offering protection against compassion fatigue and burnout [7,16]. Access to CME has been identified as being of key importance as a support for doctors in their work and may help to reduce the stress levels of GPs [17–19]. Collegial support is a protective factor for good mental health and is associated with resilience and reduced sickness [20]. A systematic review on interventions to reduce burnout among doctors found that improving communication and giving participating doctors permission to acknowledge and manage stress have proven effective [21]. The GPs who participated in the Delphi study reported that the transfer of CME-SGL online allowed them to continue their learning in established groups with which they were familiar (i.e., ‘I know the other participants well’), and in this context, they were comfortable sharing their ideas and concerns over online platforms. Colleagues provided each other with advice and reassurance. There was a sense of comradery, and GPs felt less isolated. This, in turn, helped them to cope during a time of enormous stress and change.

Adapting face-to-face SGL to online platforms required flexibility from both learners and tutors, and the rapid implementation of such technology into CME has shown the strengths of collaborative skills and adaptability in medical academic settings. A variety of online meeting tools are being used to facilitate online medical SGL [22]. Obvious benefits of online learning include that members can attend in real-time from any location, that synchronous sessions are possible and that multiple users can participate in any one session [21]. GPs in this study reported that a particular convenience of online learning was that they did not have to travel and could attend meetings from the comfort of their own home. However, technological difficulties, such as poor broadband connections, software incompatibilities or hardware malfunctions, can be disastrous for the smooth running and effectiveness of online learning [21]. The quality and rate of interaction during an online SGL session depends on the ability of each individual to remain focused and engaged, thereby preventing the online atmosphere from becoming stilted or awkward [21].

The majority of GPs in this study reported that they would not like all their CME-SGL meetings to be delivered online after COVID (89% consensus). Factors contributing to this outcome might be that GPs reported that the informal learning that often occurs before and after face-to-face meetings no longer occurred in online meetings that online meetings were ‘less social’ and had less spontaneity, and that interactions with less well-known colleagues were more awkward in this setting. The pressures facing both urban and rural GPs have risen due to escalating bureaucracy, increased patient demand, workforce shortages and a reduction in resources; moreover, despite being at the highest ever levels, they continue to rise [23–26]. It is argued that these pressures have contributed to low job satisfaction and low morale among staff, as well as stress and burnout [27]. Doctors, including GPs, are not invulnerable to the clinical and emotional demands of their work, and the impact of the emotional component of work in general practice on the personal well-being of GPs is well-known [25,28]. Access to informal and formal support is crucial in enabling GPs to do their job effectively and to stay well. The responses from the GPs who participated in this Delphi study suggest that CME-SGL is an important outlet which supported them through the period of sustained high pressure associated with the pandemic. The factors that influence whether an educational meeting is enjoyable and more sociable for GPs are multiple, and include both internal and external factors, along with group dynamics. Nevertheless, the impression given by these GPs is that they find face-to-face meetings more socially and emotionally satisfying than online interactions.

Educational interventions for doctors remain crucially important for improving the quality of health care. One of the strengths of SGL is the opportunity for doctors to become actively involved in the process of learning and tutors (who are themselves GPs) have an active role to facilitate this process in the small groups [29]. Peer-to-peer interaction is reported as one of the main factors that influence the practice of GPs, and which can directly influence GP performance [30]. The known benefits of SGL include the development of discussion skills and thinking, exploration of attitudes, and sharing and reflecting upon experiences. The impact of the COVID pandemic on medical education including CME is not yet known, and it is therefore crucial that changes made are recorded and studied [21,31]. The transfer of Irish CME-SGL to an online format as a consequence of the pandemic restrictions, the first time such a change has been made, afforded an opportunity for such a study.

With the COVID-19 pandemic, most continuing medical education activities became virtual (VCME). The authors conducted a scoping review to synthesize the advantages and disadvantages of VCME to establish the impact of this approach on inequities that physicians face along the intersections of gender, race, and location of practice. Salient advantages identified were convenience, favourable learning formats, collaboration opportunities, effectiveness at improving knowledge and clinical practices, and cost-effectiveness. Prominent disadvantages included technological barriers, poor design, cost, lack of sufficient technological skill, and time. Analysis of the studies showed that VCME was most common in the general/family practice specialty, in suburban settings, and held by countries in the Global North [32].

A recent German survey addressed the perception of participants in CME, in particular online CME, and reported on the impact of the pandemic as well as expectations for the future [33]. The authors found that the perception of users was that the CME system reacted adequately to meet their demand but that online courses are disadvantaged by the lack of direct personal interaction. In the case of large groups, face-to-face interaction during lectures is an important feedback tool for both lecturer and trainee without which valuable communication is lost. In addition, online education is associated with a risk of diminished discussions with peers and the psychological impact of isolation. These drawbacks favour the combination of traditional face-to-face learning with online formats in a blended learning approach.

Limitations of the study

The most important limitation of this study is that the sample was self-selected from the GPs who were sent the initial e-mail invitation to participate. Only 88 GPs consented to participate, of whom only around two-thirds responded in each round of the survey. The majority of these participants were well-established GPs who were longstanding members of SGL groups and regular attendees at CME meetings. It is not known whether a broader group, including younger doctors or those who attend CME-SGL infrequently may have reported differently, or whether the GPs who volunteered might have represented a group which was more (or less) engaged with the online format. Another potential limitation is that it is unclear to what extent the extreme pressures on GPs at the time of the COVID pandemic may have affected their perception of online versus traditional CME-SGL. The results of Delphi surveys are often presented on the basis of consensus judgments, yet depending on how consensus was defined, up to 40% of the experts may not agree with the outcome. The identity of these experts and the judgments they have made remains unclear, and there is a risk that valuable minority judgments will be neglected [34]. Finally, the study participants were not specifically asked whether they might support a ‘blended’ approach to CME-SGL, with some meetings held using the traditional face-to-face format, and others (e.g., in winter, or during school holidays) held online. Nevertheless, based on the comments made during round 3, it seems likely that most GPs would prefer the majority of CME-SGL meetings to continue in a face-to-face setting. A key strength of this study was that although only 88 GPs participated in the study, the sample was national, and the response rates were comparatively high. No participants withdrew and most participated in all three rounds during a very busy time for GPs in their practices.

Conclusions

Online CME was accelerated and promoted by the recent COVID pandemic. This Delphi survey found that Irish GPs in established CME-SGL groups benefited from online learning as they could discuss how to adapt to rapidly changing guidelines while feeling supported and less isolated. Nevertheless, these GPs reported that face-to-face meetings offer more opportunities for informal learning and are more socially and emotionally satisfying. A high proportion of this study group (who are longstanding attenders of CME meetings) felt that the opportunity for informal discussions about both clinical and non-clinical matters is lost during online CME-SGL because the format intrudes on the usual spontaneity of discussion. Relatively little is known about this aspect of small group learning, and to what extent such informal discussions may contribute to the success of these educational groups. Future work should try to capture what ‘other learning’ occurs before and after face-to-face CME-SGL meetings in order to better understand exactly what is lost during the online component of ongoing learning for doctors.

Acknowledgments

The authors would like to acknowledge the generosity of the GPs in participating in this study during COVID.

Disclosure statement

No potential conflict of interest was reported by the author(s).
==== Refs
References

[1] Davis D. Does CME work? An analysis of the effect of educational activities on physician performance or health care outcomes. Int J Psychiatry Med. 1998;28 (1 ):21–7.9617647
[2] Robertson MK, Umble KE, Cervero RM. Impact studies in continuing education for health professions: update. J Contin Educ Health Prof. 2003;23 (3 ):146–156.14528785
[3] Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21 (3 ):380–385.16110718
[4] Cervero RM, Gaines JK. The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Continuing Educ Health Professions. 2015;35 (2 ):131–138.
[5] Dowling S, Last J, Finnegan H, et al. Does locally delivered small group continuing medical education (CME) meet the learning needs of rural general practitioners? Educ Primary Care. 2019;May 30 (3 ):1–7.
[6] Pereles L, Lockyer J, Fidler H. Permanent small groups: group dynamics, learning, and change. J Continuing Educ Health Professions. 2002;22 (4 ):205–213.
[7] Dowling S, Last J, Finnegan H, et al. Cullen W.Impact of participation in continuing medical education small group learning (CME-SGL) on the stress, morale, and professional isolation of rurally-based GPs: a qualitative study in Ireland. BJGP Open. [2019 Oct 29];3 (4 ):bjgpopen19X101673.
[8] Edmunds S, Brown G. Effective small group learning. Med Teach. 2010;32 (9 ):715–26.20795801
[9] Löffler C, Altiner A, Blumenthal S, et al. Challenges and opportunities for general practice specific CME in Europe – a narrative review of seven countries. BMC Med Educ. 2022;22 :761. doi: 10.1186/s12909-022-03832-7 36344994
[10] Hameed T, Husain M, Jain SK, et al. Maedica (Bucur). Online Med Teach COVID-19 Era Exper Percep Undergrad Students. 2020 Dec;15 (4 ):440–444.
[11] de Villiers MR, de Villiers PJ, Kent AP. The delphi technique in health sciences education research. Med Teach. 2005 Nov;27 (7 ):639–643.16332558
[12] Berger-Estilita J, Nabecker S, Greif R. A delphi consensus study for teaching “basic trauma management” to third-year medical students. Trauma Resusc Emerg Med. [2019 Oct 17];27 (1 ):91.
[13] Weingart SN. Implementing practice guidelines: easier said than done. Isr J Health Policy Res. 2014;3 :20.24967070
[14] Fischer F, Lange K, Klose K, et al. Barriers and strategies in guideline implementation-A scoping review. Healthcare (Basel). 2016;4 (3 ):36.27417624
[15] Dowling S, Last J, Finnegan H, et al. How does small group continuing medical education (CME) impact on practice for rural GPs and their patients, a mixed-methods study. Educ Prim Care [2020 May 3];31 (3 ):153–161.32089106
[16] Benson J, Magraith K. Compassion fatigue and burnout: the role of Balint groups. Aust Fam Physician. 2005;34 (6 ):497–498.15931410
[17] Branthwaite A, Ross A. Satisfaction and job stress in general practice. Fam Pract. 1988;5 (2 ):83–93.3391360
[18] Jackson D, Jackson DJ. The Western Australian centre for remote and rural medicine. Med J Aust. 1991;155 (3 ):144–146. doi: 10.5694/j.1326-5377.1991.tb142177.x 1875806
[19] Curran V, Rourke L, Snow P. A framework for enhancing continuing medical education for rural physicians: a summary of the literature. Med Teach. 2010;32 (11 ):e501–e8.21039092
[20] Firth-Cozens J. How to survive in medicine: personally and professionally. Books B, editor. Oxford,Unitied Kingdom: Wiley-Blackwell; 2010.
[21] Panagioti M, Panagopoulou E, Bower P, et al. Controlled interventions to reduce burnout in physicians: a systematic review and meta-analysis controlled interventions to reduce burnout in PhysiciansControlled interventions to reduce burnout in physicians. JAMA Intern Med. 2017;177 (2 ):195–.27918798
[22] Al Samaraee A. The impact of the COVID-19 pandemic on medical education Br. J Hosp Med (Lond). [2020 Jul 2];81 (7 ):1–4.
[23] Tc OM, O’Kelly F, Ni Shuilleabhain A, et al. Structure of general practice in Ireland 1982-2015. 2015:1–43.
[24] Doran N, Fox F, Rodham K, et al. Lost to the NHS: a mixed methods study of why GPs leave practice early in England. Br J Gen Pract. 2016;66 (643 ):e128–35.26740606
[25] Rimmer A. Doctors call for cap on GP workload. BMJ. 2018;361 :k2810.205.
[26] Riley R, Spiers J, Buszewicz M, et al. What are the sources of stress and distress for general practitioners working in England? A qualitative study. BMJ Open. 2018;8 (1 ):e017361.
[27] Dewa CS, Loong D, Bonato S, et al. How does burnout affect physician productivity? A systematic literature review. BMC Health Serv Res. 2014;14 (1 ):325.25066375
[28] Brooks SK, Gerada C, Chalder T. Review of literature on the mental health of doctors: are specialist services needed? J Ment Health. 2011;20 (2 ):146–156.21275504
[29] Steinert Y. Twelve tips for effective small-group teaching in the health professions. Med Teach. 1996;18 (3 ):203–207.
[30] Geneau R, Lehoux P, Pineault R, et al. Understanding the work of general practitioners: a social science perspective on the context of medical decision making in primary care. BMC Fam Pract. 2008;9 (1 ):1.18173835
[31] Ferrel MN, Ryan JJ. The impact of COVID-19 on medical education. Cureus. [2020 Mar 31];12 (3 ):e7492.32368424
[32] Cheng C, Papadakos J, Umakanthan B, et al. On the advantages and disadvantages of virtual continuing medical education: a scoping review. Can Med Educ J. [2023 Jun 27];14 (3 ):41–74. doi: 10.36834/cmej.75681 PMID: 37465741; PMCID: PMC10351643.
[33] Schulte TL, Gröning T, Ramsauer B, et al. Impact of COVID-19 on continuing medical education-results of an online survey among users of a non-profit multi-specialty live online education platform. Front Med (Lausanne). [2021 Nov 15];8 :773806. doi: 10.3389/fmed.2021.773806 PMID: 34869493; PMCID: PMC8634132.34869493
[34] Niederberger M, SprangerJulia Spranger J. Delphi technique in health sciences: a map sec. Public Health Educ Promot. 2020;8 . doi: 10.3389/fpubh.2020.00457
