
==== Front
Can Med Educ J
Can Med Educ J
CMEJ
Canadian Medical Education Journal
1923-1202
Canadian Medical Education Journal

CMEJ-15-005
10.36834/cmej.76192
Original Research
A phenomenological study of resident and faculty experiences with learner engagement in the normalization of workplace-based assessment
Une étude phénoménologique des expériences des résidents et des professeurs en matière d'engagement des apprenants dans la normalisation de l'évaluation sur le lieu de travailMc Donald Melissa 1
Muir Fiona 2
1 Institute of Health Science Education, Georgetown Public Hospital Corporation, Georgetown, Guyana;
2 School of Medicine, University of Dundee, Nethergate, UK
Correspondence to: Melissa Mc Donald; email: melissafiona394@gmail.com
09 4 2024
30 8 2024
8 2024
15 4 514
© 2024 Mc Donald, Muir; licensee Synergies Partners.
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an Open Journal Systems article distributed under the terms of the Creative Commons Attribution License. (https://creativecommons.org/licenses/by-nc-nd/4.0) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is cited.
Background

Workplace-based assessments (WPBA) have become integral to learner-centred medical education. As previous research has linked learner engagement to WPBA implementation, this study explores residents’ and faculty members’ experiences with learner engagement in the normalisation of WPBA practice.

Methods

Transcendental phenomenology was used as the qualitative approach, focusing on the participants’ lived experiences. A semi-structured interview guide was used to interview five faculty members and five residents who had conducted WPBA. The interviews were transcribed and analysed using phenomenological data analysis.

Results

Three themes were identified between learner engagement and WPBA conduct: (a) work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. WPBA learner engagement occurred when participants interacted with each other and with the clinical setting to facilitate teaching and learning. Both participant groups reported a desire to participate in WPBA, but time constraints at times hindered participation. The residents indicated that WPBA improved their knowledge and admitted to experiencing negative emotions during the assessment. Overall, participants recognised the reciprocal benefits of WPBA participation for their professional development.

Conclusion

The findings of the study suggest that learner engagement influences the use of WPBA. Consequently, it may be beneficial to consider the role of learner engagement to normalise WPBA application for teaching and learning in the clinical context.

Contexte

Les évaluations en milieu de travail (EMT) font désormais partie intégrante de l'enseignement médical centré sur l'apprenant. Des recherches antérieures ayant établi un lien entre l'engagement de l'apprenant et la mise en œuvre de l’EMT, cette étude explore les expériences des résidents et des membres du corps professoral en matière d'engagement de l'apprenant dans la normalisation de la pratique de l'EMT.

Méthodes

La phénoménologie transcendantale a été utilisée comme approche qualitative, en portant une attention particulière aux expériences vécues des participants. Un guide d’entrevue semi-structurée a été utilisé pour interroger cinq membres du corps professoral et cinq résidents qui avaient pratiqué l’EMT. Les entrevues ont été transcrites et analysées à l'aide d'une analyse phénoménologique des données.

Résultats

Trois thèmes ont été identifiés entre l'engagement de l'apprenant et la conduite des EMT : (a) l'environnement de travail, (b) les rôles et les relations, et (c) l'enseignement et l'apprentissage mutuellement bénéfiques. L'engagement de l'apprenant dans le cadre de l’EMT s'est produit lorsque les participants ont interagi entre eux et avec le milieu clinique pour faciliter l'enseignement et l'apprentissage. Les deux groupes de participants ont fait part de leur désir de participer à l’EMT, mais des contraintes de temps ont parfois entravé leur participation. Les résidents ont indiqué que l’EMT avait amélioré leurs connaissances et ont admis avoir ressenti des émotions négatives lors de l'évaluation. Dans l'ensemble, les participants ont reconnu les avantages réciproques de la participation à l’EMT pour leur développement professionnel.

Conclusion

Les résultats de l'étude suggèrent que l'engagement de l'apprenant influence l'utilisation de l’EMT. Par conséquent, il peut être bénéfique de considérer le rôle de l'engagement de l'apprenant pour normaliser l'application de l’EMT pour l'enseignement et l'apprentissage dans le contexte clinique.

Funding: There was no funding given for this research.
==== Body
pmcIntroduction

Medical education in the 21st century is shifting from subject-oriented to competency-based curricula.1,2 Workplace-based assessments (WPBAs) are a type of competency-based assessment that are aligned to curriculum outcomes in order to identify gaps and strengths in a learner's knowledge and skill set.3 WPBAs are used as formative assessments for residents in the clinical environment. WPBA involves learners being assessed in the clinical setting by a faculty member who provides verbal feedback with written feedback. The clinical environment can be challenging for delivering quality education to learners while balancing patient care demands.4 As a result, incorporating teaching activities like WPBA into daily work routines necessitates the committed participation of trainers and trainees in their implementation and delivery. Several studies have shown that learner’s perceptions of the value of WPBA and desire to participate in WPBA are connected to the degree of learner engagement in the educational setting.4-6 Learner or student engagement is defined as the learner’s commitment to academic success and learning demonstrated by their interest, participation, and dedication to graduate medical training.7 An engaged learner is more inclined to participate in and benefit from learning opportunities like WPBA. In addition, the extent of learner engagement is determined by the learner’s interest in their education, the educational environment, and the faculty’s dedication to teaching.7

Learner engagement can take many forms in an educational setting, such as academic, cognitive, behavioural, or sociocultural engagement.8 Academic learner engagement is defined as the time commitment to and participation in educational activities.7 Cognitive engagement references the faculty and residents having a shared purpose and dedication to learning demonstrated by the degree to which they act on their motivation using available resources.7-8 Cognitively engaged learners self-regulate their learning by reflecting, assessing their progress, planning, and achieving goals. Participation in discussions, asking questions, and paying attention throughout the assessment are all examples of behavioural engagement.9 Sociocultural engagement extends beyond the trainer’s or learner’s role. There is a focus on learner factors such as, past experiences, learning objectives and successes.8 Affective learner engagement alludes to both positive and negative emotional experiences of learners in an educational environment. If trainees associate learning activities with negative emotions and tension, this can undermine the efficacy of the WPBA.

Learner engagement with WPBA is influenced by the “harsh reality of assessment structure,” “individual perspectives,” the “learning environment,” and the “credibility” of assessments.10 However, trainees' perceptions of the value of the assessment can be linked to a lack of trainer-trainee engagement and attitudes towards the assessment.11 The failure of trainers and trainees to acknowledge and fulfil their respective responsibilities regarding assessments, interpersonal relationships, and shared assessment objectives impacts participation in WPBA.11-12 Additionally, learner engagement occurs within and is affected by the educational environment.9 When WPBA is contextual, teaching and learning are linked to clinical activity, which is advantageous for trainees and trainers.13

The impact of work-related responsibilities on training in the clinical context does not appear to be independent of trainer-trainee interactions. Work activities often disrupt assessments, and unfamiliarity with assessment methods affects participation.16 Trainees who have a clear understanding of WPBA have positive perceptions of the value of learning. However, prospective residents view WPBA negatively due to similar concerns expressed by current residents, such as time availability, trainer engagement, and a negative perception of the assessment's utility as a means of “ticking off training requirements.” 17 In addition, when, trainers and trainees are not clear on their responsibilities, trainers may overlook opportunities for engagement, and trainees may not seek out assessments.18 Thus, trainee-trainer disengagement with WPBA influences the quality of learning, its use, and the perceived value of the feedback.15

Purpose

While implementing WPBA is the first step towards its use as a teaching activity, how the assessment is normalised into daily teaching merits more research. The Normalisation Process Theory (NPT) examines how activities or practices are socially optimized, ingrained in daily life and maintained in social circumstances.21 NPT posits that a new practice becomes normalised not only through its implementation but through the effort expended towards its embedding and integration into existing social practices.21 Implementation emphasises the manner in which the WPBA are put into practice. Embedding is the process by which the WPBA is incorporated into residents’ and faculty’s routine teaching and assessment practices. Integration is the process of maintaining the use of WPBA practice over time. Current literature primarily investigates trainers’ and trainees’ perspectives on the applicability, value, outlook, and challenges connected with WPBA implementation.4,9,12,15,22 However, the role of learner engagement in normalising WPBA practice remains understudied. This research explores the link between learner engagement and WPBA used in the clinical setting as a form of education to answer two research questions:

RQ1: What are the experiences of internal medicine residents and faculty conducting workplace-based formative assessments?

RQ2: How do internal medicine residents and faculty experience learner engagement in the practice of workplace-based assessments?

Methods

The research was conducted at Georgetown Public Hospital Corporation (GPHC), Guyana's largest teaching hospital in the country's capital. The GPHC Internal Medicine Residency Programme began in 2013, and its curriculum goals and objectives are similar to those of the Canadian Internal Medicine Programmes.23 In 2018, a review of the programme’s curriculum revealed the necessity for regular performance assessment and personalised teaching opportunities for residents. As such, WPBAs (Mini-Clinical Encounter Examination [Mini-Cex] and Direct Observation of Procedural Skills [DOPs]) were implemented to assess residents’ clinical skills, judgement, professionalism, and medical management.24

Following the initial introduction of WPBA in 2019, assessments were infrequently utilised by residents and faculty. A review of the Mini-Cex and DOPs literature was conducted in an effort to find solutions for the assessments’ low utilisation. The literature identified implementation barriers such as residents’ and faculty’s difficulty balancing assessment responsibilities while managing a heavy workload and time constraints.10,12 Additionally, negative attitudes towards the educational utility of WPBA, and the quality and value of feedback to learners contributed to a lower acceptance of WPBA.25 Between September 2019 and January 2020, measures were implemented by the programme directors to address these issues, including reducing the number of weekly assessments from five to one to accommodate faculty workloads, providing faculty development, and clarifying faculty and learner responsibilities. Despite this, assessment uptake remained between zero and one monthly assessment per resident by March 2020.

Research design

This qualitative research was designed using Husserl’s transcendental phenomenology approach. Phenomena are our perceptions of things, or the appearance of truth, which is lived and encountered in the lifeworld, i.e., “the world of human experiences.”6-27(p.7) Phenomenology examines, analyses, and describes a phenomenon to understand social interactions in a specific environment.28 Since humans are in constant conscious interaction and relationship with the world, when we have an experience, we connect with it and assign meaning to it; this becomes our reality or truth of the experience. The phenomenological researcher investigates firsthand experiences of participants to capture the essence of the experience based on how the phenomenon occurred, appeared, and created meaning for the participants.27,29 For this research, the transcendental phenomenological approach was used to illuminate the residents and faculty’s lived experiences with WPBA and learner engagement. Exploring their experiences gives an in-depth and comprehensive understanding of the essence of learner engagement in WPBA from the first-person perspective. Ethical approval was granted by the Ministry of Public Health Guyana Ethical Review Board under reference #61/2020.

Participants

Purposive sampling was utilised to recruit residents (n = 12) and faculty (n = 7) of the Internal Medicine programme who had completed WPBA. Email invitations were sent by the principal researcher, along with a participant information sheet that outlined the research’s aim, how to volunteer to be a participant and the expectations of participants in this study. Residents and faculty were assured that participation was voluntary; declining to participate in the study would have no negative implications and they may withdraw from the study at any time before the study is completed.

Seven residents and five faculty agreed to participate. Various authors have suggested different sample sizes for phenomenology studies. There is general agreement that a suitable margin for achieving thematic saturation is between three and ten individuals.30-32 Interviews were conducted with all five faculty members and the first five residents who responded to the request to the point at which thematic saturation was achieved whereby no new themes were apparent from the data. Four residents were in their last year, two in their second, and one in their first. Two faculty members had more than ten years of experience, two between three and five, and one less than a year. Two participants were native Spanish speakers, English was their second language; therefore, the dialogue was translated into English when needed. All participants had done scheduled or unscheduled assessments.

Data collection

A semi-structured interview guide was created to elicit descriptions from participants of their experiences with WPBA. The interview guide contained questions designed to explore participants’ use of WPBA as a teaching and learning tool in the clinical environment, feedback, their roles WPBA and the literature on learner engagement indicators. The interview questions were piloted iteratively with either a resident or faculty member and revised as necessary until they provided adequate information.

The first author conducted semi-structured interviews with the participants in person or via the video conferencing software ZOOM (NASDAQ:ZM) between November and December 2020. Interviews were audio-recorded and transcribed verbatim.33 All participants signed consent forms and participant identification codes were used to ensure anonymity.34 Interviews were between 30-60 minutes long. The participants were encouraged to engage in open and honest discussions regarding their own experiences. They were also prompted to give in-depth answers to share their experiences.

Data analysis

Data analysis was conducted using the transcendental phenomenology approach as shown in Figure 1.35,26

Figure 1 Transcendental Phenomenological Data Analysis Steps

Each transcript was reviewed to identify words or phrases that revealed characteristics or expressed meanings of participants' experiences with WPBA, i.e., general units of meaning. Next, units of relevant meaning were identified by determining whether the units of general meaning (what the participant has said) answer the research questions. The units of relevant meanings were then reviewed to group units that shared a common theme, i.e. clusters of meanings. A theme, in this instance, is an essence that encompasses the ideas, elements, thoughts, beliefs, or appearance of multiple relevant units of meanings.37 At this stage, the transcript was referred iteratively to cross-check that the clusters of meaning matched the participant’s meaning.

Central themes were then identified by grouping two or more clusters of meaning that expressed a common meaning and gave rise to the essence of the experience.26 Individual interview summaries incorporating the central themes were developed to illuminate the essence of each participant's experiences and the emergence of the themes. Each participant received an email with their interview summary in which they were asked to respond to whether they agreed that the summary accurately represented their experiences or if they wanted to give further information in a follow-up interview, i.e., member checking.38 All participants agreed with the themes and interview summaries; no additional interviews were conducted.

The next step was to extract general and unique themes from the central themes that appeared in the interviews. Central themes that occur in most or all the interviews and can be grouped are general themes.26 Unique themes appear in only one or a small number of interviews. The participants’ central themes were separated and recorded in two categories, i.e., residents and faculty. These themes were then examined between the two groups to see which ones were shared by most participants and could be grouped. A summary of all the interviews incorporating general themes was created to capture the essence of all participants’ experiences and transform them into expressions suitable for scientific discourse.

Reflexivity

The principal researcher of this study serves as the Programme Director of the residency programme who supported the implementation of WPBA. This may have unintentionally influenced participant responses.34 However, insider researchers have essential elements for in-depth exploration of a phenomenon, such as contextual awareness and a personal connection with the participants.31 Participants were provided with a safe space to discuss their experience, freedom to express themselves, and reassurance the interview would remain anonymous. In order to prevent researcher bias, bracketing and reduction principles were applied so that the researcher’s personal perceptions were not incorporated into the data collection and analysis. Bracketing or epoché required the researcher to suspend their own opinions and immerse themselves in the data to remain close to the participants’ meaning.35 Reflective journaling was done throughout the research process to document the research process, decisions and personal thoughts to encourage conscious self-awareness and suspending of judgements

Phenomenological reduction occurs when the researcher knowingly and deliberately brackets their beliefs in order to analyse the experience. The act of reduction is only objective once there is the awareness that suppositions cannot be perfectly bracketed. A peer-to-peer debriefing of the analysis was performed with the second author to identify and eliminate researcher bias or omissions.39 This allowed for identification and restructuring of additional meanings and themes in the data analysis.

Results

The analysis of the data revealed that the themes identified from the participants interviews answered the research questions. As represented in Figure 2, the three general themes that were identified from the central themes, there were no unique themes. The three general themes derived from phenomenological data analysis are: (a) the work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. For each theme, interview excerpts from participants are presented under their identification code represented by the letters ‘R’ for resident, ‘F’ for faculty, and a number

Figure 2 General themes derived from participants’ interviews

Work environment

Work Environment and time. Residents and faculty agreed that having the assessments performed in a clinical environment made them more relevant and beneficial for residency training. They preferred scheduled assessments over spontaneous assessments to prepare for the teaching session. However, due to clinical responsibilities and workload, residents and faculty found setting aside time was a barrier resulting in the cancellation of scheduled assessments. This, according to residents, was a cause of demotivation for their willingness to do WPBA:

It’s in a very practical setting that you’re allowed … expected to demonstrate your knowledge…(R2)

…Sometimes difficult to have regular Mini-CEX as you wish because we have a lot of patients and staff changing … that is one of the disadvantages we don’t have the time to do more and do more of the observation and teaching. (F5)

Familiarity and benefits for teaching and learning with assessment. Residents stated that being assessed in a clinical setting was beneficial to their training. They valued the use of patients in the assessment with a faculty supervising and providing feedback on their clinical skills and knowledge. This made the learning more practical and prepared them to respond to future clinical situations:

…Able to improve their knowledge base while in a workplace setting that is comfortable to them as well as their supervisor… with very common day to day engagements around you will allow you that degree of comfort when a situation may arise. (R2)

…The bedside is better because we have the patient in front of us and we usually focus on the thing that the patient has. (F4)

Interactive student-centred learning in the clinical setting. Participants agreed that the assessments could be adjusted to residents’ learning goals or needs, making them more student-centered. Residents preferred to collaborate with faculty in choosing what should be assessed; however, some faculty members preferred to make this determination alone:

…I myself would want to prepare…to be more prepared to deliver this clinical assessment to the resident. I find that preparation that they do, I find that it benefits them. (F2)

…Resident can also give an input on the style of evaluation even though they might not know exactly what they will be evaluated on … keep a variation instead of every time you have to do Mini-Cex it the same type of assessment. (R3)

Roles and relations

Enforces roles and responsibilities. In discussing their responsibilities with WPBA, faculty and residents explained how the assessments drive both sides to work together in teaching. Residents and faculty reported that some faculty were willing to do WPBA when requested by residents and others needed constant reminders. Residents felt WPBA encouraged them to actively participate in their education by seeking out and completing assessments:

[Residents] are a little bit more responsible… more responsibility to get this done and to make some effort …more effort to get to be involved with our learning and not just things being dictated to us down from faculty. (R5)

The resident will only improve if seniors take a bit more interest and demonstrate to the residents what they need to know. (F3)

Influence of interactions on quality of WPBA. Residents and faculty perceived their interactions as pleasant and professional, albeit they acknowledged that participation in WPBA depends on both sides’ desire and commitment. Residents were willing to interact with faculty and ensure the completion of WPBA to get feedback. Several faculty initiated the WPBA, while others waited for residents to express interest. The faculty emphasised the educational value of one-on-one interactions with residents for teaching with WPBA:

When we have a very proactive resident in the team…take the initiative and want to learn. It raises the whole morale of the team immediately and you feel more willing to teach the residents. (F5)

I think it is the responsibility, of the faculty should at least ensure to not just do things for doing sake. (R5)

Emotions. Residents expressed fear, anxiety, and nervousness when assessed being by faculty. Faculty members acknowledged that emotional reactions to assessments could impair performance. One faculty member was worried about upsetting residents with unfavourable feedback. One resident said failing an assessment could negatively affect the faculty’s impression of them which contributed to the resident’s unfavourable feelings towards assessments. Despite their emotions, the residents still wanted to complete the assessments to get feedback on their performance:

When you start off as a faculty and you’re conducting these assessments, really and truly we tend to feel a bit nervous as well. (F1)

Another barrier is the residents become nervous being assessed by the faculty in front of them judging them. (R1)

Mutually beneficial teaching and learning

Student-centred teaching and learning. Before administering the assessments, residents and faculty agreed that they should include student-centered learning objectives agreed upon by both sides. However, the majority of the faculty admitted to developing the objectives without consulting with the residents. A faculty member advised that the programme standardise the evaluation objectives based on resident and faculty feedback, so that residents are assessed using the same standards:

…If we could have a single sheet or a set of sheets with five or ten basic objectives …. [to] go through in a standard way and have all the residents have that standard of the vision of the assessment and feedback. (F5)

…[Faculty] should at least share the same learning objectives… the residents should be aware of what they should know. (R5)

Mutual benefits to performance and skills. The participants believed WPBA encourages faculty and residents to ask and answer questions to enhance learning, thus benefitting faculty teaching and resident learning. All residents believed that assessments helped them apply their knowledge and understand their performance. The residents indicated that WPBA made faculty teaching more engaging. The faculty acknowledged that the assessments improved their teaching, and their learning through pre-assessment reading and interactions with residents:

It builds on our teaching skills as well as knowledge …a thorough foundation or memory or knowledge of a particular topic before I go through it with the residents. (F1)

For me as a resident, it improves my knowledge, clinical skills, patient management and organisation skills. (R1)

Feedback. Residents and faculty stated that the feedback from the assessments helped highlight areas where residents needed to improve. The residents typically preferred narrative feedback from the faculty since it provided insight into their knowledge and skill gaps. Some residents worried that verbal feedback did not always match written feedback on the assessment form, leading to confusion about their performance. One senior resident addressed the advantages of narrative feedback to describe your performance, noting that some junior residents paid more attention to numbers than words. The faculty believed that receiving feedback from residents on their teaching using WPBA can help faculty improve their teaching:

…You get your feedback; you have a more holistic evaluation of what you would have just done … they’ve helped me organise my studying better so that I could focus on the areas that I need to focus on.” (R3)

I just try to say listen, you are good at this and this and that, but I think that you need to improve this or I don’t like that. (F4)

Motivates study and tracks performance. After an assessment, residents were motivated to study the topic to improve their knowledge. Pre-reading for scheduled assessments was done by both residents and faculty. Further, both groups stated that WPBA is helpful to track resident performance over time for improvement using written feedback, verbal feedback, or noting a change in performance when re-assessed on the same medical pathology:

It encourages me… to have that continuous studying, like reading around my cases because you never know maybe a question could come from there. (R4)

This [formative assessments] serves as a pretty good ruler in very spaced-out increments to know how that student is progressing in a particular area that they’re trying to work on. (F2)

Discussion

This study highlights the role of learner engagement in the normalisation of WPBA within the clinical context. WPBA are practical assessments that necessitate a conducive clinical education environment to facilitate their use.40 Both the residents and faculty members indicated a willingness to participate in assessments and expressed the view that the clinical environment offers relevant opportunities for teaching and learning. However, its use was constrained by the time available for conducting assessments. The clinical environment should encourage academic learner engagement by balancing time for education, contact with the clinical setting, and increasing resident-faculty dedication to teaching and learning.7 Academic learner engagement strategies can address WPBA teaching time. The WPBA model can be examined with resident and faculty involvement to create an assessment format that can be used efficiently in clinical environment. Incorporating resident and faculty feedback gives the group the chance to shape the assessment to address the features that enable its meaningful use.

Residents in this study found that because of interaction with faculty during WPBA, their knowledge improved, and they were motivated to learn. In addition, WPBA motivates residents to engage in self-directed learning and reflection. However, there were misconceptions about who should initiate the participation in the WPBA. Faculty regarded assessments as learner-driven, while residents saw them as faculty-guided. This contributed to a lack of accountability that affected residents’ impressions of WPBA. Participation in teaching activities is an indicator of behavioural learner engagement.7 Consequently, strategies such as the programme outlining clearly defined roles and responsibilities for initiating and participating in WPBA can mitigate these issues.

Moreover, fostering a learning culture where residents and faculty actively participate in WPBA can be improved by using sociocultural learner engagement strategies. 44 Residents and faculty must work together to identify the actions and procedures they believe are essential for the practice to be adopted and sustained. For example, the chief resident could motivate residents by providing guidance on their role and expectations for accomplishing learning objectives with WPBA. A key faculty member may observe faculty teaching with WPBA to provide them with feedback. They can also regularly communicate and address faculty issues with WPBA. This helps build relationships between faculty to support teaching and learning.

Another key element discussed by residents and faculty was the negative emotions felt when completing assessments. These emotions contributed to apprehension about participating or providing unfavourable feedback. The residents’ fear of being observed and assessed affected the learning experience and shifted the focus onto being assessed for satisfactory performance. Prior experiences that involved negative emotions related to feedback made them reluctant to seek future assessments. Emotional wellbeing and positive relationships are indicators of affective learner engagement.7 Affective learner engagement can strengthen positive relationships between residents and faculty by promoting collaborative assessment planning with student-specific objectives. Faculty should encourage residents to participate in the WPBA for purposes of learning and solicit resident feedback on their teaching. Creating these channels of interaction and responsibilities for residents and faculty ensures collective commitment to sustaining WPBA use.

Constant appraisal to address factors that inhibit or promote normalizing WPBA will be needed.21,40 Group and individual appraisal using structured methods will address institutional patterns, individual experiences, and judgement of the value of the WPBA practice. Participants can be surveyed or interviewed regularly for feedback to improve on WPBA as needed. Promoting learner engagement and establishing a connection to the WPBA with residents and faculty is a long-term commitment in which programs must invest. Figure 3 provides example of how the normalization of WPBA can be achieved by employing learner engagement strategies within a framework informed by the NPT. However, the NPT framework and learner engagement strategies should be implemented using contextual factors of the education environment.

Figure 3 Learner engagement strategies for normalization of workplace based assessments in Normalistion Process Theory framework

Limitations

This study reflects the data collected using a single technique and analysed participants’ recollections of past events. Due to the study’s small sample size and purposive sampling, participants came from a single residency programme at a single centre. This may call into question the transferability of the study. However, the goal of this study is to gain knowledge of individual experiences and conduct relevant and beneficial research for the participants.37 During the interviews, little information was collected about how individuals overcome barriers such as work-time balance. This could provide greater insight into the processes employed to ensure Mini-Cex and DOPs were completed. Though steps were taken to mitigate this effect, the principal researcher’s being an insider researcher may have affected the recruitment and interview process.

Conclusion

This study demonstrates the role of learner engagement in the practice of WPBA. The residents in this study found WPBA helpful when they were engaged with the faculty and the work environment. Both residents and faculty offered positive impressions of how being involved encouraged them to perform the assessments. Despite their motivation, time, lack of clarity on roles and work-related tasks did hinder participation. Residents and faculty should be involved at all stages of the WPBA’s development, implementation and embedding. Collaboration with residents and faculty facilitates the normalisation of a WPBA as a routine educational activity. Prior to implementing a WPBA, time management, job functions, and learning balance must be optimised.11,44 This can be accomplished by examining different learner engagement strategies in the WPBA’s integration into residents’ and faculty members’ daily responsibilities. When residents and faculty are involved in an assessment for an extended period, both benefit from the teaching and learning process. The long-term use of WPBA is enhanced by soliciting regular input from residents and faculty to improve the assessment process.

Additional research on learner engagement tactics within the normalisation process theory framework to deploy WPBA or other formative assessments may be helpful. A follow-up study might explain why residents and faculty use WPBA once learner engagement techniques are implemented. Future research should also determine how participants perceive time management and work balance when educational activities are incorporated. Obtaining data from residents and faculty in different programmes and regions would enrich and may lead to generalizable findings. Learner engagement is key to normalising WPBA processes, as learner engagement is linked to WPBA’s social and experiential learning.

Conflicts of Interest

The authors have nothing to disclose

Edited by

Tim Dubé (section editor); Christina St Onge (senior section editor); Marcel D’Eon (editor-in-chief)
==== Refs
1 Cate OT. Competency-based postgraduate medical education: past, present and future. GMS J Med Educ. 2017;34 (5 ):Doc69. 10.3205/zma001146 29226237
2 Lockyer J, Carraccio C, Chan MK, et al . Core principles of assessment in competency-based medical education. Med Teach. 2017 Jun;39 (6 ):609–16. 10.1080/0142159X.2017.1315082 28598746
3 Earl L, Steven Katz, Campbell G, et al . Rethinking classroom assessment with purpose in mind. Crown in right of the Governments of Alberta. 2006;112 . Available from: https://www.edu.gov.mb.ca/k12/assess/wncp/full_doc.pdf [Accessed Sept 24, 2020].
4 Jenkins CE. The relationship between formative assessment and student engagement at Walters State Community College. ProQuest LLC. [Tennessee]: East Tennessee State University; 2010. Available from: https://dc.etsu.edu/cgi/viewcontent.cgi?article=3034&context=etd [Accessed Jun 9, 2022].
5 Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007 Jan;29 (9–10 ):855–71. 10.1080/01421590701775453 18158655
6 Stiggins R. From formative assessment to assessment for learning: a path to success in standards-based schools. Phi Delta Kappan. 2005 Dec;87 (4 ):324–8. 10.1177/003172170508700414
7 Reschly AL, Pohl AJ, Christenson SL, SpringerLink. Student engagement effective academic, behavioral, cognitive, and affective interventions at school. 1st ed. 2020. Cham: Springer International Publishing; 2020. xix+331. 10.1007/978-3-030-37285-9
8 Kahu ER. Framing student engagement in higher education. Studies Higher Educ. 2013 Jun;38 (5 ):758–73. 10.1080/03075079.2011.598505
9 Fredricks JA, Blumenfeld PC, Paris AH. School engagement: potential of the concept, state of the evidence. review of educational research. 2004 Mar 1;74 (1 ):59–109. 10.3102/00346543074001059
10 Dijksterhuis MGK, Schuwirth LWT, Braat DDM, Teunissen PW, Scheele F. A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach. 2013 Aug;35 (8 ):e1396–402. 10.3109/0142159X.2012.756576 23600668
11 Barrett A, Galvin R, Scherpbier AJJA, Teunissen PW, O’Shaughnessy A, Horgan M. Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and experiences. Postgrad Med J. 2017 Mar;93 (1097 ):138–42. 10.1136/postgradmedj-2015-133917 27486250
12 Lörwald AC, Lahner FM, Nouns ZM, et al . The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: a systematic review and meta-analysis. Zhang Q, editor. PLoS ONE. 2018 Jun 4;13 (6 ):1–15. https://dx.plos.org/10.1371/journal.pone.0198009
13 Spike N, Alexander H, Elliott S, et al . In-training assessment-its potential in enhancing clinical teaching. Med Educ. 2000 Oct;34 (10 ):858–61. http://doi.wiley.com/10.1046/j.1365-2923.2000.00755.x 11012936
14 Billett S. Toward a workplace pedagogy: guidance, participation, and engagement. Adult Educ Quart. 2002 Nov 1;53 (1 ):27–43. 10.1177/074171302237202
15 Prentice S, Benson J, Kirkpatrick E, Schuwirth L. Workplace-based assessments in postgraduate medical education: a hermeneutic review. Med Educ. 2020 Nov;54 (11 ):981–92. 10.1111/medu.14221 32403200
16 Andreassen P, Malling B. How are formative assessment methods used in the clinical setting? A qualitative study. Int J Med Educ. 2019 Nov 22;10 :208–15. 10.5116/ijme.5db3.62e3 31759332
17 Ali J, Goh A. Student perceptions of workplace-based assessment. Clin Teach. 2017;14 (5 ):319–24. 10.1111/tct.12588 27981755
18 Acai A, Li SA, Sherbino J, Chan TM. Attending emergency physicians’ perceptions of a programmatic workplace-based assessment system: the McMaster Modular Assessment Program (McMAP). Teach Learn Med. 2019 Aug 8;31 (4 ):434–44. 10.1080/10401334.2019.1574581 30835560
19 Harlen W, James M. Assessment and Learning: differences and relationships between formative and summative assessment. Assess Educ: Princ Policy Pract. 1997 Nov;4 (3 ):365–79. 10.1080/0969594970040304
20 Liang Y, Noble LM. Chinese doctors’ views on workplace-based assessment: trainee and supervisor perspectives of the mini-CEX. Med Educ Online. 2021;26 (1 ):1869393. 10.1080/10872981.2020.1869393 33380291
21 Jones A. Bridging the implementation gap: normalisation process theory in educational settings. 2017 Apr 15; Available from: https://www.researchgate.net/publication/316666078_Bridging_the_implementation_gap_normalization_process_theory_in_educational_settings
22 Massie J, Ali J. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv in Health Sci Educ. 2015 May 24;21 (2 ):455–73. 10.1007/s10459-015-9614-0
23 Persaud D, Cole J, Jainarine R, Khalid Z. Internal medicine residency program in Guyana: a collaborative model for sustainable graduate medical education in resource-limited settings. Front Public Health. 2017 May 29;5 :112. 10.3389/fpubh.2017.00112 28611974
24 Etheridge L, Boursicott K. Performance and workplace assessment. In: A practical guide for medical teachers. 4th ed. London; New York: Churchill Livingstone/Elsevier; 2013. p. 307–14. Available from: https://www.elsevier.com/books/a-practical-guide-for-medical-teachers/dent/978-0-7020-8170-5
25 McQueen SA, Petrisor B, Bhandari M, Fahim C, McKinnon V, Sonnadara RR. Examining the barriers to meaningful assessment and feedback in medical training. Amer J Surg. 2016 Feb;211 (2 ):464–75. 10.1016/j.amjsurg.2015.10.002 26679827
26 Husserl E. Ideas: general introduction to pure phenomenology. New York, NY: Routledge; 2012;434 p. Available from: https://www.routledge.com/Ideas-General-Introduction-to-Pure-Phenomenology/Husserl/p/book/9780415519038#:~:text=The%20publication%20of%20Ideas%20in,the%20essence%20of%20consciousness%22%20itself. [Accessed on Mar 22, 2023].
27 Vagle MD. Crafting phenomenological research. 2nd edition. New York: Routledge; 2018. 214 p. 10.4324/9781315173474
28 Streubert HS, Carpenter DR. Qualitative research in nursing: advancing the humanistic imperative. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2011.
29 Smith DW. Phenomenology. In: Zalta EN, editor. The Stanford encyclopedia of philosophy. 2018th ed. Stanford, California: Metaphysics Research Lab, Stanford University; 2018. Available from: https://plato.stanford.edu/archives/sum2018/entries/phenomenology/. [Accessed On Feb 18, 2021].
30 Englander M. The interview: data collection in descriptive phenomenological human scientific research. J Phenomenol Psychol. 2012;43 (1 ):13–35. 10.1163/156916212X632943
31 Peoples K. How to write a phenomenological dissertation: a step-by-step guide. 1 edition. Thousand Oaks, California: SAGE Publications, Inc; 2020. 192 p. Available from: https://us.sagepub.com/en-us/nam/how-to-write-a-phenomenological-dissertation/book261976
32 Varpio L, Ajjawi R, Monrouxe LV, O’Brien BC, Rees CE. Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking. Med Educ. 2017 Jan;51 (1 ):40–50. 10.1111/medu.13124 27981658
33 Zoom Video Communications, Inc. Zoom. San Jose, California; 2021. Available from: https://zoom.us/
34 Fleming J. Recognizing and resolving the challenges of being an insider researcher in work-integrated learning. International J Work-Integ Learn. 2018;19 (3 ):311–20. Available from: https://files.eric.ed.gov/fulltext/EJ1196753.pdf [Accessed on Mar 22, 2023].
35 Hycner RH. Some guidelines for the phenomenological analysis of interview data. Human Studies. 1985;8 :279–303. 10.1007/BF00142995
36 Ortlipp M. Keeping and using reflective journals in the qualitative research process. The Qualitative Report. 2008 Dec 1;13 (4 ):695–705. Available from: https://nsuworks.nova.edu/tqr/vol13/iss4/8
37 Javadi M, Zarea K. Understanding thematic analysis and its pitfall. J Clie Care. 2016;1 (1 ):34–40. 10.15412/J.JCC.02010107
38 Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ. 2008 Aug 7;337 :a1035. 10.1136/bmj.a1035 18687726
39 Cohen L, Manion L, Morrison K. Research methods in education. 8th edition. London ; New York: Routledge; 2017. 944 p. Available from: https://www.routledge.com/Research-Methods-in-Education/Cohen-Manion-Morrison/p/book/9781138209886
40 May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. SAGE Publications. 2009;43 (3 ):535–54. 10.1177/0038038509103208
41 McNaughton RJ, Steven A, Shucksmith J. Using normalization process theory as a practical tool across the life course of a qualitative research project. Qual Health Res. 2020 Jan 1;30 (2 ):217–27. 10.1177/1049732319863420 31347440
42 Wood P. Overcoming the problem of embedding change in educational organizations: a perspective from normalization process theory. Manag Educ. 2017 Jan;31 (1 ):33–8 10.1177/0892020616685286
43 Harris P, Bhanji F, Topps M, et al . Evolving concepts of assessment in a competency-based world. Med Teach. 2017 Jun;39 (6 ):603–8. 10.1080/0142159X.2017.1315071 28598736
44 Konopasek L, Norcini J, Krupat E. Focusing on the formative: building an assessment system aimed at student growth and development. Acad Med. 2016 Nov;91 (11 ):1492–7. 10.1097/ACM.0000000000001171 27028028
