
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39181560
10.1136/bmjopen-2024-086775
bmjopen-2024-086775
Protocol
Oncology
1717
1506
Mapping cognitive biases in multidisciplinary team (MDT) decision-making for cancer care in Scotland: a cognitive ethnography study protocol
https://twitter.com/harini_dl
http://orcid.org/0000-0002-5201-9753
Dharanikota Harini 1l.h.dharanikota@sms.ed.ac.uk

Wigmore Stephen J 12s.wigmore@ed.ac.uk

Skipworth Richard 12richard.skipworth@nhslothian.scot.nhs.uk

Yule Steven 12steven.yule@ed.ac.uk

1 Surgical Sabermetrics Laboratory, Centre for Medical Informatics, Usher Institute, The University of Edinburgh, Edinburgh, UK
2 Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, UK
Ms; l.h.dharanikota@sms.ed.ac.uk
SY reports research grants from the National Institutes for Health, Canadian Department of National Defence, National Aeronautics and Space Association, The Melville Trust for the Care and Cure of Cancer, Royal College of Surgeons of Edinburgh and Johnson & Johnson and is a recent past member of the Global Education Council at the Johnson & Johnson Institute, outside the submitted work.

2024
24 8 2024
14 8 e08677522 3 2024
22 7 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Introduction

The efficiency of multidisciplinary teams (MDTs) in cancer care hinges on facilitating clinicians’ cognitive processes as they navigate complex and uncertain judgements during treatment planning. When systems and workflows are not designed to adequately support human judgement and decision-making, even experts are prone to fallible reasoning due to cognitive biases. Incomplete integration of information or biased interpretations of patient data can lead to clinical errors and delays in the implementation of treatment recommendations. Though their impact is intuitively recognised, there is currently a paucity of empirical work on cognitive biases in MDT decision-making. Our study aims to explicate the impact of such biases on treatment planning and establish a foundation for targeted investigations and interventions to mitigate their negative effects.

Methods and analysis

This is a qualitative, observational study. We employ cognitive ethnography, informed by the Distributed Cognition for Teamwork framework to assess and evaluate MDT decision-making processes. The study involves in-person and virtual field observations of hepatopancreaticobiliary and upper gastrointestinal MDTs and interviews with their members over several months. The data generated will be analysed in a hybrid inductive/deductive fashion to develop a comprehensive map of potential cognitive biases in MDT decision processes identifying antecedents and risk factors of suboptimal treatment planning processes. Further, we will identify components of the MDT environment that can be redesigned to support decision-making via development of an MDT workspace evaluation tool.

Ethics and dissemination

This project has received management and ethical approvals from NHS Lothian Research and Development (2023/0245) and the University of Edinburgh Medical School ethical review committee (23-EMREC-049). Findings will be shared with participating MDTs and disseminated via a PhD thesis, international conference presentations and relevant scientific journals.

Adult surgery
Clinical Reasoning
Cognition
Clinical Decision-Making
Safety
The Melville Trust for the Care and Cure of Cancer R47363
==== Body
pmcSTRENGTHS AND LIMITATIONS OF THIS STUDY

Structured framework: The study uses the established Distributed Cognition for Teamwork framework to organise data collection and analysis of cognitive biases in clinical team decision-making.

Subject-matter experts: Multidisciplinary professionals from real multidisciplinary teams (MDTs) will be involved in data collection, enhancing ecological validity.

Mixed methods: Observations and interviews will be conducted iteratively and in collaboration with MDT members to ensure continuous verification and accuracy of data representation.

Tacit knowledge: Since cancer care and MDT functions involve a lot of implicit knowledge, participants might struggle to fully explain their decision-making processes in the interviews.

Generalisability: Data collection at a single centre means the results may be limited in generalisability to other healthcare systems and institutions.

Background

Multidisciplinary teams (MDTs) are an essential part of the cancer care pathway in the UK, bringing together specialists from multiple disciplines who review patient cases to make treatment recommendations. Knowledge, information and responsibilities are distributed between team members. In the UK, MDT discussion is mandatory for all patients with confirmed cancer diagnoses. Imaging, patient history and laboratory reports are reviewed by a team of specialists to make diagnostic and treatment recommendations, during regular meetings with allocated time slots in a hybrid format (involving a combination of in-person and virtual attendees). The success of an outcome relies on the effective and efficient co-ordination and synthesis of information. The use of heuristics is an essential component of efficient clinical decision-making.1 However, cognitive biases such as search satisficing, confirmation bias and anchoring bias,2 described as ‘decisional shortcuts’3 to making judgements, can be a double-edged sword. They are time and cognitive resource efficient but can be fallible and lead to errors and undesirable outcomes. Cognitive errors and biases have been identified and elaborated on in various areas of medical decision-making.47 Given the nature of MDTs (multiple professionals, time pressure, need for certainty, consequential cases),8 they are likely not immune to biases and fallacies that may impact the quality of treatment recommendations in cancer care.

Numerous experimental paradigms and quasi-experimental studies have evidenced the prevalence of biases in and outside laboratory settings.9 10 However, studying cognitive biases in complex naturalistic settings has proven to be a challenge due to several reasons, including the fact that the negatively consequential nature of such heuristics and biases is only evident in hindsight, after a judgement has been made and the outcome apparent. In messy real-world environments, it is difficult to definitively attribute decision outcomes to specific instances of biased reasoning. Further, context specificity plays a role in the occurrence and propagation of cognitive bias within MDTs, that is, the specific conditions under which MDTs’ function can impact reasoning and collaboration, which in turn impact treatment recommendation decisions. Considering this, we have selected relevant theoretical and methodological frameworks to guide the present study, described below.

Theoretical and methodological framework

According to distributed cognition (DCog) theory, cognitive activity is organised across different members of (1) teams (ie, social environment), (2) the tools and artefacts (ie, material environment) and (3) time (ie, flow and transformation of information).11 Cognition is thus not confined within an individual, but distributed across the entire decision environment. This may be particularly relevant in the MDT context to support the idea that workflows, tools, technology, and teams must be designed in ways that complement the capacity of human function, instead of attempting to alter human capacity to fit environmental constraints. The latter limits scope for improvement of services and task performance since individuals’ cognitive capacity is finite and fallible. Hence, we employ the DCog theory to characterise cognitive processes within the MDT meeting environment.

The Distributed Cognition for Teamwork (DiCoT)12 is a qualitative methodological framework to systematically map and identify the specific reasoning tasks and information use throughout the team decision-making process. DiCoT framework provides a set of themes and principles to guide the qualitative analysis of teamwork environments. The framework allows for a systematic understanding of the propagation and transformation of information in the work system to assess their requirements and devise technological solutions addressing problem areas. The original framework comprises 5 themes and offers 18 corresponding principles (table 1).

Table 1 Themes and principles outlined by the Distributed Cognition for Teamwork framework12

Theme	Principle	
Information flow	Principle 1: Space and cognitionPrinciple 2: Perceptual principlePrinciple 3: Naturalness principlePrinciple 4: Subtle bodily supportsPrinciple 5: Situation awarenessPrinciple 6: Horizon of observationPrinciple 7: Arrangement of equipment	
Physical layout	Principle 9: Information transformationPrinciple 10: Information hubsPrinciple 11: BufferingPrinciple 12: Communication bandwidthPrinciple 13: Informal communicationPrinciple 14: Behavioural trigger factors	
Tools and artefacts	Principle 15: Mediating artefactsPrinciple 16: Creating scaffoldingPrinciple 17: Representation–goal parityPrinciple 18: Co-ordination of resources	
Social structures*	–	
Evolutionary*	–	
* The themes ‘Social structures’ and ‘Evolutionary’ are recently identified, and to date have not to date formed principles for analysis, but are included here for completeness.

Mapping how information moves and transforms throughout the entire MDT system would be possible using the DiCoT analysis technique and has the potential to reveal novel detailed insights into the MDT decision-making process, but to our knowledge has not been attempted before. Potential insights include relations among team members and between team members and the tools they use (eg, from referral letters to individuals presenting the case and from the individual presenting the case to the rest of the team). Quantifying the MDT process means we could also identify ways to modify these components (eg, designing information displays, sequence of decision tasks, team communication protocols) in order to improve and debias the MDT decision process.

The use of the DiCoT framework in this environment also offers the opportunity to develop a DiCoT framework-based tool for the evaluation of MDT workspaces. This would be particularly valuable as it goes beyond the focused examination of team collaborative behaviours,13 14 decision quality15 or ICT (Information and communication technology) use16 to address how multiple factors intersect to impact the case reviewing processes and treatment recommendation outcomes.

DiCoT framework has previously been employed in a variety of healthcare settings including ambulance dispatch services, psychiatry services, operating rooms and emergency departments. However, unlike these environments which involve a range of observable technical activities that reflect cognitive activity, most of the work performed in MDTs is ‘knowledge work’17 which is relatively difficult to outwardly observe and interpret. Analysing team communication patterns, interactions with information, tools and technology can provide insight into the cognitive tasks that are accomplished via these observable behaviours.

For the present study, we draw on cognitive ethnography, which bridges the gap between traditional ethnography and cognitive science with ‘analysis-for-purpose’ goals focusing on verifiability, observational specificity and purposiveness.18 In practice, cognitive ethnography is suited to the study of cognitive processes such as problem-solving, decision-making and sensemaking (in the MDT case review process), with the goal of developing or validating cognitive models of behaviours (ie, cognitive biases in clinical reasoning) in specific naturalistic settings via interviews and observations. It is both inductive and deductive, in that it allows us to observe cognitive processes and behaviours in context while also allowing us to apply theoretical concepts to analyse and explain behaviour.19

The primary aims of the study are to (1) map the MDT decision-making process using the DiCoT framework and (2) identify antecedents of cognitive biases using DCog theory. The secondary aim is to develop a DiCoT framework-based tool for evaluating cancer MDT meeting workspaces. To achieve these aims, we propose protocols for non-participant observation and interview studies to gather and analyse primary data from a sample of real MDT meetings for cancer care in Scotland.

Applications to enhance clinical decision-making

The present study is intended to quantify cognitive biases in clinical decision-making. The findings may improve awareness of the different ways in which biases can impact MDTs and how the design of the MDT environment can facilitate or hinder effective teamworking. The decision-bias map this study aims to produce could serve several purposes including (1) informing future hypothesis-testing investigations of cognitive bias in MDTs, (2) mitigating bias in MDTs using technological debiasing strategies which involve modifying the environment, tools and teams and (3) specifying decision components for a focused assessment of automation and algorithmic support requirements for cancer MDTs. We foresee several applications of the DiCoT framework-based tool. One, to prompt MDT leaders to assess their own MDT functioning and quality improvement at the system level and promote the idea that successes and errors in clinical practice are a function of the interactions in the collective distributed cognitive system and not solely individual responsibility.20 Two, it can inform the introduction of objective and multimodal measurement techniques21 to quantify MDT proxemics, gestures, communication, cognitive load and interaction behaviours, in the pursuit of developing objective performance metrics. Three, it could be used by clinicians establishing a new MDT process in a surgical department or hospital system, to develop countermeasures against the risk of unintentional cognitive bias negatively impacting patient care. Our work will contribute to ongoing efforts to enhance MDT practices to improve treatment decision quality and patient safety in cancer care.

Methods

Study setting and duration

The study setting is the hepatopancreaticobiliary (HPB) and upper gastrointestinal (upper GI) cancer MDT meetings from a single academic teaching hospital in Scotland. The MDT meetings take place weekly in seminar rooms in the institution. The meetings are hybrid in format, involving a combination of in-person and virtual attendees joining via videoconferencing tools. Specialists outwith the hospital also attend virtually. The study is expected to last up to 11 months, from 8 February 2024 to 31 December 2024.

Participant sampling

All attending members of upper GI and HPB MDT meetings will be observed. Attending participants include radiologists, surgeons, oncologists, physicians, pathologists, cancer nurses, MDT co-ordinators and other ad hoc specialists (eg, dietician, respiratory therapist). For the interviews, participants will be purposively sampled based on their medical specialty or role in the meeting and cancer type. We aim to recruit at least one individual per specialty (eg, surgery, radiology, oncology) per cancer type, as well as MDT co-ordinators for each MDT to ensure a breadth of perspectives, totalling between 12 and 15 participants.

Procedure

Observations and interviews will be conducted concurrently, with both procedures iteratively refined as new concepts and information are uncovered. This is preceded by development of a cognitive bias catalogue. A visual summary of the study methods is presented in figure 1 and described as follows.

Figure 1 Protocol describing procedure, data analysis and expected outputs for a study mapping the multidisciplinary team (MDT) process using the Distributed Cognition for Teamwork (DiCoT) framework. HPB, hepatopancreaticobiliary; upper GI, upper gastrointestinal.

Review of literature to develop cognitive bias catalogue

A literature review will be conducted to create a catalogue of cognitive biases in medical decision-making at the individual and team levels, similar to Stiegler et al.’s22 anaesthesiology-specific cognitive error catalogue. Systematic searches will be carried out in three databases (PsycINFO, Medline/PubMed and EMBASE). The provisional search strategy is presented in table 2 and will be modified for each database. Identified records will be screened to identify articles addressing cognitive biases and errors that medical professionals are documented as exhibiting in various medical decision-making contexts. Data extracted from identified articles include: (1) cognitive bias addressed, (2) participants (3) medical judgement context, (4) type of decision or judgement, (5) whether the bias occurred at the individual or the team level and (6) article type (eg, experimental, case study). These data will be consolidated and, in conjunction with cancer care and MDT experts, used to develop a cognitive bias catalogue, which is a comprehensive table of identified biases, their definitions and illustrative examples of how each bias may manifest in a real-world cancer MDT setting. The catalogue will guide the mapping of potential cognitive biases onto each decision point in the MDT decision-making process, as identified via observations and interviews.

Table 2 Provisional search strategy to identify common biases in medical decision-making, guided by the population, concept and context framework24

Population		Concept		Context	
Medical professionals	AND	Cognitive bias	AND	Medical decision-making	
Combined with OR		Combined with OR		Combined with OR	
ClinicianPhysicianSurge*Radiolog*Oncolog*Anaesth* OR anesthesiolog*	AND	CognitiveAnchoringAvailabilityBase-rateConfirmationFramingRepresentative*Disposition‘illusion of control’Omission‘Sunk cost’CommissionSatisf*HindsightOverconfidenceAmbiguity‘Premature closure’‘Triage cueing’‘Diagnosis momentum’IrrationalErrorRisk	AND	JudgmentThinkingDecisionReasoning‘Critical thinking’BiasFallacyHeuristicErrorShortcut	

Non-participant observations

Observations will be conducted for both HPB and upper GI cancer MDTs by the researcher (HD), using the following standard two-stage method:

Stage 1 involves gathering and organising observations to create a high-level map of the task workflow during MDTs. The scope encapsulates the processes of reviewing individual cases as well as the organisation of the entire meeting. Fieldnotes will be taken on the sequence of activities (eg, information acquisition, information synthesis, action planning and action implementation) as well as on characteristics of the tasks and teamwork (eg, task variety, task interdependencies, team member roles and contributions). These will be gathered by a single researcher (HD) who will sit at the back of the MDT room in an unobtrusive position. The researcher will gather field notes using pencil and paper. They will not participate in the MDT or interact with any of the MDT members while patient cases are discussed.

Stage 2 comprises the DiCoT analysis phase where the MDT workspace will be described according to the 5 themes and 18 principles described in table 1. These outline the potential impact of each element of the MDT environment on the case review process and collaborative decision-making for every patient. Examples of areas of focus captured by field notes include MDTs’ information use practices (eg, nature of information source, access, format), communication (eg, social relationships and nature of information exchanges between different members), changes in practices over time throughout the course of the meeting and physical layout of the meeting space.

Interviews

Concurrent with the observations, semistructured interviews will be conducted with MDT members representing each medical discipline. The interviews will take place in a private location (eg, office or approved video conferencing platform) and the researcher will establish psychological safety to enable participants to feel free to talk openly. Interviews will be audio recorded and transcribed, with notes taken to supplement the transcripts. The interview topic guide was developed and piloted by a research team of psychologists and clinicians who are MDT experts. The topic guide is outlined in table 3. Probing questions will be reframed in a manner that addresses the same topic from multiple angles within participants, to capture tacit knowledge.

Table 3 Semistructured interview guide detailing topics and specific points to guide questions

Topic	Points to cover	
Meeting overview	Structure of the MDT meeting

Structure of individual case discussions

Tasks that need to be accomplished

Changes in case reviewing over time during meetings

	
Collaboration	Team member roles, responsibilities and contributions

Process for reaching consensus

Maintaining situational awareness

Strategies to facilitate communication

Impacts of hybrid meeting

	
Information use and flow	Information requirements and acquisition

Information sources, structures, quality and access

Documenting and sharing information

	
Decision-making	Characteristics of simple versus complex tasks

Dealing with complexity, uncertainty and missing information

Prioritising information and making trade-offs

	
Problems and challenges	Barriers to effective decision-making

Suggestions for improvement

	
MDTmultidisciplinary team

Additionally, as a part of the interviews, participants will review the preliminary decision map developed via observations to ensure it appropriately represents the MDT decision process. This verification will be done iteratively and may require additional interviews to capture the tacit knowledge of members of all specialties present at the participating MDTs. Given the breadth of cancer subtypes and variations between cases within each MDT, this phase will focus on specific types (eg, pancreatic cancers or hepatocellular carcinomas within HPB) to prioritise depth over breadth. We anticipate that using a decision map to guide the interviews may help participants articulate aspects of their knowledge in greater detail than they otherwise may be able to.

Data analysis

Data will be analysed using a hybrid inductive/deductive thematic analysis approach.23 Observation field notes and interview transcripts will be inductively analysed to develop an extensive cognitive process map of MDT decision-making. From this, a hypothetical decision-bias map will be formulated, incorporating insights from the cognitive bias catalogue. This map will comprise a comprehensive list of the varied decision points identified during the case review process, aligned with the potential cognitive biases that may impact judgements at each point. Using DCog principles (eg, information transformation, representation–goal parity, communication bandwidth), we will further identify specific antecedents of cognitive biases in the MDT workspace and environment such as types of information design, task sequences and team interactions.

Interview and observation data will be deductively analysed using the five DiCoT models and principles as categories, to code and describe the distributed cognitive processes occurring in the MDT system. The MDT workspace design implications for decision-making and team communication will be systematically analysed. Through this, we will construct a DCog model of cancer MDT decision-making which will form the theoretical basis for development of a tool to evaluate MDT practices and workspaces to best support cognition and collaboration within the teams during meetings.

Ethics and dissemination

Ethical and management approvals have been obtained from the University of Edinburgh Medical School research ethics committee (23-EMREC-049) and NHS Lothian Research and Development (2023/0245). Additionally, an overview of the study has also undergone review by The Melville Trust for the Care and Cure of Cancer as a part of a competitive grant review process prior to commencement of the project.

Participant consent will be obtained separately for the observational and interview components of the study, recorded via the REDCap platform or paper consent forms prior to the start of data collection. Patient information sheets and consent information have been reviewed and approved by the required regulatory bodies. Given the unobtrusive nature of the study, this research raises only limited ethical concerns for participants. There may be a small risk of psychological distress during interviews while discussing cancer care and end-of-life patients under the care of the participating staff. Although participants routinely deal with end-of-life care as part of their profession, they will be signposted to relevant mental well-being support resources and helplines in the participant information sheet prior to giving consent. All data generated from the study will be anonymised, stored securely and accessed solely by the research team. Fieldnotes and paper consent forms will be stored securely in locked cabinets initially. They will be scanned and uploaded onto the University of Edinburgh’s secure cloud storage (DataStore) within 24 hours of data collection. DataStore will also be used to store all other data generated by the study including consent forms and interview recordings and transcripts. No identifiable patient data will be processed.

An overview of study findings will be shared with members of participating MDTs via a report at the end of the study. Findings will also be disseminated via a PhD thesis, international scientific conference presentations and peer-reviewed manuscripts submitted to high-impact journals.

Data availability statement

Data sharing not applicable as no datasets were generated and/or analysed for this study.

Review Process File
24 08 2024

Funding: This work was supported by The Melville Trust for the Care and Cure of Cancer (grant number R47363).

Prepublication history for this paper is available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-086775).

Patient consent for publication: Not applicable.

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
==== Refs
References

1 Mangus CW Mahajan P Decision Making: Healthy Heuristics and Betraying Biases Crit Care Clin 2022 38 37 49 10.1016/j.ccc.2021.07.002 34794630
2 Howard J Cognitive Errors and Diagnostic Mistakes Springer International Publishing 2019
3 Whelehan DF Conlon KC Ridgway PF Medicine and heuristics: cognitive biases and medical decision-making Ir J Med Sci 2020 189 1477 84 10.1007/s11845-020-02235-1 32409947
4 Croskerry P Singhal G Mamede S Cognitive debiasing 1: origins of bias and theory of debiasing BMJ Qual Saf 2013 22 Suppl 2 ii58 64 10.1136/bmjqs-2012-001712
5 Blumenthal-Barby JS Krieger H Cognitive biases and heuristics in medical decision making: a critical review using a systematic search strategy Med Decis Making 2015 35 539 57 10.1177/0272989X14547740 25145577
6 Saposnik G Redelmeier D Ruff CC et al Cognitive biases associated with medical decisions: a systematic review BMC Med Inform Decis Mak 2016 16 138 10.1186/s12911-016-0377-1 27809908
7 Vittorelli J Cacchillo J McCool M et al Cognitive Bias in the Management of a Critically Ill 29-Year-Old Patient Cureus 2023 15 e39314 10.7759/cureus.39314 37351237
8 Soukup T Lamb BW Shah NJ et al Relationships Between Communication, Time Pressure, Workload, Task Complexity, Logistical Issues and Group Composition in Transdisciplinary Teams: A Prospective Observational Study Across 822 Cancer Cases Front Commun 2020 5 583294 10.3389/fcomm.2020.583294
9 Armstrong BA Dutescu IA Tung A et al Cognitive biases in surgery: systematic review Br J Surg 2023 110 645 54 10.1093/bjs/znad004 36752583
10 Patel VL Kaufman DR Arocha JF Emerging paradigms of cognition in medical decision-making J Biomed Inform 2002 35 52 75 10.1016/s1532-0464(02)00009-6 12415726
11 Hutchins E Cognition in the Wild MIT Press 1995 381
12 Blandford A Furniss D DiCoT: a methodology for applying distributed cognition to the design of teamworking systems InInternational workshop on design, specification, and verification of interactive systems 2005 Jul 13 Berlin, Heidelberg Springer n.d. 26 38
13 Harris J Taylor C Sevdalis N et al Development and testing of the cancer multidisciplinary team meeting observational tool (MDT-MOT) Int J Qual Health Care 2016 28 332 8 10.1093/intqhc/mzw030 27084499
14 Wihl J Rosell L Bendahl P-O et al Leadership perspectives in multidisciplinary team meetings; observational assessment based on the ATLAS instrument in cancer care Cancer Treat Res Commun 2020 25 100231 10.1016/j.ctarc.2020.100231 33157512
15 Lamb BW Wong HWL Vincent C et al Teamwork and team performance in multidisciplinary cancer teams: development and evaluation of an observational assessment tool BMJ Quality & Safety 2011 20 849 56 10.1136/bmjqs.2010.048660
16 Janssen A Robinson T Brunner M et al Multidisciplinary teams and ICT: a qualitative study exploring the use of technology and its impact on multidisciplinary team meetings BMC Health Serv Res 2018 18 444 10.1186/s12913-018-3242-3 29898716
17 Drucker P Landmarks of Tomorrow: A Report on the New Post Modern World 1st ed Available from New York Routledge 1996 Available 10.4324/9781003423461
18 Ball LJ Ormerod TC Putting ethnography to work: the case for a cognitive ethnography of design Int J Hum Comput Stud 2000 53 147 68 10.1006/ijhc.2000.0372
19 Dubbels B Cognitive ethnography: A methodology for measure and analysis of learning for game studies Int J Gaming Comput Mediat Simul (IJGCMS) 2011 3 68 78 10.4018/jgcms.2011010105
20 Pimmer C Pachler N Genewein U Reframing clinical workplace learning using the theory of distributed cognition Acad Med 2013 88 1239 45 10.1097/ACM.0b013e31829eec0a 23887014
21 Vatral C Biswas G Cohn C et al Using the DiCoT framework for integrated multimodal analysis in mixed-reality training environments Front Artif Intell 2022 5 941825 10.3389/frai.2022.941825 35937140
22 Stiegler MP Neelankavil JP Canales C et al Cognitive errors detected in anaesthesiology: a literature review and pilot study Br J Anaesth 2012 108 229 35 22157846
23 Fereday J Muir-Cochrane E Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development Int J Qual Methods 2006 5 80 92 10.1177/160940690600500107
24 Peters MDJ Marnie C Tricco AC et al Updated methodological guidance for the conduct of scoping reviews JBI Evid Synth 2020 18 2119 26 10.11124/JBIES-20-00167 33038124
