
==== Front
Kidney Int Rep
Kidney Int Rep
Kidney International Reports
2468-0249
Elsevier

S2468-0249(24)01766-2
10.1016/j.ekir.2024.06.007
Clinical Research
Virtual Nephron: Evaluation of a Novel Virtual Reality Educational Tool
Nakhoul Georges N. nakhoug@ccf.org
12∗
Taliercio Jonathan J. 12
Bassil Elias H. 1
Arrigain Susana 34
Schold Jesse D. 354
Wardrop Richard 6
O’Toole John 12
Nally Joseph V. 12
Bierer S. Beth 2
Sedor John R. 12
Mehdi Ali 12
1 Department of Kidney Medicine, Medical Subspecialty Institute, Cleveland Clinic, Cleveland, Ohio, USA
2 Cleveland Clinic, Lerner College of Medicine of Case Western Reserve University, Cleveland, Ohio, USA
3 Department of Surgery, University of Colorado - Anschutz Medical Campus, Aurora, Colorado, USA
4 Colorado Center for Transplantation Care, Research, and Education, University of Colorado, Anschutz Medical Campus, Aurora, Colorado, USA
5 Department of Epidemiology, School of Public Health, University of Colorado - Anschutz Medical Campus, Aurora, Colorado, USA
6 Department of Hospital Medicine, Integrated Hospital Care Institute, Cleveland Clinic, Cleveland, Ohio, USA
∗ Correspondence: Georges Nakhoul, Department of Kidney Medicine, Medical Subspecialty Institute, Cleveland Clinic, 9500 Euclid Avenue - Q7, Cleveland, Ohio 44195, USA. nakhoug@ccf.org
11 6 2024
9 2024
11 6 2024
9 9 26192626
3 4 2024
3 6 2024
3 6 2024
© 2024 International Society of Nephrology. Published by Elsevier Inc.
2024
International Society of Nephrology
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Introduction

Recent technological advancements allowed the development of engaging technological tools. Using ASN funding from the ASN, we developed a 3D Virtual Reality (VR) physiology course entitled DiAL-Neph (Diuretic Action and eLectrolyte transport in the Nephron). We hereby present its evaluation.

Methods

The study consisted of 2 parts: evaluation of knowledge gain, and qualitative evaluation of platform reception. Internal medicine PGY1 residents were randomly assigned into 2 groups: a VR group and a conventional group. Knowledge acquisition was assessed with a post-test administered at the end of the course and repeated within 6 to 12 weeks. Independent t-tests were used to compare the number of correct answers between the groups. A survey and focus groups composed of medicine residents evaluated the platform. Sessions were recorded and transcribed verbatim. Data was analyzed through the content analysis approach by two independent reviewers.

Results

Of 117 PGY1 resident participants, 64 were randomized to the VR group and 53 were randomized to the traditional group. Initial test results showed higher scores among VR compared to the traditional group (76.5% correct vs. 68.8%). Seventy-eight PGY1s participated in the follow up testing (46 VR group vs. 32 traditional group) and results showed no significant difference in test results. Greater than 90% of the residents rated the platform positively and 77% preferred it as a teaching method.

Conclusion

The DiAL-Neph VR platform appeared to improve short-term learning but not long-term retention. Further studies are needed to investigate the impact of such teaching platforms on overall interest in nephrology.

Graphical abstract

Keywords

education
renal physiology
virtual reality
==== Body
pmcComputing development has blossomed in the past 2 decades leading to transformative advances and proliferation of digital education, including computer-based learning, virtual patients, and VR.1 VR provides a fully immersive 3D experience allowing the user to interact with and manipulate a computer-generated environment. Traditionally, the use of VR in medicine was relegated primarily to development of technical skills in procedural training.2 VR utilization has more recently been employed to develop cognitive interactive learning in various specialties such as cardiology,3 critical care medicine,4 emergency medicine,5 and nursing education.6 To date, no studies have examined the use of VR in nephrology education.

Nephrologists have some of the most complex patients when compared to other fields of medicine.7 Interest in nephrology as a vocation is waning due partly to this complexity coupled with a perceived lack of innovation.8 A recently published study revealed that only 2.5% of internal medicine residents were planning to pursue nephrology.9 Unfortunately in the 2022 medicine subspecialty fellowship match, 27% of nephrology positions and 42% of training programs went unfilled.10 Expectedly, there is a growing shortage of nephrologists in the workforce.11 We hope that technological advances such as VR could be utilized to demystify renal physiology, enhance the nephrology educational experience, and present nephrology as an innovative specialty. Therefore, with funding from the American Society of Nephrology William and Sandra Bennett Clinical Scholars Program, we developed a 3D-VR renal physiology course called DiAL-Neph, which is focused on renal tubular handling of water and electrolytes along with diuretic action.

This study evaluated the use of DiAL-Neph by Cleveland Clinic PGY1 internal medicine residents to improve their understanding of renal physiology.

Methods

This is a mixed method study with the following 2 key elements: (i) a randomized trial comparing DiAL-Neph with conventional learning in PGY1 internal medicine residents, and (ii) a qualitative evaluation of the VR platform and its perceived utility on learning enhancement.

DiAL-Neph VR Course

The VR course was developed using the gaming platform “Unity” and delivered on the “Oculus Quest” device by “Meta.” Oculus Quest is an advanced all-in-one VR headset that fully immerses the user in an animated virtual environment. The system utilizes inside-out tracking to power the headset without the need to tether the system to a computer or external sensors. DiAL-Neph takes the user inside a magnified virtual kidney and allows them to interact with several animations that detail the handling of sodium and water across various sites of the nephron. Interaction with the environment allows the user to visualize how the use of a specific diuretic can affect the transport of water and electrolytes in different segments of the nephron. A total of 10 Oculus Quest machines were utilized to deliver the content to the participants in small groups (Figure 1).Figure 1 VR classroom set-up, application screenshots. Use the QR code to view a video demonstrating the DiAL-Neph application. DiAL-Neph, Diuretic Action and eLectrolyte transport in the Nephron; VR, virtual reality.

Study Participants and Recruitment

The study participants consisted of PGY1 categorical internal medicine residents from the Cleveland Clinic recruited from the following 3 academic years (2020–2021, 2021–2022, and 2022–2023). Residents received an e-mail detailing the study and were given the opportunity to opt out of participating. Residents who opted out were invited to attend the 2-hour course but did not complete the knowledge assessment. They were offered access to the VR course at the completion of the study.

Course Design

Participants within each year cohort were randomized 1:1 into 2 groups: VR group (exposed to the VR session) and conventional group (received a printed script of the VR learning course). In the first year, we randomized all residents that agreed to participate in the trial. Unfortunately, many did not end up participating due to scheduling conflict. Subsequently, in the following 2 years, we only randomized the residents whose tentative schedules at the time of planning the randomization permitted them to attend the class. Nonetheless, schedules still changed for some residents, and they were unable to attend the course (Figure 2).Figure 2 Randomization protocol. PGY1, post-graduate year 1; VR, virtual reality.

Residents in the VR group met in groups of 5 to 10 and interacted with the DiAL-Neph simulation for an average of 45 minutes. Residents randomly assigned to the conventional group received a printed equivalent of the VR learning course (containing the script and images of the course). Within a week of being exposed to the VR versus conventional intervention, both groups were given a 2-hour seminar on physiology of solute or water transport and diuretics. Twenty minutes of the seminar consisted of a review of solute or water transport and diuretics, material previously covered in both the VR and script. The remaining time was devoted to discussing 4 clinical cases that highlighted practical use of diuretics in different kidney, liver, and heart pathologies. The cases also focused on diuretic pharmacokinetics, dosing, resistance, and side effects.

Evaluation of Knowledge

Knowledge acquisition and retention were assessed with a posttest administered immediately after the conclusion of the 2-hour seminar and repeated within 6 to 12 weeks. The test underwent several rounds of piloting by nephrology faculty and fellows and consisted of a total of 40 questions (15 true/false, 14 pairing questions, and 11 multiple choice questions). Each question was allotted 1 point. Half of the questions were focused specifically on mechanistic processes to ascertain if the residents gained increased understanding from the spatial visualization offered by the VR course. The remaining questions were centered on nonmechanistic information that was covered in the VR, script, and lecture. For example, a question asking about the site of action of loop diuretics (being luminal rather than basolateral) was considered mechanistic, whereas a question asking about the optimal dosing regimen of hydrochlorothiazide was not. The 40-question test was issued using the secure platform RedCAP. Tests were anonymous, thereby preventing paired test comparisons.

Evaluation of Platform Reception

Evaluation of the VR platform was performed using a RedCAP-based questionnaire that was administered to the VR group at the end of the 2-hour lecture. The questionnaire combined open-ended questions for narrative feedback along with Likert-scale questions focused on the following parameters: interface, clarity of the delivered material, educational value, user engagement, likability, need for improvement, and interest in dissemination. The Guiding Questions for Focus Group are presented in the Supplementary Material.

Evaluation of Perceived Course Utility

Residents in the VR group were asked to participate in a focus group so that their feedback could be elicited. The focus group gathered residents’ perceptions after the completion of the second posttest. A total of 3 focus groups were convened, 1 for each academic year, for an average of 1 hour. The focus groups’ sessions were recorded and transcribed verbatim. Data were analyzed using the content analysis approach,12 a qualitative method designed to identify patterns and themes within the data in order to detect implicit and explicit ideas and provide an explanation to observed phenomena.12 Two independent reviewers formally trained in medical education and qualitative analyses (GNN and AM) evaluated focus group data in 2 phases: (i) line-by-line coding to extract ideas; and (ii) categorizing line-by-line ideas into themes. Then, the reviewers met to compare coding and reach consensus on themes and categories.

Statistics

To assess knowledge acquisition and retention, we scored the test by adding the number of correct test answers (maximum score of 40) and calculated the percentage of correct answers. We used 2-sample independent t tests to compare the test scores and the percentage correct between the VR and conventional groups at 2 different time points: initial testing, immediately following the 2-hour session; and follow-up testing, 6 to 12 weeks later. Tests were anonymous and it was not possible to link results from both test administrations, so we did not know which residents completed both session tests, and the initial and follow-up tests were evaluated separately.

For the platform assessment (VR group only), we summarized the number and percentage of residents who agreed or strongly agreed to each given statement.

The study was reviewed and approved by the Cleveland Clinic’s Office of Institutional research.

Results

Of the 188 PGY1 residents who were eligible to participate, 167 had tentative schedules that would potentially allow class attendance and were randomized to participate. Of these, 117 had open schedules on the day of the course and attended the 2-hour seminar and completed the initial testing. Sixty-four had been randomized to the VR group and 53 had been randomized to the conventional group. Of the participants, 51.3% were males. Average age was 27.3 ± 2.0 years (Table 1).Table 1 Participant characteristics by DiAL-Neph VR participation

Factor	Overall
(N = 117)	VR
(n = 64)	Conventional
(n = 53)	
Agea	27.3 ± 2.0	27.2 ± 1.7	27.3 ± 2.4	
Gender				
 Male	60 (51.3)	34 (53.1)	26 (49.1)	
 Female	57 (48.7)	30 (46.9)	27 (50.9)	
Medical training program attended.				
 US MD program	69 (59.0)	41 (64.1)	28 (52.8)	
 US DO program	19 (16.2)	8 (12.5)	11 (20.8)	
 Other international	29 (24.8)	15 (23.4)	14 (26.4)	
DiAL-Neph, Diuretic Action and eLectrolyte transport in the Nephron; DO, Doctor of Osteopathic Medicine; MD, doctor of medicine; VR, virtual reality.

Statistics presented as mean ± SD, or N (column %).

a Age was missing for 7 participants

Results of the initial posttest showed higher scores among VR versus conventional group (76.5% correct vs. 68.8%, P < 0.01; Table 2). Seventy-eight PGY1s participated in follow-up testing (46 VR group vs. 32 conventional group) and results showed no significant difference in test results (P = 0.4). Test score results are summarized in Table 2. Of the 64 PGY1 residents who were randomized to the VR group and completed the 2-hour seminar, 62 completed the platform assessment. Overall, more than 90% of the residents rated the platform positively in all parameters, and 77% preferred it as a teaching method (Table 3).Table 2 Knowledge by DiAL-Neph VR participation

	Overall
(N = 117)	VR
(n = 64)	Conventional
(n = 53)	t test
P-value	
First session					
Absolute test score (total correct answers)	29.2 ± 5.0	30.6 ± 4.4	27.5 ± 5.3	<0.001	
Test score percentage	73.0 ± 12.6	76.5 ± 11.1	68.8 ± 13.2	<0.001	
Nonmechanistic questions Correct	15.9 ± 3.0	16.5 ± 2.6	15.0 ± 3.3	0.006	
Mechanistic questions correct	13.4 ± 2.7	14.1 ± 2.5	12.5 ± 2.7	0.002	
Follow-up session	Overall
(N = 78)	VR
(n = 46)	Conventional
(n = 32)	t test
P-value	
Absolute test score (total correct answers)	26.2 ± 5.0	26.6 ± 5.2	25.6 ± 4.7	0.40	
Test score percentage	65.5 ± 12.5	66.5 ± 13.0	64.1 ± 11.7	0.40	
Nonmechanistic questions correct	13.4 ± 3.4	13.6 ± 3.4	13.1 ± 3.5	0.52	
Mechanistic questions correct	12.8 ± 2.8	13.0 ± 3.0	12.5 ± 2.5	0.46	
DiAL-Neph, Diuretic Action and eLectrolyte transport in the Nephron; VR, virtual reality.

Statistics presented as mean ± SD.

Table 3 Survey evaluating the VR platform

Survey question	N agree or strongly agree (Total N = 62)	% agree or strongly agree	
The program is user friendly	58	94	
The response time of the program is adequatea	59	97	
I was absorbed in the activity of the simulation	61	98	
The delivered material was clear	59	95	
The delivered material appropriately covered the learning objectivesa	59	97	
DiAL-Neph made it easy to understand the learning objectives	57	92	
DiAL-Neph is a useful learning aid	57	92	
DiAL-Neph enhanced my understanding of nephrology concepts	58	94	
I found DiAL-Neph enjoyable to use while learning	60	97	
I preferred using DiAL-Neph to the standard teaching method	48	77	
I would like the DiAL-Neph to be utilized for more topics in nephrology	58	94	
DiAL-Neph, Diuretic Action and eLectrolyte transport in the Nephron; VR, virtual reality.

a Missing data for 1 respondent.

Three focus groups met for 1 hour. The focus groups comprised 9, 8, and 15 PGY1 residents from 3 different academic years. Several recurring themes emerged in our analysis as presented in Table 4. Among the positive themes, what stood out was the ability of DiAL-Neph to capture the attention span of the learner and keep them engaged throughout the simulation, as supported below with representative comments from participants:“Usually, when I watch a video, I feel like it’s so easy for me to doze off. I can’t stay focused for too long. But this was very engaging. I could turn around and see the different structures.”

Table 4 Focus group themes

Positive themes	Example of quote (s)	
Recall: memory anchor with recall that was described as relevant to clinical practice	“I was on cardiology rotation and the attending was asking me about the action of diuretics. I was able to tell him precisely where it worked in the tubules. I remembered the images very well.”	
	“I also think that it was helpful as a memory anchor […]. I think I’m more likely to remember it.”	
	“When I did the follow-up test, I was imagining those images in my head.”	
Attention span: hook that holds users’ focus	“When I watch a video, I feel like it’s so easy for me to doze off. I can’t stay focused for too long. This was very engaging. I could turn around and see the different structures”	
	“I like that you can’t zone out either […], if I’m watching a YouTube video, I’m looking around at other things. When you’re in the VR, there’s literally no way that you can escape what you’re watching.”	
Interaction: engaging and powerful	“I think the strength for me was that it’s more so interactive. It’s more of like a give-take than just looking at an image or watching a video on YouTube.”	
Spatiality: improves understanding of structures	“I never understood how calcium and magnesium are reabsorbed in the TAL until I did this course. The VR made it so clear to me.”	
	“[…] the 3D nature of it, it helps a lot to understand 3D structures […]. But being able to actually turn around and see a 3D structure […], I think that’s what this tool is very, very good for.”	
Enjoyable: cool/fun/graphically appealing	“I think that the content there was really powerful. I thought that seeing it all in VR was very cool […]. I really enjoyed the experience.”	
	“I thought that was really cool and definitely a positive.”	
	“I want to say I really enjoyed the experience.”	
Great supplemental resource	“This was awesome! We want the other specialties to do the same!”	
	“I had my nephrology rotation in medical school, but I felt that this was supplementing my knowledge.”	
Negative themes	Example of quote (s)	
Lack of immediate clinical relevance	“I didn’t find myself at all in the past few months referring back to it to make clinical decisions necessarily. It doesn’t stick out to me. I didn’t think back saying: Oh, I remember at the VR this was how this worked.”	
More interaction: somewhat passive experience despite interactive format	“I also think it would be really fun if there were games.”	
	“I think just jumping off the interaction, gamifying it for sure would make it more interesting.”	
Organization: best to teach VR and lesson concurrently	“I would have liked incorporating [the lecture] into the VR somehow. Or doing even multiple-choice questions on the VR.”	
Environment control: ability to change the speed and graphics of the simulation	“I didn’t realize that I could actually walk closer to the animation. So, I was actually watching the graphics from really far away.”	
	“I didn’t know I could pause the animation. I would have liked to be able to change the speed.”	
Logistical challenges: inability to take notes	“I would have liked to be able to take notes, but it was not possible during the simulation.”	
Technical challenges: motion sickness, lag	“I felt a little nauseous during that time.”	
	“There was in it was just a little bit of a lag occasionally, where I would move my head and it wouldn’t move with me.”	

DiAL-Neph was also felt to improve recall of learned material in a way that was relevant to clinical practice.“I was on cardiology rotation and the attending was asking me about the action of diuretics. I was able to tell him precisely where it worked in the tubules. I remembered the images very well.”

Many residents felt that the DiAL-Neph improved their understanding of nephrology and recommend extending this learning method to other specialties.“I never understood how calcium and magnesium are reabsorbed in the TAL until I did this course. The VR made it so clear to me.”

“I felt like: I got this!”

“This was awesome! We want the other specialties to do the same!”

However, several logistical challenges were raised by the learners. One resident highlighted the inability to take notes during the simulation as potentially impeding learning. Another suggested having the clinical cases run in parallel to the simulation rather than in a separate session. Finally, 1 resident disagreed with their colleagues’ consensus and did not feel that the simulation was relevant to this resident’s clinical practice.

Discussion

Our study showed that the DiAL-Neph VR program is a useful learning adjunct which can improve short-term knowledge acquisition but perhaps without affecting long-term retention. In addition, the program was overwhelmingly well-received by the residents and was perceived to improve their learning experience. To our knowledge, this is the first study assessing the use of a VR program in nephrology.

Numerous studies have assessed the impact of VR interventions on knowledge acquisition and interactive interventions improve knowledge acquisition.2 However, given the wide variety of topics studied, those findings are not unanimous and can be conflicted. For example, a meta-analysis assessing VR in anatomy found improvement in test scores compared to conventional approaches,13 whereas a meta-analysis of VR simulation training in endoscopy found no difference with conventional training.14 However, most VR intervention studies assessed anatomic learning and procedural skill acquisition; 2 instances where spatial understanding is paramount. This is relevant because individual spatial ability is a predictor of improved learning.15,16 Therefore, VR technology could be particularly useful for learning that requires a higher degree of spatial understanding and navigation of anatomical structures. Our study is unique in that it evaluated both medical knowledge and understanding of spatial nephron structures. In both instances, we found short-term knowledge acquisition to be superior in the VR DiAL-Neph group.

Our study did not show differences in test scores 6 to 12 weeks after the initial intervention. Accurate assessment of long-term retention of medical knowledge in trainees is difficult due to “maturation.” Maturation argues that a trainee’s knowledge acquisition is not confined to an isolated experience but rather continues as the resident advances in medical education and clinical exposure. For this reason, only a few studies in the literature pursue long-term teaching periods and assessments with most of them limiting evaluations to <1 month.17 We believe that maturation is responsible for the lack of long-term differences in retention among the 2 groups. We also suspect that the educational seminar following the intervention may have blunted the difference between the 2 groups because many of the concepts presented during the intervention were repeated and applied to clinical cases. In addition, it is possible that participants in both groups forgot some of the learning they initially received, and that the impact of the intervention simply diminished over time. Finally, the loss to follow-up may have altered the comparisons, and the study might not have been powered to detect long term differences, but it’s notable that numerical scores remained slightly higher in the VR group.

The residents’ feedback of DiAL-Neph was overwhelmingly positive with more than 90% of them rating the platform positively in all parameters and 77% preferring it as an alternative teaching method. We surmise that some of the enthusiasm for VR was due to the novelty of the experience as compared to PowerPoint fatigue and more conventional didactics. Nonetheless, the strength of the positive response appears superior to what has been typically observed by other studies.16,18 Notably, DiAL-Neph was highlighted for its capacity to engage the learners and capture their attention span, ultimately serving as a memory anchor facilitating recall during clinical practice. Students who are motivated and engaged by innovative tools are more likely to retain knowledge19 and which could explain the observation of improved short-term outcomes. We believe that this positive response is more relevant than improvement in test scores and is likely the most important finding of the study. Nephrology is struggling to recruit, and some of the proven culprits include topic complexity, lack of innovation, and insufficient exposure to the specialty.8 An intervention that can present renal physiology in an innovative and enjoyable way has the potential to reengage prospective residents and positively alter their perception of the specialty. Anecdotally, since we commenced our trial, we observed a pipeline of 2 to 3 internal medicine applicants per year who have pursued nephrology fellowships, whereas we did not have any in the previous 4 years.

Our study had a unique design coupling or pairing of the intervention with a 2-hour case-based seminar. We postulated that the DiAL-Neph program would be most helpful as an educational adjunct rather than a solitary course. Indeed, if the VR group had a different understanding of the concepts based on the visual experience, then the participants may better absorb the information presented in the seminar. Based on the study’s findings, we believe that we were correct in that assessment and that the DiAL-Neph program is indeed a useful adjunct.

Our study’s strength is in the administration of a randomized educational intervention in 1 of the largest residency programs of the country over a period of 3 years. Tests were administered during a dedicated lecture time and were undertaken under supervision, which precluded the participants from utilizing outside resources to answer the questions. There are limitations of our study. First, residents were not rewarded for achievement, and therefore may not have been fully motivated to fully participate. Second, we recognize the logistical challenges in replicating our educational method. Although the price of the device itself is cheaper than a cell phone, its widespread use would require considerable resources for software development, faculty training, and physical space. It is worthy to note that the VR experience can be delivered asynchronously, and that this potentially could cut down on some of the cost by decreasing both the number of headsets as well as the physical space required. Third, we recognize that the animations are presented in a 2-dimensional format within a 3D environment, and this may not be fundamentally different from watching a video on a 2-dimensional screen. However, based on the residents’ feedback and focus group discussions, we found that this 3D environment offered an attention hook that kept the residents engaged, and a “wow” factor that made them appreciate the simulation far beyond a 2-dimensional video.

Conclusion

The DiAL-Neph VR platform appeared to improve short-term learning but not improving long-term retention. It was immensely well-received by the residents and was perceived as fun and engaging. Based on the experience we described, we believe that cutting edge innovation may have the potential to affect residents’ understanding of complex renal physiology. Further studies are needed to assess if demystifying nephrology and improving understanding of renal physiology-notably through the use of cutting-edge technology and innovative modalities-can actually impact the interest in nephrology as a field and reverse this decade-long negative trend.

Disclosure

All the authors declared no competing interests.

Supplementary Material

Supplementary File (PDF)

Guiding Questions for Focus Group.

Acknowledgments

Funding was provided by the American Society of Nephrology William and Sandra Bennett Clinical Scholars Program (grant number ASN1805 GN).

Data Sharing Statement

The clinical trial data of this article are available upon reasonable request to the corresponding author.

Author Contributions

Research idea and study design was by GNN, JJT, RW, JO, JVN, JRS, and AM. Data acquisition was done by GNN and EHB. Data analysis and interpretation was done by GNN, JJT, EHB, SA, JDS, RW, JO, JVN, SBB, JRS, and AM. Statistical analysis was done by SA and JRS. Supervision or mentorship was by JJT, RW, JO, JVN, SBB, JRS, and AM. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved.

Supplementary File (PDF)

Guiding Questions for Focus Group.
==== Refs
References

1 Car J. Carlstedt-Duke J. Tudor Car L. Digital education in health professions: the need for overarching evidence synthesis J Med Internet Res 21 2019 e12913 10.2196/12913
2 Kyaw B.M. Saxena N. Posadzki P. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration J Med Internet Res 21 2019 e12959 10.2196/12959
3 O’Sullivan D.M. Foley R. Proctor K. The use of virtual reality echocardiography in medical education Pediatr Cardiol 42 2021 723 726 10.1007/s00246-021-02596-z 33856495
4 Bruno R.R. Wolff G. Wernly B. Virtual and augmented reality in critical care medicine: the patient’s, clinician’s, and researcher’s perspective Crit Care 26 2022 326 10.1186/s13054-022-04202-x 36284350
5 Wu T.C. Ho C.B. A scoping review of metaverse in emergency medicine Australas Emerg Care 26 2023 75 83 10.1016/j.auec.2022.08.002 35953392
6 Plotzky C. Lindwedel U. Sorber M. Virtual reality simulations in nurse education: a systematic mapping review Nurse Educ Today 101 2021 104868 10.1016/j.nedt.2021.104868
7 Tonelli M. Wiebe N. Manns B.J. Comparison of the complexity of patients seen by different medical subspecialists in a universal health care system JAMA Netw Open 1 2018 e184852 10.1001/jamanetworkopen.2018.4852
8 Nakhoul G.N. Mehdi A. Taliercio J.J. “What do you think about nephrology?” a national survey of internal medicine residents BMC Nephrol 22 2021 190 10.1186/s12882-021-02397-9 34020598
9 Paralkar N. LaVine N. Ryan S. Career plans of internal medicine residents from 2019 to 2021 JAMA Intern Med 183 2023 1166 1167 10.1001/jamainternmed.2023.2873 37639258
10 Pivert K.A. First look: AY 2023 match. ASN Data https://data.asn-online.org/posts/ay_2023_match/index.html
11 Sozio S.M. Pivert K.A. Caskey F.J. Levin A. The state of the global nephrology workforce: a joint ASN-ERA-EDTA-ISN investigation Kidney Int 100 2021 995 1000 10.1016/j.kint.2021.07.029 34474074
12 Hsieh H.F. Shannon S.E. Three approaches to qualitative content analysis Qual Health Res 15 2005 1277 1288 10.1177/1049732305276687 16204405
13 Zhao J. Xu X. Jiang H. Ding Y. The effectiveness of virtual reality-based technology on anatomy teaching: a meta-analysis of randomized controlled studies BMC Med Educ 20 2020 127 10.1186/s12909-020-1994-z 32334594
14 Khan R. Plahouras J. Johnston B.C. Scaffidi M.A. Grover S.C. Walsh C.M. Virtual reality simulation training in endoscopy: a Cochrane review and meta-analysis Endoscopy 51 2019 653 664 10.1055/a-0894-4400 31071757
15 Berney S. Bétrancourt M. Molinari G. Hoyek N. How spatial abilities and dynamic visualizations interplay when learning functional anatomy with 3D anatomical models Anat Sci Educ 8 2015 452 462 10.1002/ase.1524 25689057
16 Yammine K. Violato C. A meta-analysis of the educational effectiveness of three-dimensional visualization technologies in teaching anatomy Anat Sci Educ 8 2015 525 538 10.1002/ase.1510 25557582
17 Jiang H. Vimalesvaran S. Wang J.K. Lim K.B. Mogali S.R. Car L.T. Virtual reality in medical students’ education: scoping review JMIR Med Educ 8 2022 e34860 10.2196/34860
18 Murgitroyd E. Madurska M. Gonzalez J. Watson A. 3D digital anatomy modelling - practical or pretty? Surgeon 13 2015 177 180 10.1016/j.surge.2014.10.007 25593105
19 Battulga B. Konishi T. Tamura Y. Moriguchi H. The effectiveness of an interactive 3-dimensional computer graphics model for medical education Interact J Med Res 1 2012 e2 10.2196/ijmr.2172
