
==== Front
MedEdPublish (2016)
MedEdPublish (2016)
MedEdPublish
2312-7996
F1000 Research Limited London, UK

10.12688/mep.19535.3
Research Article
Articles
A mixed methods, longitudinal study: characterizing the differences in engagement and perceived learning of medical students in online and in-person team-based learning classes
[version 3; peer review: 2 approved, 3 approved with reservations, 1 not approved]

Lee Irene Cheng Jie Conceptualization Data Curation Formal Analysis Funding Acquisition Investigation Methodology Project Administration Validation Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-6862-6438
1
Wong Peiyan Conceptualization Data Curation Formal Analysis Funding Acquisition Investigation Methodology Project Administration Resources Software Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-3734-2461
a12
1 MD Programme, Duke-NUS Medical School, Singapore, 169857, Singapore
2 Department of Pharmacology, National University of Singapore, Singapore, Singapore
a gmswpy@nus.edu.sg
No competing interests were disclosed.

11 7 2024
2023
13 338 7 2024
Copyright: © 2024 Lee ICJ and Wong P
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background

The rapid transition from in-person to online delivery of medical curriculum has facilitated the continuation of medical education during the COVID-19 pandemic. Whilst active learning approaches, including Team-Based Learning (TBL), are generally more supportive of the learner’s needs during such transition, it remains elusive how different learning environments affect a learner’s motivation, engagement, and perceived learning over a prolonged period. We leveraged on the Self-Determination Theory (SDT) and key learners’ characteristics to explore the levels of student’s engagement and perceived learning in two TBL learning environments, online and in-person, over an extended period. We hypothesize that students’ self-reported perceptions of engagement and learning will be lower in online compared to in-person TBL classes.

Methods

This is a mixed methods study with 49 preclinical graduate medical students completing the same questionnaire twice for each learning environment, online TBL and in-person TBL, over an eight-month period. Quantitative data were collected on learners’ characteristics, basic psychological needs satisfaction, motivation, student’s engagement and perceived learning. Additionally, the final questionnaire also explored the participants’ perception on which learning environment better supported their learning.

Results

We found that autonomy support, perceived competence and needs satisfaction, and perceived learning were higher in-person than online. Additionally, most learners felt that in-person TBL was better for learning, as the concepts of learning space and the community of practice were mediated by being in-person.

Conclusions

TBL, being an active instructional method, can maintain students’ engagement because it supports many aspects of SDT constructs and perceived learning. However, online TBL is unable to fully support the students’ needs and perceived learning. Hence, we strongly advocate for any in-person opportunities to be included in a course, as in-person classes best support students’ engagement and perceived learning.

TBL
SDT
motivation
cognitive strategies
community of practice
ALSET seed grant programme provided by the NUS Office of the Senior Deputy President & Provost and the NUS Institute for Applied Learning Sciences and Education Technology.This research was supported by the ALSET seed grant programme provided by the NUS Office of the Senior Deputy President & Provost and the NUS Institute for Applied Learning Sciences and Education Technology. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Revised Amendments from Version 2

Introduction: We moved the description and figure about Team-Based Learning from the Methods section to the Introduction. Additionally, we made textual edits in the first paragraph on the initial reports of the lessons learnt in the online transition.   Methods: Under the header "Experimental Design," we revised the text to provide better clarity on the overall timeline of the administered survey.  Result: We revised the text under the header “Which learning environment did students feel that they learnt better in?” to provide better clarity. This segment is on how interpersonal interaction and team dynamics influence students’ preference for a learning environment.    Figures: We swapped Figure 1 and Figure 2 to align with the textual edits in the Introduction. Figure 1 is now the Team-Based Learning process, while Figure 2 is the theoretical framework of the current study.
==== Body
pmcIntroduction

Enhanced social distancing measures due to the COVID-19 pandemic had transformed the delivery of education, from onsite physical lessons to online settings 1– 3 . Due to the looming threat of infectious variants and the rapidly changing pandemic situation, many educational institutions implemented either online learning or a hybrid arrangement of blended in-person and online learning for a prolonged period 4 . This led to many initial reports on the lessons learnt and efforts in the online transition 1, 5 , particularly in terms of learners’ perception of the online learning environment 3 . However, it remains unclear whether being in an online learning environment over an extended period changes a learner’s motivation and in turn, the level of engagement and the perception of learning. Hence, gaining a better understanding of the effects of learning environments on students would help to identify key elements needed to support learners in a prolonged online learning environment, and inform the future integration of online learning into the existing medical curriculum.

At Duke-NUS Medical School, Singapore, we adopt Team-Based Learning (TBL) as our main instructional method in the preclinical curriculum ( Figure 1). In brief, for a typical TBL class at Duke-NUS, students study the assigned learning material prior to classes. During class, students take the readiness assurance tests individually and then as a team. After which, there is a clarification phase for students to clear up their remaining doubts, and their queries would be addressed as a class during an instructor-facilitated discussion. We have previously shown that a week-long, online learning experience did not affect students’ perception of their classes 2 . However, the online learning classes in that study did not encompass all aspects of a typical TBL session, as no video-conferencing was used, therefore there was no active discussion between students and faculty. Furthermore, there is a lack of research on the effect of online versus in-person TBL on students’ engagement and perceived learning. Hence, in this study, we used the self-determination motivational theory framework, together with key learner’s characteristics, to identify changes to the factors underlying learners’ engagement and perceived learning when taking classes in two different learning environments, over an extended length of time.

Figure 1. An infographic showing the sequence of events that happens for a TBL class at Duke-NUS.

Students undergo these TBL phases for all classes in the first year of medical school at Duke-NUS, regardless of whether the classes were held online or in-person. The way that pre-class preparation materials were assigned, and the TBL-supported software used for the assurance tests were the same in both learning environments. Please see Lee et al., 2022 for a review comparing the resources required for each learning environment 4 .

Theoretical framework

According to the Self-Determination Theory (SDT) motivational theory, motivation is a continuum, ranging from amotivation, a lack of motivation state, to extrinsic motivation which is propelled by an external factor or reward, and then to intrinsic motivation, a self-generated state 6 . Intrinsic motivation, which is the pursuit of an activity for one’s personal interest, is shown to associate with better learning outcomes and well-being 7– 10 . Such motivation fosters engagement on any task. SDT further differentiates the types of extrinsic motivation based on the level of autonomy 6, 11 . External regulation, the least autonomous one, is outside of personal control and driven by external reward or avoidance of punishment, whereas identified regulation, the most autonomous one, is associated with valuing an activity. Both intrinsic motivation and identified regulation are the autonomous types of self-determination for a learner.

Importantly, SDT hypothesizes that learner’s motivation on any given task can change from extrinsic to intrinsic, depending on the satisfaction of the learner’s three basic psychological needs: autonomy, competence, and relatedness. Autonomy refers to acting with a sense of volition; competence is self-perceived efficacious in a learning environment; relatedness is the feeling of being connected and belonging to other or one’s community. It was further demonstrated that students are more able to stay motivated and engaged in a learning task when the instructional method fulfills these psychological needs 12 . Furthermore, when in-person, the TBL instructional method was better than the didactic instructional method, at fostering intrinsic motivation, perceived competence, autonomy and overall needs satisfaction 13 . Using the SDT framework, it helps us to understand why learners stay engaged and develop greater sense of perceived learning in a TBL environment.

While SDT is important in shaping the fundamental aspects of engagement and learning, research has shown that there are other factors, such as learners’ characteristics, that interact with the SDT framework, and motivation via their correlations with the basic psychological needs satisfaction. These key characteristics include curiosity, resilience, and growth mindset 14– 20 . Pintrich (1990) also suggested that motivated students were cognitively engaged and more likely to apply meta-cognitive learning strategies, becoming self-regulated learners 21 . Hence, we included the use of cognitive strategies and self-regulation in our theoretical framework 21 .

Given the dynamic relationship between the learning environment and various learners’ characteristics, and the satisfaction of the basic psychological needs for motivation, we seek to elucidate how these factors influence learners’ engagement and perceived learning ( Figure 2). We posit that the reduced or lack of social interactions of the students, with their team members and faculty, in an online TBL learning environment might lead to reduced satisfaction of basic psychological needs compared to an in-person TBL learning environment. Hence, students’ engagement and perceived learning in online TBL will be adversely affected.

Figure 2. Theoretical framework to elucidate student’s engagement and perceived learning (measured outcomes) via the measured SDT and learner’s characteristics constructs in the in-person and online TBL learning environments.

Specifically, we aim to characterise the differences in the learners’ characteristics (such as resilience, curiosity, growth mindset, use of cognitive strategies and self-regulation), basic psychological needs satisfaction (autonomy, competence and relatedness), situational motivation (intrinsic, identified, external and amotivation), student’s engagement and perceived learning between the two TBL learning environments, online and in-person, over an extended period of about four months in each environment. Understanding such differences between the learning environments will inform the academic support system on how best to engage learners in their learning environment. Besides, the key elements identified in this relationship will also facilitate integration of online TBL learning into existing medical curriculum in this post-COVID-19 era.

Methods

Participants

We recruited, via e-mail, 49 participants from a pool of 72 graduate medical students (comprised of 45 females and 27 males), who were in the first-year of their studies at Duke-NUS Medical School. There were 30 females and 19 males who participated in our study. Their age distributions were as follows: 24 participants were between the ages of 21–25 and 25 participants were over the age of 26.

Context of the learning environment: Preclinical Curriculum

Our preclinical curriculum integrates the basic science content into a one-year programme, Foundations of Patient Care (FPC), which is covered in two parts: (1) FPC Part 1 adopts an integrated, systems-based approach to study the human body’s structure and function, (2) Part 2 progresses to a systems-cutting approach, focusing on disease. This developmental framework is designed to establish an understanding of the "normal" body before delving into the "abnormal," while also introducing essential clinical reasoning skills. To deliver this curriculum, we utilize a range of teaching and learning methods with TBL as one of the main strategies. The curriculum assesses students with summative high-stake knowledge and clinical performance assessments.

Recruitment procedure

This mixed-methods, quasi-design study used survey methodology and was exempted by the National University of Singapore Institutional Review Board (NUS-IRB-2020-346). An administrative staff member with no grading responsibilities emailed all first-year graduate medical students with links to the questionnaire, consisting of demographic questions and validated survey instruments measuring the various constructs (detailed below). Participants indicated their informed consent electronically before completing the questionnaire.

In-person and online TBL instructional methods

At Duke-NUS, TBL is the main instructional method 22 for first year classes, regardless of whether they are online or in-person ( Figure 1). Both the online and in-person TBL classes consist of all the phases of a typical TBL class, except that in an online TBL class, a video-conferencing software, Zoom, would be used in place of where there would be face to face discussions in the in-person TBL class 1 . Hence, students would be put in breakout rooms when they are doing the team readiness assurance tests and working through the clarification questions. The instructor-facilitated discussions would follow in the main room on Zoom. When in-person, the whole team would be in the classroom and students would be able to carry out group discussions with their teammates face to face. The administrative staff and facilitating instructors would be present in the classroom.

Experimental design

In our study, all 49 participants completed a total of four surveys, interspersed throughout the academic year, with two surveys administered for each learning environment, online and in-person ( Figure 3). All Year 1 classes were carried out using the TBL format.

Figure 3. Timeline of the experiment showing when the four surveys, two for each learning environment, were administered.

The social distancing requirements by the Singapore government and the university were included to provide context for why and when classes were online and in-person, in a hybrid format. All first-year classes were carried out using the TBL format. This cohort of students started TBL classes online in August of 2020 due to the high number of cases in Singapore. In January 2021, synchronous hybrid TBL classes were implemented for this cohort, when cases were lower and social distancing measures were relaxed. Students followed a roster for when they were supposed to attend classes in-person (see Figure 4).

Figure 4. An example of how the teams were rostered for in-person and online classes.

Due to the government-imposed, social distancing requirements, the classroom at Duke-NUS was only able to accommodate half of the first-year cohort at any given time. Hence, we adopted a synchronous, hybrid TBL teaching, and a roster, for teams to attend in-person or online classes, was implemented to ensure that every student had the same number of in-person and online sessions. All students underwent live classes, held at the same time, regardless of whether they rostered to be in-person or online. If the team was rostered to have in-person classes (blue circles), students were expected to be in the classroom, unless they were under quarantine. The teams rostered to have online classes (pink circles), would take part in the lessons from home.

During this study period, which started in August of 2020, this cohort of students began their first semester of Year 1 in the online TBL environment. This was because the high number of COVID-19 cases in Singapore had resulted in heightened social distancing measures being implemented. The participants received the first survey 75 days after their first online TBL class. This was to allow sufficient time for the participants to get to know their teammates and immerse in the TBL instructional method 23 . The second survey for the online TBL classes followed 50 days later, to assess for any changes after a longer exposure in the online learning environment.

In December 2020, the COVID-19 infection clusters came under control, and social distancing measures were relaxed sufficiently for some in-person activities to resume. However, the prevailing social distancing requirements from the Government and the school meant that the classroom at Duke-NUS was only able to accommodate half of the first-year cohort at any given time. Hence, synchronous hybrid TBL classes were implemented for this cohort in semester 2, where half the cohort to attend lessons in-person, with the other half attending synchronously on Zoom. Details of how the hybrid TBL class was carried out was published elsewhere 4 . To ensure equal exposure for both halves of the cohort to in-person TBL classes, students were rostered, by teams, to come to school on alternate weeks ( Figure 4). Due to this hybrid class arrangement, participants were reminded to only take into consideration their experiences for the in-person sessions when completing the surveys (see Extended data 24 ) for the in-person learning environment. The first survey for in-person TBL classes was administered 75 days after their first hybrid TBL class, and the second survey followed 50 days later.

During the fourth and final survey, we included additional questions that explored the participants’ perception of their overall learning experiences in both online and in-person TBL, by asking them to indicate which learning environment they thought they learnt better in and why.

Scales used

At each sampling timepoint, the questionnaire measured all the learner’s characteristics (resilience, curiosity, growth mindset, use of cognitive strategy and self-regulation), basic psychological needs constructs (autonomy, perceived competence, basic needs satisfaction), situational motivation constructs, and engagement and perceived learning (see Extended data 24, 25 ). All these were validated survey instruments.

Learner’s characteristics: Students’ resilience was measured using the six-item Brief Resilience Scale 26 . Mindset of students was determined using the three-item Growth Mindset Scale 27, 28 . Curiosity of students was measured using the 5D Curiosity Scale that focuses on five-item joyous exploration 19 . Self-regulated learning of students was measured using the Motivated Strategies for Learning Questionnaire on the subscale of twelve-item cognitive use strategy and nine-item self-regulation 21 .

Basic psychological needs and situational motivation constructs: Students’ perception of the instructor’s autonomy support was measured using the six-item Learning Climate Questionnaire 13, 29, 30 . Students’ perceived competence was measured using the four-item Perceived Competence Scale 30 . Students’ needs satisfaction in general was measured using the Basic Psychological Needs Scale 31, 32 which is made up of three subscales: autonomy; competence; relatedness. The three subscales were combined to form a general needs satisfaction scale. To measure the students’ situation motivation during the learning environment, we used a sixteen-item Situational Motivation Scale which determines the level of intrinsic motivation, identified regulation, external regulation and amotivation 33 .

Outcome measurements of perception of engagement and learning: 22-item Multidimensional Engagement scale was used to measure four aspects of students’ engagement including agentic engagement, behavioural engagement, emotional engagement and cognitive engagement 34 . The nine-item CAP Perceived Learning Scale was used to measure perceived cognitive, affective and psychomotor learning gain 35 .

Quantitative data analysis

The R statistical program (R Foundation for Statistical Computing, Vienna, Austria), version 3.6.2, was used for data analyses. Data were expressed as means ±SEM and p<0.05 was considered statistically significant. As all participants completed every administered survey, two for each learning environment, we conducted a within subjects ANOVA with repeated measures to examine the differences in measured constructs in the in-person and online learning environment across the timepoints. The normality of residuals and homogeneity of variance were checked to ensure that there were no violations before the ANOVA was run. We also calculated the Cronbach’s Alpha to assess the internal consistency of each scale and used Cohen’s d to measure the effect sizes for mean differences 36 . The R packages used to run the analyses were as follows: OLS regression, the Levene’s test from the car statistical package, lmerTest statistical package, anova() function from the stats package, cohen.d() function from the effsize package and Cronbach.alpha() function from the ltm package 37, 38 .

Thematic analysis

We used thematic analysis to explore the reasons why students perceived a particular environment better for learning. We used an inductive, reflexive thematic analysis through the 6 phases for thematic analysis 39 . The author became familiarized with the data, taking casual observational notes about the content. This was followed by inductive question informed coding, at both sematic and latent levels. Themes were then constructed from these codes and their associated data, first, by establishing candidate themes, which were tested for their utility in telling the story of the data, then reviewing and finalizing themes. Two authors then discussed the content of codes and candidate themes using thematic maps to make sense of them. We generated two themes from the data describing the reasons underlying the better learning in a particular environment.

Results

Reliability statistics

Internal reliability of survey instruments was compared to previous values reported in the literature. The obtained Cronbach’s alpha values for all the survey instruments were indicated in Table 1, together with Cronbach’s alpha values that have been reported in the literature. In short, the obtained reliability statistics values in our study were comparable to the previously reported values.

Table 1. Reliability statistics of the survey instruments used in this study in comparison to previously reported values.

Measured Constructs	Cronbach’s α	Published Cronbach’s α	References	
Learner’s Characteristics	
Resilience	0.86	0.8-0.91	26	
Growth Mindset	0.94	0.94-0.98	27	
Curiosity	0.88	0.87-0.9	19	
Cognitive Strategy	0.74	0.76-0.83	41	
Self-Regulation	0.73	0.74	21	
Basic Psychological Needs	
Autonomy	0.87	0.75-0.93	13, 29	
Needs Satisfaction	0.74	0.66-0.86	42, 43	
Perceived Competence	0.81	0.8	30	
Situational Motivations	
Intrinsic Motivation	0.66-0.89	0.58-0.93	13, 33	
Identified Regulation	
External Regulation	
Amotivation	
Outcomes	
Engagement	0.76	0.78-0.94	34	
Perceived Learning	0.64	0.79	35	

Effect of learning environment on variables of the learner’s characteristics and self-determination theory framework

All variables showed homogeneity of variance. The residuals of cognitive strategy, self-regulation, identified regulation, external regulation and perceived learning did not follow normal distribution due to the presence of outliers. The removal of these outliers, as identified by Cook’s Distance 40 , showed that these observations did not have any influence on the statistical analysis model. As such, these observations remained in the analyses. Our ANOVA analysis showed that within each learning environment, in-person and online, there were no significant differences in each of the measured variables between the first and second timepoint ( Table 2). Since there were no significant differences between the early and late survey timepoints for each variable within each learning environment, they were averaged for subsequent analysis. There were also no detectable differences between male and female participants, and hence, the data were aggregated for subsequent analyses.

Table 2. Means, standard error of means for the constructs measured at the early and late timepoints in the in-person and online TBL learning environment.

Measured Constructs	In-Person 1
(Day 75)	In-Person 2
(Day 125)	Online 1
(Day 75)	Online 2
(Day 125)	
Mean	SEM	Mean	SEM	Mean	SEM	Mean	SEM	
Learner’s Characteristics	
Resilience	3.74	0.09	3.69	0.11	3.79	0.08	3.81	0.09	
Growth Mindset	4.40	0.22	4.25	0.23	4.58	0.20	4.34	0.20	
Curiosity	5.44	0.13	5.51	0.13	5.43	0.13	5.46	0.10	
Cognitive Strategy	5.03	0.09	5.08	0.10	5.05	0.10	5.04	0.08	
Self-Regulation	5.11	0.10	5.20	0.10	4.99	0.09	4.99	0.10	
Basic Psychological Needs	
Autonomy	5.32	0.11	5.28	0.12	5.18	0.13	4.89	0.10	
Needs Satisfaction	3.76	0.07	3.79	0.07	3.42	0.09	3.54	0.07	
Perceived Competence	4.61	0.09	4.67	0.10	4.32	0.12	4.37	0.10	
Situational Motivations	
Intrinsic Motivation	5.36	0.14	5.33	0.14	5.50	0.15	5.13	0.13	
Identified Regulation	5.99	0.11	5.95	0.09	5.97	0.09	5.85	0.10	
External Regulation	5.04	0.13	4.95	0.18	4.99	0.14	5.28	0.12	
Amotivation	2.46	0.17	2.43	0.17	2.28	0.16	2.51	0.15	
Outcomes	
Engagement	5.27	0.10	5.27	0.08	5.28	0.08	5.12	0.08	
Perceived Learning	3.58	0.09	3.59	0.07	3.22	0.09	3.38	0.08	

We assessed the effect of learning environment on the five variables from the learner’s characteristics framework: resilience, growth mindset, curiosity, cognitive strategy, and self-regulation ( Table 3). The ANOVA analysis of averaged responses showed no differences between in-person and online TBL classes for learner’s characteristics ( Figure 5A).

Table 3. Means and standard error of means for learner’s characteristics, basic psychological needs and situational motivation constructs, and study outcomes (engagement and perceived learning) averaged across the two timepoints.

*These measured variables showed significant differences between in-person and online learning environments (ps<0.05).

Measured Constructs	In-Person	Online	
Mean	SEM	Mean	SEM	
Learner's Characteristics	
Resilience	3.72	0.09	3.80	0.08	
Growth Mindset	4.33	0.19	4.46	0.19	
Curiosity	5.48	0.12	5.44	0.11	
Cognitive Strategy	5.05	0.08	5.05	0.08	
Self-Regulation	5.15	0.09	4.99	0.09	
Basic Psychological Needs	
Autonomy	5.3	0.10	5.04	0.10	
Needs Satisfaction *	3.77	0.06	3.48	0.07	
Perceived Competence *	4.64	0.09	4.34	0.10	
Situational Motivations	
Intrinsic Motivation	5.34	0.12	5.32	0.12	
Identified Regulation	5.97	0.08	5.91	0.08	
External Regulation	4.99	0.13	5.14	0.12	
Amotivation	2.45	0.16	2.39	0.15	
Outcomes	
Engagement	5.27	0.08	5.20	0.07	
Perceived Learning *	3.58	0.07	3.30	0.07	

Figure 5. Panels A to D. Measured constructs for in-person (pink bar) and online (blue bar) TBL learning environment. Averaged scores are shown for learner’s characteristics constructs ( Panel A ), and basic psychological needs ( Panel B ), situational motivations ( Panel C ) and measured outcomes ( Panel D ). Effect sizes (Cohen’s d) for differences between in-person and online: autonomy support, d = 0.367; needs satisfaction, d = 0.620; perceived competence, d = 0.446; perceived learning, d = 0.569. Data is represented as means and standard error of mean.

We also assessed the effect of learning environment on the basic psychological needs and situational motivation variables ( Table 3, Figure 5B and 5C). Perceived competence and needs satisfaction showed significant main effects of learning environment ( Figure 5B; p = 0.028 and p = 0.003 respectively), with in-person being higher than online. Autonomy support showed increases with in-person compared to online, although this was not significant (p = 0.073). There were no significant changes detected in the situational motivation variables: intrinsic motivation, identified regulation, external regulation and amotivation ( Figure 5C).

Of our outcome measures, engagement and perceived learning, only perceived learning showed significant main effect of learning environment ( Table 3, Figure 5D; p=0.00584), with in-person being higher than online.

Which learning environment did students feel that they learnt better in?

At the last survey timepoint, we asked students to reflect upon their class learning experience and choose which environment they thought they learnt better in. From the 49 responses, 29 picked in-person TBL, 14 did not have a preference, and 6 picked online TBL ( Figure 6A). We did not find any impact of age (21–25, >26) and gender differences on the preferred learning environment.

Figure 6. Panel A. Most of the participants perceived in-person classes to be better for learning than online classes. Panel B. A thematic analysis showed two main themes of why students perceived a particular environment to be better for learning. The community of practice supported by interpersonal interaction opportunities and the learning space played crucial roles in shaping the students’ perception of which learning environment was better for learning.

The comments reflected two themes that shaped a student’s perception of which learning environment was better for learning ( Figure 6B). Each theme cohered around a central organising concept that underpinned the thematic explanation of the data: (1) Being in the community of practice, supported by the interpersonal interaction opportunity and its associated value, influences the students’ preference of a learning environment for learning (2) How learning space directly impacts students’ ability to learn and focus.

The main theme was the community of practice, supported by the interpersonal interaction opportunity and its associated value that was offered in that learning environment.

A majority of students valued such community of practice as evident by the interpersonal interaction amongst peers which was facilitated during in-person TBL. Students perceived ease of communication, reflected by the authenticity of non-verbal cues and real-time feedback, to support team building and bonding during in-person TBL. For example, these were the comments made by participants favouring in-person TBL:

“Face-to-face TBL is more convenient and conducive way to work together with our team members and our classmates. It is easier to forge friendships and learn from one another.”

“Zoom has its limitations in team learning- it doesn't feel as "natural" and it is lacking in the nuances of non-verbal communication. I feel more engaged during face-to-face discussions, either with my team or with the faculty facilitator during teamlead sessions.”

The ability to clarify concepts either with peers or faculty during in-person TBL also shaped such perception of community of practice. Students favoured in-person TBL because of the flexibility to clarify concepts with peers face-to-face. For instance,

“…can ask for help from friends whenever we have questions or didn't catch what the lecturer said.”

“Discussion during GRA is more conducive in person where it is easier to express our thought process and interject each other”.

However, such interaction with faculty remained limited, as the faculty did not co-share the same physical learning environment with the students, given the prevailing social distancing measures on teaching venue. Thus, this was highlighted in students who found no difference between the two learning environments. Such sentiments were reflected by students who commented with the following:

“Both modalities feel rather the same to me with the main bulk of the teaching done by the professors and doctors over zoom and the location where I am receiving and learning such information does not really matter much to me”.

“I miss the moments when after class we can go up to the lecturer to ask questions freely + listening to the lecturer interacting with us answering our questions.”

Additionally, another critical sub-theme was the sense of belonging, as students perceived having common goals of working and learning together being possible only with in-person TBL.

“It is easier to concentrate when physically surrounded by classmates focusing on doing the same thing.”

“..because I like the atmosphere of doing the same thing achieving the same goal with a crowd of people.”

Most students supported in-person TBL due to the abundance of interpersonal interaction with peers. However, a small subset of students expressed a preference for online TBL, citing challenges with in-person team dynamics. This preference may be attributed to their extended exposure in the online learning environment, potentially leading to difficulties in adjusting to face-to-face discussions. For example, two students perceived being disconnected from team members and less engaged when in-person.

“Don't really feel connected to teammates… Better to focus at home instead”

“While discussing things over zoom, everyone is on the same page whereas in school, discussions will leave some members out”.

The second theme, learning space , also played crucial role in shaping students’ preference of a learning environment better for learning.

Students perceived the importance of spatial and physical space for learning, by which the campus provided that unique identity for students to stay focused, as shared by the following comments:

“…less distracted in person because the environment is encouraging you to pay attention to the most salient thing around you which is the lecture (on the projector)”.

“The school's environment is more conducive for learning, and I get into a state where I know "it's time to study".

On this note, such spatial identity for learning, in part, supported the main theme via providing the opportunity for community of practice.

“Easier to keep engaged/pay attention during physical lessons, being in a physical study space surrounded by peers and faculty is much more conducive for learning and retention.”

Similarly, some students experienced more distractions at home, negatively impacting their learning and preference, as shown below:

“At home unengaged, mixed environment cues from bedroom, distractions of home and family,”

“My home environment is not optimized for online TBL, and there are too many distractions (construction outside my house, parents also working on call, internet instability, etc) for online TBL to be effective”

On the other hand, students, who favoured online TBL, enjoyed the direct benefits of home learning environment including convenience and comfort. Students valued more time studying at home as it saved the travelling time to and from campus.

“Less time travelling = more time studying”

“…home-based learning is less distracting, allows you to study more efficiently during breaks, and allows me to use my own work desk with laptop stand, external keyboard, mouse and footstool etc. in comfort so I can study more effectively.”

While these two themes shaped and influenced how students favoured a particular learning environment, it also emphasized the critical element of satisfying the needs of being part of community of practice supported via interpersonal interaction.

Discussion

Previous studies have shown that instructional methods and environment influenced self-determination theory constructs, which in turn affected engagement and perceived learning 13, 44 . Ours was the first longitudinal study we know of that looked at whether the learner’s characteristics and SDT constructs could be impacted by the type of TBL learning environment over time.

Although self-determination theory constructs were stable across time in an online learning environment, some were lower than in-person

In this study, we showed that the learner’s characteristics and self-determination theory constructs remained unchanged across time in each learning environment ( Table 2). More importantly, these constructs remained stable across time, within the online learning environment, irrespective of the subject matter being learnt, as the participants were undergoing different courses at each survey timepoint. Our observation of the stable behaviours in the online learning environment contrasts with previous studies that showed declining learner’s motivation and engagement, and high attrition rates, in online courses using didactic instruction methods 45– 48 . As such, we posit that, in an online environment where physical interactions are lacking, the TBL instructional method would be better at satisfying the student’s basic needs for learning, compared to a didactic instructional method, such as lectures.

When comparing across different learning environments for TBL, we found that the self-determination theory constructs, such as autonomy support, perceived competence and needs satisfaction, and perceived learning were significantly higher for in-person than in online TBL classes ( Figure 5B and 5D). Our observations were in line with previous research, which showed how TBL supported learner’s engagement and perceived learning via SDT 13 . Similarly, a recent paper applied the SDT framework to demonstrate the importance of autonomy support in enhancing the engagement of K12 students in an online learning environment during the pandemic 49 . Here, we show that online TBL classes cannot support student’s needs satisfaction to the same extent as that of in-person TBL classes.

Notably, our study showed no significant changes in the levels of engagement between online or in-person TBL classes. This suggested that the TBL instructional method could maintain student engagement across time, regardless of the learning environment. Consistent with our findings, e Silva and colleagues (2021) reported widespread acceptance of fully online TBL sessions, at comparable levels to in-person TBL sessions, across various disciplines 50 . However, Shoair and colleagues (2023) found that students preferred in-person TBL and considered it superior to online TBL, which aligned with the themes identified by our thematic analysis 51 .

In terms of academic performance, Anas et al. (2022) noted that students who attended either in-person or online TBL sessions as a supplemental activity performed better in their coursework, compared to those who did not participate in these sessions 52 . However, when used as a replacement, there was no consensus on how student learning and performance were affected by online versus in-person TBL. Cleland et al. (2022) observed an increase in performances of online compared to in-person TBL in the class and end-of-year exam scores of year 2 medical students 53 . Conversely, Babenko et al. (2022) reported similar learning between online and in-person TBL in a family medicine clerkship 54 . Our results add to the growing body of literature that supports online TBL as a viable alternative when in-person classes are unavailable and highlights its potential for use in diverse educational settings.

Finally, we did not find differences in learner’s characteristics for in-person versus online TBL classes. Yet, previous studies have shown that the relationships with instructors and peers in the classroom affects the students’ resilience and in turn engagement, because these social relationships formed the support network for the student and provided a sense of competence, relatedness and autonomy 55– 57 . One possibility for the lack of differences in our study may be due to the combined effect of the use of active instructional method like TBL and the higher intrinsic motivation observed among medical students 58 . Hence, if online TBL classes are to be implemented, instructors should still be mindful and explore ways to support these learner’s characteristics.

What do these findings mean for online classes?

Taken together, our findings suggest that using TBL as the instructional method for an online class would help instructors to avoid the pitfalls that long-term online courses often face 45– 48 , because online TBL classes were able to sustain the students’ engagement over time and at levels comparable to that of in-person TBL classes. This may be because the online TBL class structure retained features, such as opportunities for feedback and collaboration 59 , that align well with the best practices in creating an autonomy-supportive classroom 10 . These features have been shown to enhance students’ self-regulation and performance 60 , and promote learning accountability and responsibility during class 61 , which in turn increases support for competence, relatedness, and autonomy 13, 42 . Additionally, these opportunities for interaction are important for creating a sense of belonging to the learning community, which is especially important in an online learning environment, where learners may experience social isolation for a long period of time 62, 63 . In contrast, didactic instructional methods reduce learner’s responsibility and interpersonal relations, and offers fewer opportunities for optimal challenges 64 .

However, we noted that online TBL classes were still unable to replace in-person TBL classes, suggesting that the ‘live’ factor cannot be fully replicated in an online setting. Whilst there was no change in the levels of engagement, we found that perceived learning was higher for in-person TBL classes than online ( Figure 5D). Consistent with these findings, in our final survey question showed that most of the students perceived in-person TBL classes to be better for their learning than online TBL classes ( Figure 6A). From our thematic analysis, there were two main themes of student’s perception of learning that further supported our quantitative findings: 1. Community of practice, 2. Learning space ( Figure 6B). Students valued the in-person opportunities because the socialization and team building opportunities contributed to being part of the community of practice. Importantly, in-person TBL promoted the sense of belonging because students could experience common goals in the same physical space. Similarly, recent studies showed a greater preference for in-person TBL than online TBL, with poorer ratings for teamwork interdependence in online TBL environment compared to in-person TBL 65, 66 . Another recent study also reported similar benefits of in-person education compared to online education 67 . Given that the sense of belonging to the community of practice is critical for professional identity formation 68, 69 , having solely online TBL classes might affect this crucial developmental process for future physicians. Furthermore, the second identified theme of learning space also interacts with the community of practice theme. This theme suggested that the spatial identity of a learning environment influenced the learner’s perceived learning. Previous studies have shown that the design of a learning space influenced engagement, motivation, professional preparation, and knowledge transfer of learners 70– 72 . Thus, it is conceivable that when a TBL class is conducted in-person, instead of remotely, there would be a significant impact on learning.

Caveats

There are two limitations to our study. First, our institute’s safe management measures implemented in response to the pandemic precluded an in-person TBL class for the full cohort of students. Hence, the participants would attend in-person TBL on a rotational basis in a synchronous hybrid TBL format. This meant that even though they were in class, with their teammates, administrative staff and facilitating instructor, some of their classmates would be online. We have tried to mitigate the impact of such hybrid TBL arrangement by ensuring that every participant experienced a similar number of in-person TBL classes before responding to the survey. Additionally, we gave explicit instructions in the survey for participants to respond based on their experience from the in-person classes. However, we cannot rule out the existence of unknown intermediate effects that may have an impact our findings. Second, our study data is obtained from self-reported surveys administered during the COVID-19 pandemic, when the stress due to the general uncertainty of the situation and social isolation could have an impact on the students’ psychological states 73 . Hence, our data may not be representative of the perception of online and in-person TBL classes during normal times.

Conclusions

Taken together, our findings suggested that while online TBL was able to maintain students’ engagement, some of the basic psychological needs constructs, such as perceived competence and needs satisfaction, and perceived learning were significantly lower than in-person TBL. Similar to previous studies, our students felt that online classes could not replace in-person classes 74 . Hence, we suggest that if classes must be held online, instructors should consider implementing the TBL instructional method, with in-person opportunities be provided where possible 4 . Considering the limitations of our study implementation and design, we recommend that future studies focus on obtaining objective measures of the SDT constructs, learner’s characteristics and learning gain, with a cleaner online and in-person intervention in a cross-over experimental design, to shed more light into supporting students in an online learning environment.

Acknowledgements

We are very grateful for invaluable advice and input from Professors Scott Compton and John Allen on our experimental design. We would like to thank all participating students for their contribution to this study.

Data availability

Underlying data

The datasets generated and analysed during the current study are not publicly available due to university policy. According to our institutional policies, data generated from any research project carried out by NUS staff or students are the property of NUS and university staff must not share data with other parties. Permission must be sought from PI, Head of Department and NUS Research Compliance and Integrity Office. Therefore, data can be made available from the corresponding author on a reasonable request and clearance with respective parties.

Extended data

Figshare: ExtendedData1_Survey for online sessions.pdf

https://doi.org/10.6084/m9.figshare.21982223.v1 25

This project contains the following extended data:

ExtendedData1_Survey for online sessions.pdf

Figshare: ExtendedData2_Final survey for in person sessions.pdf

https://doi.org/10.6084/m9.figshare.21982226.v1 24

This project contains the following extended data:

ExtendedData2_Final survey for in person sessions.pdf

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

10.21956/mep.21971.r38095
Reviewer response for version 3
Hyde Sarah 12Referee https://orcid.org/0000-0001-9858-9527

1 School of Rural Medicine, Charles Sturt University, Albury, New South Wales, Australia
2 Education, Learning, and Assessment, The Royal Australasian College of Physicians (Ringgold ID: 7796), Sydney, New South Wales, Austria
3 9 2024 Copyright: © 2024 Hyde S
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove-with-reservations
This paper provides an account of how the authors adapted their school’s TBL experience during the pandemic to an online format, and then once restrictions were lifted, to a hybrid of online and face to face. The authors have postulated that engagement and perceived learning of medical students was higher in the in-person TBL classes and used a series of surveys to measure these constructs at two time points in each learning environment, comparing results over 8-9 months.

Although the study is well intentioned and has generated interesting ideas for adapting TBL to an online format, the study design has several limitations which make the conclusions somewhat flawed. The first concern I had was that the authors use the phrase “in-person TBL” throughout and it was not until the Methods section that I learned it was in fact more of a hybrid format than “in-person”. The authors do state that they instructed the survey respondents to think about the in-person lessons only when responding, but this is a major limitation that needs more acknowledgement.

The potential for this paper to generate interest and impact lies in the messaging of the Introduction where it is stated that the findings can “help to identify key elements needed to support learners in a prolonged online learning environment and inform future integration of online learning into the existing medical curriculum”. The idea of learning space and separating this by spatial identity for learning and convenience and comfort level was novel and I would have liked to read more about this finding in Figure 6. The other significant point being made is that online TBL is a viable alternative when in-person classes are unavailable. More description of the set up of the online TBL is required though in order to make meaningful comparisons down the track with other online TBL research.

I was curious to know how the results would differ if the study was repeated in the current climate where there are no restrictions. It might be reasonable to expect that the students may have felt safer completing TBL online in the pandemic, and it was also novel, whereas now it is the ‘new norm’ in some cases. In making conclusions about online versus in-person learning, the broader context matters.

I would like to recommend the following improvements for consideration:

  Make it clearer in the abstract and introduction, and throughout the whole paper, that the second learning environment was a hybrid one – as mentioned in Figure 3. In the Discussion claims are made about the negative comparison between online and in-person when it is really about a comparison between online and hybrid environments. Some of the claims made in the Discussion and Conclusion are therefore erroneous.

The number of students in the sample is quite small, and although I am not a statistician I do wonder about the usefulness of parametric tests in this situation.

The research could have been made stronger with triangulation by comparing the student self-reports with the observations of the TBL facilitators and student performance on assessments.

The conclusions made, especially in the Abstract need to be more attenuated and circumspect as a result of all the above points. The claim that “ online TBL is unable to fully support the students’ needs and perceived learning” in the conclusion of the abstract is over stating the findings and is misleading when in fact at the time the study was conducted the online implementation of TBL did in fact meet their learning needs given the circumstances, and for students who require more flexibility in learning, such as mature-aged or those with a degree of physical or mental illness, online TBL might meet their learning needs to a greater extent than in-person.

More detail is required about the characteristics of the course at that time – for example, were there any exams in the 8 month period?

The paper could be strengthened by incorporating research and theory from the Community of Inquiry framework – see for example Garrison and Anderson’s work https://coi.athabascau.ca/coi-model/ The authors seem to be presenting a biased viewpoint of online learning rather than a more balanced argument – online engagement can in fact be quite high when the principles of CoI are implemented.

The theoretical framework presented in Figure 2 is more of a study design, consider re-labelling it as such.

In the Methods section – please describe how many questions the entire survey included altogether and how long it took to complete. Was it implemented online or paper-based or both?

Also in the Methods – I am not sure about the efficacy of combing the three subscales (autonomy, competence, relatedness) of the Basic Psychological Needs Scale into one general needs satisfaction scale. As this is the basis of the results and conclusions more information about the validity of doing this is warranted, and if no information is available about this it needs to be mentioned as a major limitation and less emphasis and confidence in the results need to be declared. Those three subscales can’t be conflated if they have already been validated in a factor analysis as separate constructs.

The fact that the ‘mixed’ component of study consists of a single open-ended question needs to be declared early on at the first mention of this being a mixed methods study. It is not really mixed methods in the true sense of the study design and is misleading to say so at the beginning.

In the Methods – describe the group composition in more detail n- was it purposefully created with a mix of gender, age, and balanced for educational background? Did the groups remain the same for the whole year or were they mixed in the hybrid situation? Did they change each term?

What was the matched response rate like? Did all students completed both surveys at each time point? Was there much attrition in the survey respondents between time 1 and time 2 in each setting?

In presenting the results of the thematic analysis, each quote needs to be identified with a participant number, as theoretically they could have all been from a single participant.

On page 13 of the Discission, in the third paragraph, it is stated that “the TBL instructional method could maintain student engagement across time, regardless of learning environment” – this is a key finding which is supported by the results. However, it does seem odd that engagement can be maintained despite needs satisfaction to being met, as claimed earlier. This also relates back to point 9 above and the doubtful efficacy of combining those subscales into one needs satisfaction scale and is what may be leading to these conflicting results. I suggest the authors rethink or get some expert advice on the usefulness of combining those subscales or revise back the claims and conclusions made in relation to the findings about them.

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Educational psychology, research methods, PBL, clinical reasoning, medical education, evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

10.21956/mep.21971.r38092
Reviewer response for version 3
Olivares Olivares Silvia Lizett 1Referee https://orcid.org/0000-0002-8311-9589

1 Tecnologico de Monterrey, Monterrey, Nuevo Leon, Mexico
12 8 2024 Copyright: © 2024 Olivares Olivares SL
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove
The article shows quantitative and qualitative evidence that students prefer face-to-face over online learning when applying TBL.  

The theoretical framework for Self-Determination Theory (SDT) was adequate selection to measure and compare learning contexts. Results and conclusions are aligned with that framework. 

However, there is more literature related to self-regulation and motivation. The authors may include other authors to complement the review.

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Medical education. Competences. Assessment

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

10.21956/mep.21646.r36010
Reviewer response for version 2
Govind Nambiar Vinod 1Referee
1 College of Medicine and Health Sciences, National University of Science & Technology, Sohar, Oman
22 2 2024 Copyright: © 2024 Govind Nambiar V
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove
This is an important study about TBL as an effective means of active teaching-learning strategy, be it an online or in-person approach. 

The authors have meticulously revised the article incorporating all the revision suggestions and the article has come out very well. This revision has given more clarity on the pre-clinical curriculum, TBL process, and the timeline of study.  Online TBL versus in-person TBL has been compared to reflect the aim of the study.

So, there are no more comments about the article and its accepted.

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Microbiology & Immunology, Infection control, Medical education, Quality Assurance in higher education

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

10.21956/mep.20929.r34741
Reviewer response for version 1
Roberts Chris 1Referee https://orcid.org/0000-0001-8613-682X

1 Medical Education, The University of Sheffield, Sheffield, England, UK
23 8 2024 Copyright: © 2024 Roberts C
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove-with-reservations
I found this article an interesting read, and there is much of merit in this evaluative research . It addresses the transition to online education during the COVID-19 pandemic, focusing on the impact of different learning environments on students' motivation, engagement, and perceived learning.   The main instructional method is TBL.

The authors use Self-Determination Theory (SDT) framework which ensures that the study's purpose, methods, and outcomes are logically connected to well established existing theory. The research questions are clearly stated, focusing on the impact of different learning environments on students' motivation, engagement, and perceived learning. The use of a mixed methods approach is a strength, as it combines quantitative and qualitative data to provide a comprehensive view of the research questions.  The results were clearly presented. The discussion summarised the key findings and reported how this research compared/added to the existing literature. There is a discussion of the implications for educators and future research. The limitations of the study are discussed.

Potential areas for strengthening include. Further develop the intro stating the problem in the research context and identifying the gap in the international research literature, and emphasising how this study addresses those challenges?

More clarity over the quasi-experimental design comparing online and in-person TBL groups. It can be a little difficult to separate the impact of the local contextual factors from the research design, and whether students could realistically only take into consideration their experiences for the in-person sessions when completing the surveys. Can the authors claim an effect rather than an association?

Provide more information about the coding process steps taken to ensure the trustworthiness and credibility of qualitative findings from the theoretical lens of SDT.

Some more info on the validity of the chosen research instruments and justification of the use of many scales in relation to the specific hypothesis that was stated and a small population of students.   

More emphasis on the key findings of this research in the opening discussion section in the light of the SDT and an integration of the qual and quant findings.

The article briefly mentioned some caveats, but it could elaborate on the methodological challenges of the study, including the sample size and potential confounding variables. Also, to take the opportunity to mention the strengths of the study.

For those schools who for various reasons are still using on line TBL and face to face is not fully possible are there implications for them?

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Assessment, instructional methods, professionalism, quant and qual methodologies

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

10.21956/mep.20929.r34740
Reviewer response for version 1
Leupen Sarah 1Referee
1 University of Maryland Baltimore County, Baltimore, Maryland, USA
25 5 2024 Copyright: © 2024 Leupen S
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationreject
In this paper the authors compared in-person and online experiences of students using a pedagogy that takes some aspects, specifically the readiness assurance process, from the Team-Based Learning pedagogical strategy. Students experienced the pedagogy online, and then experienced it in person (with some of their colleagues still online), and some aspects of their experience and performance improved when they were in person.

I have two major criticisms, and more minor comments.

Major issues:

1. This seems to be a modified form of TBL with no application questions. Most TBL practitioners would argue that the application questions are the heart of TBL, and the other stuff (RAT, etc.) simply prepares them for the application questions. The RAT itself can act as a motivation to prepare for class, so it certainly has value, but this is not TBL as described.

2. The in-person students experienced TBL more positively. But they were also just older and had more TBL, so we don’t know if it was the format, their experience in medical school, age, experience with TBL, etc. that caused the difference. To find out the answer to their question, they should simply give the survey to current first-year students who are presumably learning in person. Then you are comparing apples to apples: students in their first semester of medical school and first semester using TBL.

Related to the above, comments on the Discussion and Conclusions are as follows:

Discussion: the conclusion of the first paragraph (“In this study…”) is warranted.

The conclusion of the second paragraph (“When comparing…”) is not, since the students experiencing the in-person class are the same students later in the class/curriculum, as noted above; there is no real control, and it is likely that they were simply delighted to be back in person after COVID. This aspect could be corrected by a crossover design in which a new cohort of students have in-person followed by online (if considered ethical), or simply by comparing students in their first semester now, in person, to the previous students who experienced their first semester online, in this study. (Later, the authors recommend this themselves—great!) Also, I don’t think it’s appropriate to compare to other TBL studies, since this is only RATs, not TBL itself, which is centered on application exercises, with the RATs functioning only to prepare students for those applications. This could be corrected by either labeling this pedagogy as one that has aspects of TBL practice in it, or otherwise described according to what happened without reference to TBL, or by clarifying, if true, that there were actually application exercises that took up the bulk of class time, whether in-person or online.

It is also possible that the online TBL was not carried out as effectively as it could have been, which is perfectly understandable since we were all scrambling to do the best we could, but could mean that TBL *could* be carried out effectively online, but might not have been here. Certainly many TBL practitioners believe that TBL can be effective in fully online environments.

I agree with the authors that the relatively high level of engagement in the online class is an indication that didactic methods are inferior to active learning in this way. That is already well-established, but additional confirmation is always useful. So, the first paragraph of “What do these findings mean” is supported (except maybe the TBL label).

The findings described in the second paragraph of that section provide reasons to explore the differences in online vs. in-person TBL (or whatever this pedagogy is exactly) in a controlled way. I would look forward to reading that study.

We know that perceived learning and actual learning are often different, especially in an active learning context (see DesLauriers et al. 2017 for example), so we don’t know if those are related here, but the authors do use careful language (“it is conceivable that”). If the authors did a more controlled study of this question, they could look at results of validated exam questions, or ideally a pre-post test, to see if actual learning did differ.

Caveats: As noted, I think the major caveat is the uncontrolled nature of the study as described above in major issue #2.

Conclusions: Not bad! The authors do not recommend that TBL be used in person instead of online, which indeed we can’t tell from this, but that a cross-over study be used to gather more data, which I completely agree with. They also recommend that online instructors consider TBL, which, certainly compared to a didactic method, is indeed likely to be more effective in the measures described here.

Minor comments:

1. The introduction needs to be restructured. You have to say what TBL is, how it works, and why you think it should be better on these scales based on self-determination theory. THEN you can cite that the paper showing that it is better. But, considering this does not seem to be ‘real’ TBL, maybe it is better to talk more generally about the pedagogy, so this criticism may depend on how the authors feel about describing this as something other than TBL.

2. [very minor] In the Results: I would think the Cronbach’s alpha levels would not rise to the level of appearing in a table in the results, but that’s for the editor to decide. Otherwise, no major issues with the results reporting. I like the thematic analysis and Fig 6B.

3. I didn’t understand the sentence: “a handful of students preferred online TBL as a result of dysfunctional team dynamics presented over the initial phase of prolonged online learning environment.” I’m good up to “dynamics” but if they preferred online, wouldn’t it be because the in-person learning environment was perceived as poor?

4. Underlying data sets: I don’t care that the underlying data sets are inaccessible. We have to trust authors of papers to some degree. But I understand this is the policy of this journal, for authors to provide them.

5. Have someone, perhaps a copyeditor, check the paper to add missing articles, especially “the,” in the appropriate places, and fix all the incorrect apostrophes. Also I think this is a writing mistake: “The increasing number of reports on the lessons learnt and efforts in the online transition 1, 5 , particularly in terms of learners’ perception of the online learning environment 3 , had enabled the swift implementation of online learning.” The reports came after the implementation, not at all enabling them.

Have any limitations of the research been acknowledged?

Partly

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

team-based learning and other evidence-based pedagogies that use teams or groups; active learning broadly to some degree

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Lee Irene Duke-NUS Medical School, Singapore

5 7 2024 We thank Reviewer 3 for their comments. We have noted that the reviewer was reviewing the first version of our paper. As such, some of the comments made, especially for the major concern regarding the form of TBL, have been addressed in our second version. However, we have kept Reviewer 3’s comments in mind as we revised the manuscript for version 3. We provide a point by point response highlighted in blue below. We thank Reviewer 3 again for the thoughtful review and input to improve the paper.

I have two major criticisms, and more minor comments.

1 st Major concern:

- This seems to be a modified form of TBL with no application questions. - Also, I don’t think it’s appropriate to compare to other TBL studies, since this is only RATs, not TBL itself, which is centered on application exercises, with the RATs functioning only to prepare students for those applications.

Response: We thank Reviewer 3 for pointing this out. We had missed out on representing the application section in Version 1, which was also pointed out by other reviewers. In Version 2, and now 3, our revised Figure 2 now includes the “Knowledge Application” section. We implemented the full TBL experience throughout the whole study period. A minor note, with reviewer 3 comment about introduction, this Figure is now Figure 1.

 2 nd Major concern:

- The in-person students experienced TBL more positively. But they were also just older and had more TBL, so we don’t know if it was the format, their experience in medical school, age, experience with TBL, etc. that caused the difference. To find out the answer to their question, they should simply give the survey to current first-year students who are presumably learning in person. Then you are comparing apples to apples: students in their first semester of medical school and first semester using TBL. - The conclusion of the second paragraph (“When comparing…”) is not, since the students experiencing the in-person class are the same students later in the class/curriculum, as noted above; there is no real control, and it is likely that they were simply delighted to be back in person after COVID. Response: We appreciate the reviewer's insightful point regarding the influence of various factors (age, experience with TBl, etc.) on students' perceptions of TBL. This concern is precisely why we explored various characteristics, SDT framework, engagement and perceived learning, over an extended period to account for possible changes due to the time effect. Circling back to our study aim, to accurately determine these perceptions within different learning environments, we characterized differences in learner characteristics (such as resilience, curiosity, growth mindset, use of cognitive strategies, and self-regulation), basic psychological needs satisfaction (autonomy, competence, and relatedness), situational motivation (intrinsic, identified, external, and amotivation), student engagement, and perceived learning between the two TBL learning environments, online and in-person, over an extended period of time. To account for the potential effect of having more TBL sessions, we administered two surveys per learning environment. The first survey was conducted after 20 TBL sessions (early timepoint), and the second one was conducted 50 days later (late timepoint) for each learning environment. We did not detect any changes between the early and late timepoints within each learning environment. This stability in measures is shown in Table 2 and discussed in our conclusion: “In this study, we showed that the learner’s characteristics and self-determination theory constructs did not change across time in each learning environment (Table 2).” This suggests our measures were stable and unaffected by the number of TBL sessions, a point also supported by Reviewer 3.

 On the reviewer’s point about “simply delighted coming out of COVID,” we appreciate that this is a possible phenomenon given the social isolation and other psychosocial factors that students underwent, as shared in our discussion. However, if the observed positive effects were due to this factor alone, constructs that measure the emotional perspective on perceiving positivity, delight, and joy would likely show a significant increase in self-reported ratings. We would then expect significant differences in the measures for the Situational Motivation Scale (such as intrinsic motivation), emotional engagement, and curiosity for joyous exploration. Yet, we only observed a difference for perceived learning, with in-person TBL scoring higher than in online TBL, while situational motivations, curiosity, and engagement showed no significant differences (Table 3 and Figure 3). Our reliability statistics results also added confidence to the constructs we measured (Table 1). Consistent with this finding, we did not identify any themes from students reflecting on "simply delightedness" to come out of COVID as a reason for their preferred learning environment being in-person. Therefore, our findings suggest that the positive perception of in-person TBL is not solely due to the relief of coming out of COVID, but is likely influenced by other factors inherent to the in-person TBL experience.  

The reviewer suggested that the in-person students experience TBL more positively because they were older and had more TBL. Hence, to validate our findings, we should administer the survey the current first-year students for an “apple-to-apple comparison”.

We thank the reviewer for the suggestion of using a cross-cohort experimental design to reduce the effect of time and experience as possible confounding factors. However, we would first like to point out that if the effect of time and experience were factors that could cause students to experience in-person TBL more positively, then we should expect to see a difference between the first time-point and second time-point surveys that were administered for the in-person TBL, after all, they would have more in-person TBL experience by the time they had to fill out the second in-person survey. However, no such differences were observed (excerpt from Table 2, with measured constructs with observed differences shown below).

Measured Constructs In-Person 1 (Day 75) In-Person 2 (Day 125)

Mean SEM Mean SEM Autonomy 5.32 0.11 5.28 0.12

Needs Satisfaction 3.76 0.07 3.79 0.07

Perceived Competence 4.61 0.09 4.67 0.10

Perceived Learning 3.58 0.09 3.59 0.07

Second, it can be seen from figure 3 that the participants completed all 4 surveys in their first year of medical school.

Summary of survey timeline: 1st online survey: 3rd month 2nd online survey: 5th month 1st in-person survey: 8th month 2nd in-person survey: 10th month If the differences observed between online and in-person TBL surveys are primarily attributable to increased experience over time, we would expect to see a larger change between the 1st online survey (3rd month) and the 1st in-person survey (8th month) compared to the change between the 2nd online survey (5th month) and the 1st in-person survey (8th month). This is because the former comparison spans a longer period (5 months) than the latter (3 months). However, we did not observe such differences (Table 2). Third, comparing across cohorts may not lead to an “apple-to-apple comparison”, because the absolute values measured from surveys could reflect different baseline levels across cohorts. These differences may arise from varying prior experiences, backgrounds, and contexts, which are challenging to account for in the current study. Such variability could confound the results, making it difficult to attribute differences solely to the learning environment, thus presenting a threat to internal validity (Shadish et al., 2002). Yet, we acknowledge that our experimental design has its weaknesses. As noted in our discussion, future studies could involve a pure cross-over experimental design to shed more light on supporting students in purely online and in-person learning environments. We also recognize that there are ways to account for baseline differences in cohort comparison, such as using propensity score matching and pretest-posttest designs. If we had administered a baseline survey, we could have been able to account for these baseline differences. Unfortunately, due to the timing and constantly changing situation during the COVID pandemic, we were unable to administer the baseline survey. Hence, we opted for within-cohort comparisons to control for these differences. Using the repeated measures ANOVA across an extended period allowed us to elucidate any changes caused by the long-term immersion in either of the learning environments, within the same cohort. Our study design provided valid findings based on our reliability statistics, and we observed specific differences in measurements, as opposed to generic changes, across different learning environments. While we appreciate the reviewer’s suggest of adopting a cross-cohort design, the additional few months needed to include the upcoming student cohort may not significantly enhance the validity of our findings at this stage.

Minor concerns:

1.  If the authors did a more controlled study of this question, they could look at results of validated exam questions, or ideally a pre-post test, to see if actual learning did differ.

Response: We thank the reviewer for the suggestion and will keep it in mind for future studies. For this study, all our TBL assessments were formative. As such, we focused on characterising the student’s perceptions of online versus in-person TBL, rather than actual learning.

2. The introduction needs to be restructured. You have to say what TBL is, how it works, and why you think it should be better on these scales based on self-determination theory. THEN you can cite that the paper showing that it is better.

Response: We thank the reviewer on the feedback of the introduction. We have moved the explanation of what TBL is from the methods to the introduction. For the rest of the introduction, while we understand reviewer’s suggestion on restructuring the introduction, the current structure effectively conveys the necessary information and builds a logical argument for the study, as we start upfront with the problem/unknown, then proposed the framework that we used to characterise the  unknown, specifying the scope and questions that we aim to answer.

3. I didn’t understand the sentence: “a handful of students preferred online TBL as a result of dysfunctional team dynamics presented over the initial phase of prolonged online learning environment.” I’m good up to “dynamics” but if they preferred online, wouldn’t it be because the in-person learning environment was perceived as poor?

  Response: Referring to this specific section, we have revised it to provide better clarity to our point. Please see the highlighted portion for the revised text. Based on the thematic analysis, the first theme identified is that interpersonal interaction and team dynamics influence students' preference for a learning environment. For students who preferred online learning, their comments indicate that inherent tension in team dynamics shaped their preference. This insight goes beyond simply perceiving the in-person environment as poor and highlights the complexity of team interactions in different settings. Original Revised While most students who supported in-person TBL due to the abundance of interpersonal interaction with peers, a handful of students preferred online TBL as a result of dysfunctional team dynamics presented over the initial phase of prolonged online learning environment. For examples, these two students perceived being disconnected from team members and not being engaged in an in-person TBL, suggestive of inherent challenges with team cohesion and dynamics.     “Don't really feel connected to teammates… Better to focus at home instead” “While discussing things over zoom, everyone is on the same page whereas in school, discussions will leave some members out”.   While Most students who supported in-person TBL did so  due to the abundance of interpersonal interaction with peers. However, a small subset of students expressed a preference for online TBL, citing challenges with in-person team dynamics. This preference may be attributed to their extended exposure in the online learning environment, potentially leading to difficulties in adjusting to face-to-face discussions. For examples, two students perceived being disconnected from team members and less engaged when in-person.  

4. Have someone, perhaps a copyeditor, check the paper to add missing articles, especially “the,” in the appropriate places, and fix all the incorrect apostrophes. Also I think this is a writing mistake: “The increasing number of reports on the lessons learnt and efforts in the online transition 1 , 5 , particularly in terms of learners’ perception of the online learning environment 3 , had enabled the swift implementation of online learning.” The reports came after the implementation, not at all enabling them.

Response: We have gone through the paper to correct the grammatical errors. Additionally, we have rephrased the highlighted statement to as follows: “Due to the looming threat of infectious variants and the rapidly changing pandemic situation, many educational institutions implemented either online learning or a hybrid arrangement of blended in-person and online learning for a prolonged period of time 4 . This led to many initial reports on the lessons learnt and efforts in the online transition 1 , 5 , particularly in terms of learners’ perception of the online learning environment 3. However, it remains unclear whether being in an online learning environment over an extended period changes a learner’s motivation and in turn, the level of engagement and the perception of learning.”    

References: Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

10.21956/mep.20929.r34743
Reviewer response for version 1
Govind Nambiar Vinod 1Referee
1 College of Medicine and Health Sciences, National University of Science & Technology, Sohar, Oman
17 10 2023 Copyright: © 2023 Govind Nambiar V
2023
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove-with-reservations
1. This research on TBL is noteworthy as TBL is gaining importance as a resource-efficient, student-centered active learning and teaching strategy.

2. The authors have done good work on TBL using various validated survey instruments.

3. The paper is well written with an appropriate analysis of the findings.

4. However, some points in the paper which can be improved upon:  In the Methods section, the authors can give in few words more clarity on the 1 st year pre-clinical curriculum – duration, number of subjects/courses taught, total number of TBL classes taken semester-wise, number of assessments done, etc.(page 4)

‘In-person and online TBL methods’ – there is no mention of the “Knowledge Application step” component of the TBL process. Whether ‘instructor-facilitated discussion’ and ‘Application step’ is the same?(page 4)

Experimental design – A flowchart on study cohorts, teams, timelines, and surveys administered can give more clarity.

Figure 4 – Each team included only study participants or was there a mix of study participants and non-participants.

Results – Whether instructors were present in the same classroom to facilitate discussions?  However, it’s mentioned that – ‘such interaction with faculty remains limited as the faculty did not co-share the same physical learning environment with the students, given the prevailing social distancing measures on teaching venue. Thus, this was highlighted in students who found no difference between the two learning environments. (page 9) But on page 4, it is mentioned that ‘The administrative staff and facilitating instructors would be present in the classroom’.

Discussion – Needs to be more focused on the aims of the study: ‘to characterize differences in the two learning environments of TBL -online vs. In-person’.

However, a comparison between TBL and didactic lectures is made in some areas (page 11), but student characteristics during didactic lectures were not studied in this research.

Was TBL activity graded/assessed at the end of the semester or at the end of 1 st year? Whether this will impact student participation in the study and survey?  

The authors have clearly noted the limitations of the study and the conclusions of the study.

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Microbiology & Immunology, Infection control, Medical education, Quality Assurance in higher education

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Lee Irene Duke-NUS Medical School, Singapore

6 2 2024 We thank the reviewer for the positive comments and constructive suggestions. We have addressed the reviewer's comments and suggestions point-by-point as outlined below:  In the Methods section, the authors can give in few words more clarity on the 1 st year pre-clinical curriculum – duration, number of subjects/courses taught, total number of TBL classes taken semester-wise, number of assessments done, etc.(page 4)

Response: We thank the reviewer for highlighting this. For clarity, we have expanded the description on our preclinical curriculum in the Methods section.

  ‘In-person and online TBL methods’ – there is no mention of the “Knowledge Application step” component of the TBL process. Whether ‘instructor-facilitated discussion’ and ‘Application step’ is the same?(page 4)

Response: We thank the reviewer for pointing this out and have revised Figure 2 to include “Knowledge Application”, which is the knowledge application step of the TBL process.   Experimental design – A flowchart on study cohorts, teams, timelines, and surveys administered can give more clarity.

Response: We have revised Figure 3 to provide spatial separation between the different sets of information, such that the timeline of the study, and when the surveys are administered relative to the background of the academic semester and reopening of Singapore during the COVID pandemic can be appreciated. Additionally, as mentioned in our methods, this is a quasi-design study, with only 1 study cohort, and the teams in this cohort were illustrated in Figure 4.

  Figure 4 – Each team included only study participants or was there a mix of study participants and non-participants.

Response: The teams were formed prior to the recruitment of participants for this study, therefore there was a mix of study participants and non-participants in each team. This study was designed to have minimal disruption on the running of the classes and the students’ learning experiences, with the surveys being the main tool used to gather data, which were administered outside of class time.

  Results – Whether instructors were present in the same classroom to facilitate discussions?  However, it’s mentioned that – ‘such interaction with faculty remains limited as the faculty did not co-share the same physical learning environment with the students, given the prevailing social distancing measures on teaching venue. Thus, this was highlighted in students who found no difference between the two learning environments. (page 9) But on page 4, it is mentioned that ‘The administrative staff and facilitating instructors would be present in the classroom’.

Response: As the reviewer has rightly noted, administrative staff and facilitating instructors were indeed present in the classroom. These groups of people are different from the faculty, who take on the role of content experts in each TBL class. As most of our faculty are clinical faculty, during COVID, due to the cohorting and other safe-distancing measures, the clinical faculty could not come to the campus, hence, they were not present in the same classroom as the students. This might be the reason why some students perceived no difference between the two learning environments. Discussion – Needs to be more focused on the aims of the study: ‘to characterize differences in the two learning environments of TBL -online vs. In-person’.

However, a comparison between TBL and didactic lectures is made in some areas (page 11), but student characteristics during didactic lectures were not studied in this research.

Response: We thank the reviewer for this suggestion and have revised the discussion to redirect the focus to online and in-person TBL. We have also added latest references to discuss such differences.    Was TBL activity graded/assessed at the end of the semester or at the end of 1 st year? Whether this will impact student participation in the study and survey?  

          Response: TBL assessments were all formative. As such, we do not expect any impact of grades on student participation in the study and survey. All students, whether study participants or not, had access to the same learning resources throughout the study.

10.21956/mep.20929.r34200
Reviewer response for version 1
Malik Alam Sher 1Referee
1 Management and Science University, Shah Alam, Selangor, Malaysia
24 8 2023 Copyright: © 2023 Malik AS
2023
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
recommendationapprove-with-reservations
This paper addresses a very important aspect in relation to teaching/learning methods and approaches.

Authors have worked meticulously to get the information through the surveys and analysis thereafter.

It is amazing to note that there was no drop-out among the participants during the study.

Sections on “Introduction” and “Results” have been very well written.

However, there are following concerns: The TBL process seems to be truncated as shown in figure 2. There is no mention of “Team Application” step in the process. Although in the text it is mentioned as follows:

  “Students would be put in breakout rooms when they are doing the team readiness assurance tests and working through the clarification questions”. (Page 4).

It is not clear, if it is the same cohort of students in both for online and in-person TBL classes.

In figure 3 two different dates are mentioned in the diagram and text below it i.e. 27 th July and 20 th August 2020.

First survey was conducted after twenty TBL classes. How many sessions were held before conducting the second survey?

In discussion section a comparison has been drawn between TBL and other methods of teaching e.g., didactic lectures – which is not the states focus of this paper.

Have any limitations of the research been acknowledged?

Yes

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Medical education; Paediatric Medicine

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Lee Irene Duke-NUS Medical School, Singapore

6 2 2024 We thank the reviewer for the positive comments and constructive suggestions. We have addressed the reviewer’s comments and suggestions point-by-point as outlined below in blue.   Response to Reviewer 1 The TBL process seems to be truncated as shown in figure 2. There is no mention of “Team Application” step in the process.

Response: We thank the reviewer for pointing this out and have revised Figure 2 to include “Knowledge Application”. It is not clear, if it is the same cohort of students in both for online and in-person TBL classes.

Response: It is the same cohort of students, as can be seen from Figure 3, where the study took place between 2020 to 2021, spanning 2 academic semesters within the same Academic Year 2020/2021. Also, in the text “During this study period, the participants started their Year 1, semester 1, in the online TBL environment, due to the heightened social distancing measures during the COVID-19 pandemic. In semester 2, as the infection clusters came under control, social distancing measures were relaxed sufficiently for half the cohort to attend lessons in-person, with the other half attending synchronously on Zoom.” In figure 3 two different dates are mentioned in the diagram and text below it i.e. 27 th July and 20 th August 2020.

Response: The revised Figure 3 now clearly delineates the timeline of Singapore's reopening during the COVID-19 pandemic, the University's COVID-19 measures, the academic semester at Duke-NUS, and our study timeline. Specifically, it shows that Singapore began its reopening phase on 27th July 2020, as indicated by the grey text beneath this date, allowing small group gatherings. Furthermore, the figure highlights that the first semester at Duke-NUS commenced in August 2020. This spatial separation in the figure should aid in distinguishing these key timelines.

  First survey was conducted after twenty TBL classes. How many sessions were held before conducting the second survey?

Response: We conducted the first survey following twenty Team-Based Learning (TBL) sessions. This timing was chosen to ensure participants had ample opportunity to familiarize themselves with their teammates and fully engage with the TBL method. The second survey was administered 50 days later to assess the impact of being immersed in a learning environment for a prolonged period. In the second semester, despite a change in the learning environment, we maintained the same schedule for the first survey and the same 50-day interval before conducting the second survey. In discussion section a comparison has been drawn between TBL and other methods of teaching e.g., didactic lectures – which is not the states focus of this paper.

Response: We thank the reviewer for this suggestion and have revised the discussion to redirect the focus to online and in-person TBL.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.
==== Refs
1 Wong P Jumat MR Lee ICJ : Redesigning team-based learning facilitation for an online platform to deliver preclinical curriculum: a response to the COVID-19 pandemic [version 1]. MedEdPublish. 2020;9 (1 ):135. 10.15694/mep.2020.000135.1 38073806
2 Jumat MR Wong P Foo KX : From trial to implementation, bringing Team-Based Learning Online-Duke-NUS medical school's response to the COVID-19 pandemic. Med Sci Educ. 2020;30 (4 ):1649–1654. 10.1007/s40670-020-01039-3 32837796
3 Daniel M Gordon M Patricio M : An update on developments in medical education in response to the COVID-19 pandemic: a BEME scoping review: BEME guide no. 64. Med Teach. 2021;43 (3 ):253–271. 10.1080/0142159X.2020.1864310 33496628
4 Lee ICJ Wong P Goh SPL : A synchronous hybrid Team-Based Learning class: why and how to do it? Med Sci Educ. 2022;32 (3 ):697–702. 10.1007/s40670-022-01538-5 35493983
5 Binks AP LeClair RJ Willey JM : Changing medical education, overnight: the curricular response to COVID-19 of nine medical schools. Teach Learn Med. 2021;33 (3 ):334–342. 10.1080/10401334.2021.1891543 33706632
6 Ryan RM Deci EL : Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol. 2000;25 (1 ):54–67. 10.1006/ceps.1999.1020 10620381
7 Martela F Ryan RM : The benefits of benevolence: basic psychological needs, beneficence, and the enhancement of well-being. J Pers. 2016;84 (6 ):750–64. 10.1111/jopy.12215 26249135
8 Milyavskaya M Philippe FL Koestner R : Psychological need satisfaction across levels of experience: their organization and contribution to general well-being. J Res Pers. 2013;47 (1 ):41–51. 10.1016/j.jrp.2012.10.013
9 Raj SJM Chettiar C : Need satisfaction, goal content and subjective well-being. Indian J Posit Psychol. 2012;3 (1 ):10–13. Reference Source
10 Kusurkar RA Croiset G Ten Cate TJ : Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory. Med Teach. 2011;33 (12 ):978–82. 10.3109/0142159X.2011.599896 22225435
11 Ryan RM Deci EL : Overview of self-determination theory: an organismic dialectical perspective. Handbook of Self-Determination Research. 2002;2 :3–33. Reference Source
12 Hsu HCK Wang CV Levesque-Bristol C : Reexamining the impact of self-determination theory on learning outcomes in the online learning environment. Educ Inf Technol. 2019;24 (3 ):2159–2174. 10.1007/s10639-019-09863-w
13 Jeno LM Raaheim A Kristensen SM : The relative effect of Team-Based Learning on motivation and learning: a Self-Determination Theory perspective. CBE Life Sci Educ. 2017;16 (4 ):ar59. 10.1187/cbe.17-03-0055 29146665
14 Ng B : The neuroscience of growth mindset and intrinsic motivation. Brain Sci. 2018;8 (2 ):20. 10.3390/brainsci8020020 29373496
15 Neufeld A Malin G : Exploring the relationship between medical student basic psychological need satisfaction, resilience, and well-being: a quantitative study. BMC Med Educ. 2019;19 (1 ): 405. 10.1186/s12909-019-1847-9 31690292
16 Schutte NS Malouff JM : Increasing curiosity through autonomy of choice. Motiv Emot. 2019;43 (4 ):563–570. 10.1007/s11031-019-09758-w
17 Zhao Y Niu G Hou H : From growth mindset to grit in Chinese schools: the mediating roles of learning motivations. Front Psychol. 2018;9 :2007. 10.3389/fpsyg.2018.02007 30405492
18 Beswick D : Cognitive process theory of individual differences in curiosity. Intrinsic motivation: A new direction in education. 1971;156–70.
19 Kashdan TB Stiksma MC Disabato DJ : The five-dimensional curiosity scale: capturing the bandwidth of curiosity and identifying four unique subgroups of curious people. J Res Pers. 2018;73 :130–49. 10.1016/j.jrp.2017.11.011
20 Litman J : Curiosity and the pleasures of learning: wanting and liking new information. Cogn Emot. 2005;19 (6 ):793–814. 10.1080/02699930541000101
21 Pintrich PR De Groot EV : Motivational and self-regulated learning components of classroom academic performance. J Educ Psychol. 1990;82 (1 ):33–40. 10.1037/0022-0663.82.1.33
22 Parmelee D Michaelsen LK Cook S : Team-based learning: a practical guide: AMEE guide no. 65. Med Teach. 2012;34 (5 ):e275–87. 10.3109/0142159X.2012.651179 22471941
23 Michaelsen LK Knight AB Fink LD : Team-based learning: a transformative use of small groups in college teaching. 2004. Reference Source
24 Lee I : ExtendedData2_Final survey for in person sessions.pdf. figshare. Journal contribution,2023. 10.6084/m9.figshare.21982226.v1
25 Lee I : ExtendedData1_Survey for online sessions.pdf. figshare. Journal contribution,2023. 10.6084/m9.figshare.21982223.v1
26 Smith BW Dalen J Wiggins K : The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15 (3 ):194–200. 10.1080/10705500802222972 18696313
27 Dweck CS Chiu CY Hong YY : Implicit theories and their role in judgments and reactions: a word from two perspectives. Psychol Inq. 1995;6 (4 ):267–285. 10.1207/s15327965pli0604_1
28 Dweck CS : Self-theories: their role in motivation, personality, and development.Psychology press, 2013. Reference Source
29 Black AE Deci EL : The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: a self-determination theory perspective. Sci Educ. 2000;84 (6 ):740–756. 10.1002/1098-237X(200011)84:6<740::AID-SCE4>3.0.CO;2-3
30 Williams GC Deci EL : Internalization of biopsychosocial values by medical students: a test of self-determination theory. J Pers Soc Psychol. 1996;70 (4 ):767–79. 10.1037//0022-3514.70.4.767 8636897
31 Deci EL Ryan RM : The "What" and "Why" of goal pursuits: human needs and the self-determination of behavior. Psychol Inq. 2000;11 (4 ):227–268. 10.1207/S15327965PLI1104_01
32 Gagné M : The role of autonomy support and autonomy orientation in prosocial behavior engagement. Motiv Emot. 2003;27 (3 ):199–223. 10.1023/A:1025007614869
33 Guay F Vallerand RJ Blanchard C : On the assessment of situational intrinsic and extrinsic motivation: the Situational Motivation Scale (SIMS). Motiv Emot. 2000;24 (3 ):175–213. 10.1023/A:1005614228250
34 Reeve J Tseng CM : Agency as a fourth aspect of students' engagement during learning activities. Contemp Educ Psychol. 2011;36 (4 ):257–267. 10.1016/j.cedpsych.2011.05.002
35 Rovai AP Wighting MJ Baker JD : Development of an instrument to measure perceived cognitive, affective, and psychomotor learning in traditional and virtual classroom higher education settings. Internet High Educ. 2009;12 (1 ):7–13. 10.1016/j.iheduc.2008.10.002
36 Cohen J : Statistical power analysis for the behavioral Sciences. Hillsdale (NJ): Lawrence Erlbaum Associates,1988;18–74. Reference Source
37 Bates D Mächler M Bolker B : Fitting linear mixed-effects models using lme4.arXiv preprint arXiv: 14065823.2014. 10.48550/arXiv.1406.5823
38 Kuznetsova A Brockhoff PB Christensen RH : lmerTest package: tests in linear mixed effects models. J Stat Softw. 2017;82 (13 ):1–26. 10.18637/JSS.V082.I13
39 Braun V Clarke V : Thematic analysis. 2012. 10.1037/13620-004
40 Cook RD : Detection of influential observation in linear regression. Technometrics. 1977;19 (1 ):15–8. 10.1080/00401706.1977.10489493
41 Erturan Ilker G Arslan Y Demirhan G : A validity and reliability study of the motivated strategies for learning questionnaire. Educ Sci: Theor Pract. 2014;14 (3 ):829–33. Reference Source
42 Jeno LM Diseth Å : A self-determination theory perspective on autonomy support, autonomous self-regulation, and perceived school performance. Reflecting Education. 2014;9 (1 ):1–20. Reference Source
43 Ntoumanis N : A prospective study of participation in optional school physical education using a self-determination theory framework. J Educ Psychol. 2005;97 (3 ):444–453. 10.1037/0022-0663.97.3.444
44 Jang H Kim EJ Reeve J : Why students become more engaged or more disengaged during the semester: a self-determination theory dual-process model. Eur Int. 2016;43 :27–38. 10.1016/j.learninstruc.2016.01.002
45 Hew KF Cheung WS : Students' and instructors' use of Massive Open Online Courses (MOOCs): motivations and challenges. Educ Res Rev. 2014;12 :45–58. 10.1016/j.edurev.2014.05.001
46 Kim KJ Frick TW : Changes in student motivation during online learning. J Educ Comput Res. 2011;44 (1 ):1–23. 10.2190/EC.44.1.a
47 Schaeffer CE Konetes GD : Impact of learner engagement on attrition rates and student success in online learning. International Journal of Instructional Technology & Distance Learning. 2010;7 (5 ):3–9. Reference Source
48 Angelino LM Williams FK Natvig D : Strategies to engage online students and reduce attrition rates. J Educ Online. 2007;4 (2 ):n2. 10.9743/JEO.2007.2.1
49 Chiu TKF : Applying the Self-Determination Theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic. J Res Technol Educ. 2022;54 (sup 1 ):S14–S30. 10.1080/15391523.2021.1891998
50 Silva ECE Lino-Neto T Ribeiro E : Going virtual and going wide: comparing Team-Based Learning in-class versus online and across disciplines. Educ Inf Technol (Dordr). 2022;27 (2 ):2311–2329. 10.1007/s10639-021-10683-0 34421327
51 Shoair OA Smith WJ Abdel Aziz MH : Pharmacy students’ perceptions and attitudes toward face-to-face vs. virtual Team-Based Learning (TBL) in the didactic curriculum: a mixed-methods study. Med Educ Online. 2023;28 (1 ): 2226851. 10.1080/10872981.2023.2226851 37343668
52 Anas S Kyrou I Rand-Weaver M : The effect of online and in-person Team-Based Learning (TBL) on undergraduate endocrinology teaching during COVID-19 pandemic. BMC Med Educ. 2022;22 (1 ): 120. 10.1186/s12909-022-03173-5 35193577
53 Cleland JA Ng KB Rotgans JI : Classroom versus online Team-Based Learning: effects on students’ learning and performance. Health Professions Education. 2022;8 (1 ):9–16. Reference Source
54 Babenko O Ding M Lee AS : In-Person or online? The effect of delivery mode on Team-Based Learning of Clinical reasoning in a family medicine clerkship. Med Sci (Basel). 2022;10 (3 ):41. 10.3390/medsci10030041 35997333
55 Furrer CJ Skinner EA Pitzer JR : The influence of teacher and peer relationships on students' classroom engagement and everyday motivational resilience. National Society for the Study of Education. 2014;116 (13 ):101–23. 10.1177/016146811411601319
56 Fernández-Martínez E Andina-Díaz E Fernández-Peña R : Social networks, engagement and resilience in university students. Int J Environ Res Public Health. 2017;14 (12 ): 1488. 10.3390/ijerph14121488 29194361
57 Skinner EA Pitzer JR : Developmental dynamics of student engagement, coping, and everyday resilience. Handbook of research on student engagement. Springer,2012;21–44. 10.1007/978-1-4614-2018-7_2
58 Lee SS Samarasekera DD Sim JH : Exploring the cultivation of Self-Regulated Learning (SRL) strategies among pre-clinical medical students in two medical schools. Med Sci Educ. 2020;30 (1 ):271–280. 10.1007/s40670-019-00894-z 34457667
59 Burgess A van Diggele C Roberts C : Team-Based Learning: design, facilitation and participation. BMC Med Educ. 2020;20 (Suppl 2 ): 461. 10.1186/s12909-020-02287-y 33272267
60 Wang JCK Ng BLL Liu WC : Can being autonomy-supportive in teaching improve students' self-regulation and performance? Building autonomous learners. Springer,2016;227–43. 10.1007/978-981-287-630-0_12
61 Michaelsen LK Sweet M : The essential elements of Team-Based Learning. New directions for teaching and learning. 2008;2008 (116 ):7–27. 10.1002/tl.330
62 McInnerney JM Roberts TS : Online learning: social interaction and the creation of a sense of community. J Educ Techno Soc. 2004;7 (3 ):73–81. Reference Source
63 Wong J Baars M Davis D : Supporting Self-Regulated Learning in online learning environments and MOOCs: a systematic review. Int J Hum Comput Interact. 2019;35 (4–5 ):356–73. 10.1080/10447318.2018.1543084
64 Chang R Fukuda E Durham J : Enhancing students' motivation with autonomy-supportive classrooms. Development of self-determination through the life-course. Springer; 2017;99–110. 10.1007/978-94-024-1042-6_8
65 Franklin AS Markowsky S De Leo J : Using Team-Based Learning to teach a hybrid pharmacokinetics course online and in class. Am J Pharm Educ. 2016;80 (10 ): 171. 10.5688/ajpe8010171 28179720
66 DeMasi J Harvan RA Luca M : Online and in-class Team-Based Learning in undergraduate immunology: a comparative analysis. Med Sci Educ. 2019;29 (4 ):1193–1199. 10.1007/s40670-019-00814-1 34457603
67 Nikolopoulou K : Face-to-face, online and hybrid education: university students' opinions and preferences. J Digit Educ Technol. 2022;2 (2 ): ep2206. 10.30935/jdet/12384
68 Lave J Wenger E : Legitimate peripheral participation. Situated learning: Legitimate peripheral participation 1991;29–43. 10.1017/CBO9780511815355.003
69 Cruess SR Cruess RL Steinert Y : Supporting the development of a professional identity: general principles. Med Teach. 2019;41 (6 ):641–649. 10.1080/0142159X.2018.1536260 30739517
70 Sasson I Malkinson N Oria T : A constructivist redesigning of the learning space: the development of a sense of class cohesion. Learn Environ Res. 2021;25 :1–15. 10.1007/s10984-021-09356-z
71 Adedokun OA Parker LC Henke JN : Student perceptions of a 21st century learning space. J Learn Spac. 2017;6 (1 ). Reference Source
72 Donkin R Kynn M : Does the learning space matter? An evaluation of active learning in a purpose-built technology-rich collaboration studio. Australas J Educ Tec. 2021;37 (1 ):133–46. 10.14742/ajet.5872
73 Lyons Z Wilcox H Leung L : COVID-19 and the mental well-being of Australian medical students: impact, concerns and coping strategies used. Australas Psychiatry. 2020;28 (6 ):649–652. 10.1177/1039856220947945 32772729
74 Phillips JA : Replacing traditional live lectures with online learning modules: effects on learning and student perceptions. Curr Pharm Teach Learn. 2015;7 (6 ):738–44. 10.1016/j.cptl.2015.08.009
