
==== Front
Acta Med Philipp
Acta Med Philipp
AMP
Acta Medica Philippina
0001-6071
2094-9278
University of the Philippines Manila

AMP-58-15-7996
10.47895/amp.vi0.7996
Original Article
Survey of Students and Alumni of Clinical Epidemiology Graduate Programs in the Philippines: A Descriptive Cross-sectional Study of Program Strengths and Weaknesses
Cordero Cynthia P. MSc PH, MMedStat 1
Tan-Lim Carol Stephanie C. MD, MSc 1
Panelo Carlo Irwin A. MD, MA 1
Cabaluna Ian Theodore G. MD, Dip (Clinical Epidemiology) 2
Monis Girlie C. BSc 1
Famador Paul Erich R. IT-CSS 1
1 Department of Clinical Epidemiology, College of Medicine, University of the Philippines Manila
2 Institute of Clinical Epidemiology, National Institutes of Health, University of the Philippines Manila
Corresponding author: Carol Stephanie C. Tan-Lim, MD, MSc, Department of Clinical Epidemiology, College of Medicine, University of the Philippines Manila, 547 Pedro Gil St., Ermita, Manila 1000, Philippines. Email: cctan7@up.edu.ph. ORCiD: https://orcid.org/0000-0001-8815-4191
30 8 2024
2024
58 15 1123
© 2024 Acta Medica Philippina
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Articles are published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
Background and Objectives

The Department of Clinical Epidemiology (DCE) of the University of the Philippines Manila is the only higher education institution in the Philippines offering graduate studies in clinical epidemiology. The Master of Science Epidemiology (Clinical Epidemiology) was first offered in 1992, while the Diploma in Epidemiology (Clinical Epidemiology) was offered in 1998. While the courses of the programs are continuously updated based on students’ feedback and advances on topics covered, the point of view of the students and alumni on the program as a whole has not been done. This study aimed to determine 1) self-reported current positions and affiliations, work areas where clinical epidemiology (CE) training is useful, and skills gained from CE training; 2) research studies completed and deemed by respondents to have considerable impact; and 3) strengths, weaknesses, and areas of improvement of the DCE graduate programs.

Methods

This is an online survey of students and alumni of the DCE graduate programs. We sent email invites to all 287 students and alumni. We collected data on their profession, institutional affiliations, positions, skills gained from their training, areas of clinical epidemiology applications, important research involvement, reasons for recommending or not recommending the programs, and how the graduate programs can be improved. Responses were summarized by frequencies and percentages. An analyst performed qualitative content analysis (QCA) to generate strengths and weaknesses of the program. We validated the results of the QCA through 1) presentation to the research team, 2) sending the survey report to study participants and other students and alumni for feedback, and 3) presentation to the DCE faculty and staff.

Results

We received 159 responses (55.4% of the total study population)—145 (91.2%) were from the MSc program and 11 (6.9%) were from the Diploma program. Majority of the respondents were physicians (93.7%), had hospital affiliations (81.8%), and were affiliated with the academe (61%). Majority of the respondents used clinical epidemiology in their research endeavors (87.4%), clinical practice (85.5%), and teaching (78%). Majority (93.1%) would recommend the program they have taken. Eleven (6.9%) respondents were hesitant due to the possible mismatch with the students’ career path, challenging thesis work, and potential conflicting personal responsibilities. Several strengths of the programs were identified, including excellent and well-implemented programs, supportive faculty and staff, and relevant course work. While completing the course work had not been a problem in general, the main challenge encountered by students is the completion of their thesis, leading to a low graduation rate in the Master of Science program. Suggestions to improve the Master of Science and Diploma programs include 1) Improvement in program implementation, including thesis policies and support, smoother transition from Diploma to MSc Program and vice-versa, and implementation of a blended learning platform; 2) Curricular improvements such as wider choices for electives and tracking towards specialty areas; 3) Innovations in conduct of courses; and 4) Personnel and infrastructure development.

Conclusions

This survey reiterated the importance of clinical epidemiology graduate programs in research capacity building of health care professionals. Students and alumni occupied diverse positions in academic, research, clinical, and pharmaceutical setting, and majority accomplished research studies with considerable impact. A major challenge leading to a low graduation rate in the Master of Science program is the completion of thesis work. The survey identified several initiatives towards continuous quality improvement of clinical epidemiology programs, including improvement of thesis policies and support, updating the curriculum content and materials, increasing allotment of hours for hands-on activities, exploring possibilities of offering electives in partnership with other institutions, offering a blended learning platform, maintaining an efficient administrative support for students, and continuing education for alumni. Strong institutional support for personnel and infrastructure development is essential for these initiatives to succeed.

research capacity building
clinical epidemiology
graduate programs
==== Body
pmcINTRODUCTION

The Department of Clinical Epidemiology (DCE) of the University of the Philippines (UP) Manila offers two graduate programs—the Master of Science (MSc) and Diploma programs. The MSc Epidemiology (Clinical Epidemiology) program is on its 30th year. As of 2022, 269 students have been accepted in the MSc program of which 99 (46.5%, 99 out of 213 students) have graduated and 56 are currently enrolled. The Diploma in Epidemiology (Clinical Epidemiology) is on its 23rd year. As of 2022, 22 students have been accepted of which 19 students (86.4%) have graduated while 3 (13.6%) dropped from the program. Considering both programs, the percentage of graduation is 50.2% (118 out of 235 students).

The DCE graduate programs are two of the clinical epidemiology graduate programs offered by the Clinical Epidemiology Resource and Training Centers (CERTCs) of the International Clinical Epidemiology Network (INCLEN).1 INCLEN is a network of Clinical Epidemiology Units (CEUs) and CERTCs in 89 academic institutions and 34 countries. It was established through the sponsorship of the Rockefeller Foundation, DCE started as a CEU in 1983 and was designated as an INCLEN CERTC in 1992 when the MSc Epidemiology (Clinical Epidemiology) program was launched. In 1999, DCE became one of the basic departments of the UP College of Medicine.

There are two tracks in the UP Manila MSc Epidemiology umbrella program, the clinical epidemiology track offered by DCE and the public health track of the College of Public Health (CPH). In the original program, students of both tracks take 11 units of core courses (5 units of biostatistics, 4 units of epidemiology at the CPH, and 2 units of fundamentals of clinical economics, health social science, and ethics at the DCE). In addition, the DCE program requires 9 units of major courses, 6 units of elective subjects, and 6 units of thesis. The program has undergone several internal evaluations which resulted in streamlining of the course syllabi and materials. In 2002, DCE commissioned the National Teachers Training Center for the Health Professions (NTTCHP) to conduct an external evaluation resulting in curricular revisions in 2005.2 Minor revisions were also instituted in 2008.

In the current program, there are 2 core courses (biostatistics courses) with a total credit of 5 units, 10 major courses with a total credit of 19 units, and 7 electives among which students choose 2 courses (4 units). The current electives offered include courses in statistics, clinical economics, informatics, medical writing, and social science. A Diploma student must complete 28 units to graduate, while an MSc student must complete 34 units (including 6 units of thesis) to graduate.

The current programs may be completed by students as full-time or part-time. Full-time students in the MSc and Diploma programs complete their coursework in one year (2 semesters), while part-time students complete their coursework in two years (4 semesters). MSc students also need to complete their thesis work within five years from enrollment in the program.

For the first 10 years of the program, students were recruited intentionally from medical schools, including De La Salle University, University of Santo Tomas, and Cebu Institute of Medicine. Other applicants with institutional affiliations were also prioritized. The aim was to establish a critical mass of clinical epidemiologists and eventually establish a CEU in these institutions. Building a critical mass of clinical epidemiologists is recognized by the World Health Organization as an essential element to strengthen national capacity for prevention and control of various diseases.3

Students were previously offered the Rockefeller Foundation’s scholarship which included funding for their thesis. Students could also avail of scholarships from the Philippine Council for Health Research and Development. Over the years, the available scholarships dwindled but the number of applicants remained high at 9 to 15 per school year. Many are self-supporting.

While the courses/subjects of the programs are continuously updated through the initiatives of the course coordinators based on students’ feedback, no program evaluation from the point of view of the students and alumni has been done since the 2002 NTTC-HP evaluation. Studies have shown that students play a crucial role in innovating and expanding the curricula of graduate programs.4-6 The alumni can also provide valuable feedback on the usefulness of the graduate programs on their career paths after graduation.4

Several advancements in the field of clinical epidemiology occurred in the recent years, including the development of new methodologies such as the network meta-analysis, and advancements in data science, genomics, transcriptomics, and other health technologies. Moreover, the COVID-19 pandemic and the passage of the Universal Health Care (UHC) Act in 2019 highlighted the critical role of clinical epidemiology in responding to health issues in the country, including the timely development of clinical practice guidelines and rapid reviews. In line with DCE’s goal towards continuous quality improvement of its graduate programs that is responsive to international and national needs, and with the need of UP Manila to comply with accreditation requirements through evaluation of its programs to ensure quality assurance and internationalization, we conducted a study to evaluate the current DCE programs from the perspective of the students and alumni. This study aimed to determine: 1) self-reported current positions and affiliations, work areas where clinical epidemiology (CE) training is useful, and skills gained from CE training; 2) research studies completed and deemed by respondents to have considerable impact; and 3) strengths, weaknesses, and areas of improvement of the programs. The results can inform the proposed curricular revisions for the MSc and the Diploma programs. It can also be used for the development of a new graduate program, the MD-Master of Clinical Epidemiology (MD-MCE) two-track program.

METHODS

Research design

This descriptive cross-sectional study is an online survey of students and alumni of the two graduate programs, the MSc Epidemiology (CE) and the Diploma in Epidemiology (CE). We crafted the survey questionnaire using Google form based on the objectives of the survey. We pre-tested the form among selected alumni of the programs. We revised the questionnaire according to the pre-test results, and used the revised questionnaire for the online survey.

Study population

We sent invitations to participate in the survey through electronic mail (e-mail) to all students and alumni of the DCE graduate programs. The inclusion criteria included all students who were accepted in the DCE graduate programs regardless if they were able to complete the programs or not. Students who took courses in the DCE but were enrolled in other graduate programs were excluded.

On its 30th year of implementation, DCE has accepted 291 students, including 4 students from China and 2 from Indonesia. On the average, DCE receives 15 applicants and accepts 10 students per batch (Figure 1). The smallest batch consisted of 4 students and the largest batch consisted of 17 students. Applications increased especially during the COVID-19 pandemic. DCE accepted all 17 applicants in 2020 and 13 students out of 22 applicants in 2021. For the school year, 2022-2023, DCE accepted 17 students (13 and 4 for the MSc and Diploma programs, respectively) out of 29 applicants.

Figure 1 Number of applicants and accepted students per batch.

This survey used total enumeration method. At the time of the survey, it has come to our knowledge that four students are deceased. We included all 287 students who were accepted in the past 30 years (1992-2021).

In the invitation, we emphasized that 1) participation is voluntary, 2) only de-identified data (demographic information and quotations) will be included in the report, 3) it will not be possible to identify individual respondents in the report, including the published paper, 4) the survey aims to evaluate the strengths, limitations, and areas of improvement of the two graduate programs of the DCE towards continuous quality improvement. After the first round of responses, we sent an e-mail to the survey respondents requesting them to encourage their batchmates to participate in the survey. After the second round of responses, we sent an e-mail to the remaining students who did not participate with a personal request for his/her participation. We also contacted non-responders by SMS and social media networks. After several weeks of no additional response, we closed the survey. The survey was conducted from March 16, 2022 to August 17, 2022 remotely from the DCE Office.

Responses were summarized by frequencies and percentages. For qualitative data, an analyst performed Qualitative Content Analysis. Responses were categorized into common themes. The themes that emerged during the analysis were presented, with selected verbatim responses to provide further details for each theme. We validated the results of the QCA through 1) presentation to the research team, 2) sending the survey report to study participants and other students and alumni for feedback, and 3) presentation to the DCE faculty and staff. Missing data was recorded as “no response given”.

The completed study was registered at the UP-Manila Research Grants and Administration Office and was submitted to the UP-Manila Research Ethics Board. The latter certified that the protocol of the study qualifies for exemption for ethical review.

RESULTS

Study participants

We received 159 responses (55.4% out of the target 287 students) from March 16, 2022 to August 17, 2022. Of the 159 respondents, 145 (91.2%) were from the MSc program and 11 (6.9%) were from the Diploma program (Figure 2). There were 3 respondents (1.9%) who graduated with a Diploma degree and are currently enrolled in the MSc program. Majority of the respondents (149 or 93.7%) were physicians (Table 1).

Table 1 Distribution of Respondents according to Profession (n=159)

Profession	Number	Percent	
Physician	142	89.3	
Allied medical professional	5	3.1	
Nurse	4	2.5	
Pharmacist	1	0.6	
Physician, Pharmacist	2	1.3	
Physician, Medical Technologist	1	0.6	
Physician, Nurse	1	0.6	
Physician, Nutritionist-Dietitian	1	0.6	
Physician, and allied medical profession	1	0.6	
Physician, Research Consultant	1	0.6	

Figure 2 Flow diagram of survey participation.

Of the 149 physicians, 146 underwent specialty training (Table 2). The most common specialty of respondents was Internal Medicine (33.6%), followed by Pediatrics (21.2%). There were 111 physicians who underwent subspecialty/fellowship training, with a wide range of disciplines such as genetics, clinical immunology, trauma surgery, and regional anesthesia.

Table 2 Distribution of Physician Respondents according to Specialty Training (n=146)

Residency Training	Number	Percent	
Internal Medicine	49	33.6	
Pediatrics	31	21.2	
Obstetrics and Gynecology	11	7.5	
Surgery	11	7.5	
Family and Community Medicine	7	4.8	
Anesthesiology	6	4.1	
Dermatology	6	4.1	
Neurology	6	4.1	
Ophthalmology	5	3.4	
Orthopedics	5	3.4	
Otorhinolaryngology	3	2.1	
Emergency Medicine	2	1.4	
Psychiatry	2	1.4	
Radiation Oncology	1	0.7	
Rehabilitation Medicine	1	0.7	

Majority of the respondents had hospital affiliations (81.8%) and were affiliated with the academe (61%), as shown in Table 3. There were 27 respondents (17.0%) working in government institutions, including the Department of Health (DOH), Philippine Health Insurance Corporation (PhilHealth), and Philippine Council for Health Research and Development (PCHRD).

Table 3 Current Affiliations of the 159 Respondents (multiple responses)

Affiliation	Number	Percent	
Hospital	130	81.8	
Academe	97	61.0	
Government institutions1 (e.g. Philhealth, PCHRD)	16	10.1	
Non-Government Organizations (NGOs)	15	9.4	
Department of Health	11	6.9	
Pharmaceutical/medical device industry	11	6.9	
Research consultancy/Clinical research Organization	5	3.1	
International organizations2	3	1.9	
1 Excluding Department of Health

2 United States National Institutes of Health, World Health Organization, United Nations

Figure 3 shows the geographic distribution of the current primary affiliation of the respondents. Three respondents were based in North America at the time of the survey.

Figure 3 Geographic mapping of current affiliations of the survey respondents.

(Image adapted from Google Maps).7

Respondents generally occupied several professional roles at a time (Table 4). There were several respondents in the academe, with 76 (47.8%) teaching in a clinical academic department, 32 (20.1%) in a basic academic department, 19 (11.9%) in a clinical epidemiology department or unit, and 4 (2.5%) who are research faculty members.

Table 4 Current Professional Roles of the 159 Respondents (multiple responses)

Position	Number	Percent of 159	
Faculty of a clinical academic department	76	47.8	
Staff of a hospital	59	37.1	
Research coordinator for residents and fellows	45	28.3	
Head of a research department or unit	45	28.3	
Faculty of a basic academic department	32	20.1	
Staff in a medical device or pharmaceutical company	22	13.8	
Faculty of a clinical epidemiology department or unit	19	11.9	
Member of a research committee	8	5.0	
Medical director of a medical device or pharmaceutical company	5	3.1	
Private practitioner	5	3.1	
Technical staff for clinical practice guideline development	5	3.1	
Research faculty	4	2.5	
Clinical fellow	3	1.9	
Heads of units/offices in hospitals	3	1.9	
Editor-in-Chief of a peer-reviewed journal	2	1.3	
Medical director of a hospital	2	1.3	
Policy research and development work	2	1.3	
Staff in international research agency*	2	1.3	
Research fellow	1	0.6	
Staff in government research agency	1	0.6	
* World Health Organization and United Nations

Impact of Clinical Epidemiology Education

Majority of the respondents use clinical epidemiology in research endeavors (87.4%), clinical practice (85.5%), and teaching/training (78%), as shown in Table 5.

Table 5 Areas of Application of Clinical Epidemiology among 159 Respondents (multiple responses)

My Clinical Epidemiology Training is Useful in my:	Number	Percent of 159	
Research projects	139	87.4	
Clinical practice	136	85.5	
Teaching/Training	124	78.0	
Research consultancy	68	42.8	
Research management	56	35.2	
Health policy formulation	48	30.2	
Engagements with policy makers	43	27.0	
Involvement in clinical practice guideline development	3	1.9	
Pharmaceutical industry engagements	2	1.2	
Clinical risk management	1	0.6	
Pharmacovigilance work	1	0.6	

The most frequently reported skill that MSc and Diploma respondents attribute to their clinical epidemiology education is critical appraisal of scientific literature (95.6%), followed by comprehensive search of scientific literature (91.8%), teaching critical appraisal of scientific literature (87.4%), and helping others develop research protocols (87.4%). More than 70% can conduct systematic reviews and meta-analyses, write manuscripts for scientific publications, and develop research protocols (Table 6).

Table 6 Self-reported Skills Attributed to Clinical Epidemiology Education of 159 Respondents

My clinical epidemiology training enables me to:	Number	Percent of 159	
Critically appraise scientific literature	152	95.6	
Perform a comprehensive search of scientific literature	146	91.8	
Teach critical appraisal of literature	139	87.4	
Help residents, fellows, and colleagues develop research protocols	139	87.4	
Conduct systematic reviews and meta-analyses	118	74.2	
Write manuscripts for scientific publications	117	73.6	
Develop research protocols	114	71.7	
Provide technical services in guideline and policy formulation	82	51.6	
Develop evidence-based medicine training programs	74	46.5	
Present research results to policy makers and the general scientific community	74	46.5	
Conduct peer review of research papers	3	0.6	
Contribute technical knowledge for research ethics boards	3	1.9	
Teach research methods to students	2	1.3	
Apply evidence-based medicine to clinical practice	1	0.6	
Provide consultancy services for research studies	1	0.6	
Establish research-fortified residency training programs	1	0.6	
Organize a health and research development unit	1	0.6	
Select quality/performance metrics for monitoring	1	0.6	
Perform post-marketing surveillance data analysis	1	0.6	

Research studies done by respondents with considerable impact

Respondents were asked to identify research initiatives that they spearheaded or were involved in which they think had considerable impact. Of 159 respondents, 136 (85.5%) cited at least one study. Citation of the research work by other authors was the most common impact cited by respondents at 27.2% (Table 7). This was followed by use of their research in clinical practice guidelines (CPGs) or clinical practice in general (23.5%). Examples of specific responses in this category are 1) influenced clinical practice of local specialists, 2) basis for providing necessary supplements, 3) used in the local clinical management algorithm, 4) led to adaptation of an equation derived from the research, and 5) enabled diagnosis of a condition specific to the Filipino population. Research studies of the respondents were also used in policy formulation in health institutions, departments/units of a hospital, and specialty societies (22.1%). Some research studies assisted the DOH through provision of feedback to improve existing DOH guidelines and administrative orders, and serving as reference for the DOH CPG Guideline Development Manual.

Table 7 Impact of Research Studies of 136 Respondents (multiple responses)

Impact of research studies done by respondents	Number	Percent	
Cited in other research publications	37	27.2	
Used in clinical practice guidelines or clinical practice	32	23.5	
Formulated health policies for the department, institution, specialty society, or government	30	22.1	
Resulted in a patented/copyrighted product	5	3.7	
Impact on residency program or other training program	3	2.2	
Cited in lectures/books	2	1.4	
Increased the H Index (Scientific Journal Ranking) of a journal	1	0.7	
Inspired more research in the field	1	0.7	
Used in health technology assessment	1	0.7	
Published in book of abstracts/won international oral presentation	1	0.7	
Started a study group that opened opportunities for international recognition of a school	1	0.7	
Assisted in outbreak control	1	0.7	
Improved institution outcomes	1	0.7	
Potential use in clinical practice, local setting, policies	3	2.2	

Evaluation of the DCE Graduate Programs

Of the 159 respondents, 148 (93.1%) would recommend the program they have taken. Of the 148 respondents with a positive response, 134 were from the MSc program, 11 from the Diploma program, and 3 from both the MSc and Diploma programs. Nine respondents (5.7%) who were all from the MSc program answered ‘Maybe’. Two respondents (1.2%) who were also both from the MSc program answered ‘Yes and No.’

Respondents were asked to provide reasons for recommending or not recommending the programs. Of 148 who responded that they will recommend the program, 21 did not give any reason. We also asked respondents for suggestions to improve the program, we got 126 suggestions. The remaining 33 did not have any suggestions. From these responses, qualitative content analysis yielded 11 themes for strengths of the programs (Table 8) and 13 themes for weaknesses/areas for improvement (Table 9).

Table 8 Strengths of the DCE Programs from the Perspective of Students and Alumni

	Strengths (Number of respondents)	Selected Verbatim responses	
1.	The programs are excellent (5), fulfilling (2), comprehensive (2), and well-implemented (1).	Enrolling into the DCE program was one of the best decisions I made in my life, and I truly appreciate and thank everyone who made the journey meaningful and fulfilling.

	
2.	The programs are useful/relevant (8), empowering (8), and have good clinical emphasis (2).	Very useful for personal professional growth and helping others.

Teaches the principles and application of CE in the clinical setting.

Clinical epidemiology is empowering. (It) provides direction for both physicians and patients.

	
3.	The program promotes good attitude towards research, emphasizing its social value and role in clinical practice and policy. (3)	Two of the most valuable aspects of this program are: (1) it teaches us how to do research in the right manner, with emphasis on strengthening the methodology and making it ethically sound, and (2) it emphasizes how important it is for our researches to have clinical and social value.

	
4.	The program develops a holistic, well-rounded health professional. (14) It enables health professionals to perform various roles as:Teacher/Trainer (20)

Researcher (19)

EBM practitioner (10)

Clinical practitioner (7)

Health policy developer (1)

	It has enriched me as a clinician, researcher, teacher, and administrator.

The program has helped me become a holistic physician.

Very helpful in doing research and health policy development.

	
5.	Sound research (what the CE programs teach) is important for improving health policy and consequently health care delivery (4)	The competencies that can be acquired in the program richly complements the practice of specialty and provide opportunities in contributing to the directions of healthcare practice in general.

There is also a high premium set for accuracy and reliability of data, in order to counter the disinformation and misinformation that is replete especially in mainstream (including web-based) media. The clinical epidemiology program propels students to become active participators in this never-ending quest for truth.

	
6.	The program stimulates critical thinking and lifelong learning (3)	It also stimulates skills for critical thinking and lifelong learning.

CE provides fundamental exposure to concepts & basic skills essential to any inquiring mind.

	
7.	The program opens collaborations in research and teaching research, promoting success in one’s career (20)	The Program…gave me the confidence to collaborate with colleagues on research, (and) gave me opportunities for work.

Without my CE degree, I will still be a nobody in the field. It became my ace in the field game.

Provides many career opportunities and leverage in applying or being considered in various academic, clinical/medical, research and administrative job positions.

	
8.	Support from competent DCE faculty (16), staff (2) and peers (1) help students achieve their goals	Professors are very knowledgeable and competent.

The faculty are very helpful and approachable. I have high regards to all the faculty and staff.

The faculty and peers form a supportive environment to help the student achieve their personal goals.

	
9.	The program is useful to maintain a critical mass of clinical epidemiologists in health institutions (7).	Every clinical training institution needs this expertise in their institution

DOH-retained and referral hospitals with training residents should have permanent positions for DCE graduates for better implementation of research in the institution

	
10.	The programs’ schedule and teaching strategies fit the schedule of busy clinicians, especially with the online platform (2)	The program’s curriculum, schedule and teaching strategies are flexible enough to fit the schedule of busy clinicians, especially the online platform.

	
11.	Thumbs Up for the Diploma Program (4)	The Diploma program is substantial despite its short duration.

(The Diploma program) without thesis can be a good alternative track that physician enrollees can choose.

The (Diploma) program opens an entire world of possibilities in such a short time of academic commitment.

	

Table 9 Weaknesses and Areas of Improvement of the DCE Graduate Programs

	Weaknesses/Areas of Improvement	Selected Verbatim Responses	
1.	The program, especially the MSc is not for everyone. Depends on the goal and career path of the person (5)

Applicant must be prepared for the challenging thesis work (5)

Depends on the person’s current responsibilities and personal commitments (2)

	(I) will not recommend (the program) if clinical epidemiology will not complement the long-term career path of the person.

Completing the thesis part of the full program is the ultimate challenge. Those who will take the MSc track must really be motivated to complete the thesis work.

It depends on their depth of responsibilities with their personal life…if single parent or with multiple kids, sole breadwinner, and active in clinical practice…. then the decision to enter DCE must be thought of considerably since the course involve much focused and time.

	
2.	Too heavy workload for the degree being granted (3)	The MSc felt like a PhD (and this was after comparing the program with those attended by my international colleagues).

Bit heavy on the readings because everything is new.

	
3.	Smoother transition for students shifting from Diploma to MSc Program and vice-versa (4)	Create a ladderized education program for Diploma CE and MSc CE.

Allow previously enrolled as MSc to change to diploma program.

	
4.	Keep the curriculum and courses updated and more useful to clinicians (with focus on EBM, critical appraisal, and meta-analysis) (3)	Have more exercises on critical appraisal, especially the meta-analysis and CPGs, since these are most useful in our clinical practice.

	
5.	Incorporating/strengthening the following topics in the curricula: (10) Translational research

Rapid and scoping reviews

CPG development

Survey methods

Statistical software

Non-inferiority trials

Survival analysis

Network meta-analysis

Mixed-methods

Qualitative research methods

Health economics and health policy

Good Clinical Practice course with certification

Convert elective courses (Pharma 250 and CE 207) as required courses

	Suggest to teach translational research.

Add other forms of evidence synthesis in our curriculum such as rapid reviews, scoping reviews, etc. which are gaining traction in health policy and systems research arena.

Strengthen or add advanced courses on survey development.

Include a session on mixed-methods approaches.

Strengthen the health policy and health economics sections of DCE.

I was given an overview of health economics in CE 201. For me, it is an interesting course but with limited time to fully understand and do critical appraisal of health economic articles.

(It) would be helpful to have courses covering certain topics like survival analysis, setting non-inferiority margins for trials, and network meta-analyses.

Some classes which I thought were essential were electives. Advance statistics should be a mainstay aside from Biostat 1 and 2 and also Pharma 250.

	
6.	Wider choices for electives (8): Local or international electives outside DCE

Advance meta-analysis, global health and technology assessments

Offer all listed elective courses during the semesters

	Offer additional electives and/or guidance for electives abroad.

More options for elective courses offered during the semester. There were some electives that were listed but not available/open. Would be useful for those who want to go deeper into a specific aspect of CE.

Include an elective course on advance meta-analysis.

Add Global Health and Health Technology Assessment in the electives.

	
7.	Offer tracking towards specialty areas (clinical trials, critical appraisal, meta-analysis, guideline development, health policy) (5)	(Provide) a track for mastery on a particular subject matter such as Major in clinical trials or Major in critical appraisal or meta-analysis.

For those in MSc program, they could have a track on their 2nd year, probably 6 months, whether they want to hone their skills on health economics, meta-analysis, health policies.

	
8.	Improvement of learning strategies: More hands-on activities or practical application (2)

Research immersion (3)

Continue remote/online learning (8)

	Tap fellows for CPG creation and projects of National Institutes of Health. This will give them ideas on research topics and help them with their thesis.

Continue providing virtual platform to enable access to students from different areas of the Philippines.

	
9.	Improvement of conduct of courses: More interactions/mentoring between faculty and students (7)

Better scheduling of classes and other activities (limit ‘broken’ schedules, minimize class cancellations or postponements, and more flexible schedules) (3)

Improvement of student evaluation to include qualitative feedback (1)

Offer certifications for DCE courses completed (1)

	I found that I needed feedback in other terms than just a grade, like an editor's assessment of your research output work or going over the exams.

More interaction between thesis/course adviser and student off-hours.

Provide better direction to the students with mentoring. This might help them narrow down the area of research they're most interested in.

The coursework schedule is not so “student friendly” especially for those who are in clinical practice, since the broken schedule would usually mean missing many hours/days from the clinic.

Maybe offer certifications for the subjects offered in DCE.

	
10.	Strengthen thesis policies and support: (10) Dedicated faculty advisers with relevant training/background

More intensive mentoring (one-on-one mentoring)

Require students to present regular updates

Orientation on the thesis process

Offer an atmosphere of encouragement and a collegial relationship

Assist in funding acquisition

Research apprenticeship with a senior researcher as thesis

Allow more feasible thesis concepts/topics such as systematic reviews

i. Consider accepting returning students

	There must be dedicated faculty for students (doing their) thesis.

The thesis is difficult to finish. More one-on-one mentoring.

There should be quarterly or bi-annual updates of the students where they would present to at least the head of DCE or training head of DCE along with mentor about any updates of their thesis protocol. And most importantly offer solutions for any identified problems.

The program should offer an atmosphere of encouragement with an aim of at least 100 percent completion of the program.

DCE should direct students on where to get funding for their thesis

Students can be matched to join/partner with more senior researchers with big projects and perform a smaller/shorter component of the research as part of their thesis.

I hope the program and faculty can still consider accepting “old” students.

	
11.	Faculty Development (4) More sincerity and patience especially for the not so 'bright'

Should be more accepting of organizations with different culture from UP

Larger and more varied (multi-disciplinary) faculty pool

	For some faculty to sincerely share their time and patience to students who are not as “bright” as they are.

They should be more accepting of organizations that are different from the “UP mentality” culture.

The program would have benefited from having a larger faculty pool, with varied backgrounds.

	
12.	Improve facilities and infrastructure Upgrade the library, study areas, and materials (2)

Subscription to databases related to research (2)

Exposure of librarians to CE concepts for better appreciation of the needs of CE students (1)

	I wish the Department can provide a library/study area, more study materials.

A better list of references and perhaps improved access to these.

Subscriptions to the major and often-relied upon search engines.

Exposure and training of librarians to CE concepts to increase their appreciation of and be more in sync with the needs of research students and university students in general.

	
13.	Provide alumni support through: (7) Refresher courses

Invitation to lectures on new topics

Online access to the courses

Get-together for alumni

Support for conduct of research

Maintain communication lines with alumni (2)

	Put us in the loop whenever needed and if there are current trends and knowledge applicable to clinical epidemiology.

Continue online access to CE 217 and other courses if possible, so that we can have a chance to update ourselves as alumni.

Get together for graduates.

Provide modules online as refresher courses or continuing education.

	

Some students suggested initiatives other than the existing graduate programs such as developing a PhD program for clinical epidemiology, improving international collaborations, and offering short courses to health professionals.

DISCUSSION

This study demonstrates that majority of the students of the DCE graduate programs are physicians who use clinical epidemiology in a variety of roles, including research endeavors, clinical practice, and teaching. Majority of the respondents (93.1%) would recommend the program they have taken. The study described several strengths and limitations of the programs, as well as the suggestions for improvement of the programs from the perspective of the students and alumni.

This survey reiterated the importance of DCE graduate programs in capacity building of healthcare professionals, particularly in using and doing research. DCE students and alumni currently serve in various healthcare and health-related institutions—hospitals, academe, NGOs, DOH, pharmaceutical industries, and research organizations. Respondents expressed how the programs helped them in their careers. They were able to apply the knowledge and skills gained from the programs in their clinical practice, teaching, research consultancy, policy formulation, and CPG development. These findings are similar to the results of the previous DCE program evaluation by Sana et al., reflecting that the DCE programs are sustainable and remain relevant in spite of the continuously changing global and local healthcare and health research landscape.3

This survey identified several strengths of the programs. The programs were considered useful in maintaining a critical mass of clinical epidemiologists in health institutions. In addition, the programs promote good attitude towards research, emphasizing its social value and role in clinical practice and health policy, develop well-rounded health professionals, stimulate critical thinking and lifelong learning, and open collaborations promoting success in the students’ career. Support of the faculty, staff, and peers allowed students to achieve their goals. The programs’ curriculum, schedule, and teaching strategies, especially the online platform, fit the schedule of busy clinicians.

Despite the many strengths of the programs, several weaknesses lead to the low graduation rate of only 46.5% of MSc students. The graduation rate is higher at 86.4% for the Diploma program. DCE, in its continuing commitment towards continuous quality improvement of its programs, is determined to consider the findings of this survey and institute appropriate actions to the best of its ability.

Marketing and admissions

The survey showed that in the past 30 years, 93.7% of admitted applicants are physicians. This is reflective of the manner of invitation of potential students, since invitations are sent mainly to hospitals and medical schools. More recently, invitations are also sent to other colleges in the health sciences. Hence, a few students/alumni are nurses, pharmacists, and allied medical professionals. To expand the scope of the DCE programs, a possible strategy is intentional recruitment from disciplines and specialties that are under-represented. The strategy of targeted mailing is recommended by international graduate programs as among one of the best practices for successful graduate student recruitments. Other recommended practices include creation of accessible websites, advertising through social media, and developing effective networks for recruitment.8,9

Majority of the students in the Diploma program graduate. There are a lot of students in the MSc program who are unable to complete their thesis but have completed the course work of the program, which is equivalent to the requirements of the Diploma program.

The survey showed that the self-reported skills of graduates for both the MSc and Diploma programs encom-pass several critical skills for a clinical epidemiologist. Given these findings, DCE can consider increasing its slots for the Diploma program. Of the 291 accepted applicants, only 22 (8%) were accepted into the Diploma program. DCE can reverse this pattern to a 10:90 MSc: Diploma ratio, with 10% of the accepted applicants enrolling into the MSc program and 90% of the accepted applicants enrolling into the Diploma program. Only students who aim to be doers of research with demonstrated interest and abilities (e.g., publications, research involvement) will be considered for the MSc program.

There are only a few applicants for the Diploma program, which may be due to the misconception that the Diploma program is not a graduate degree. It is sometimes falsely viewed as a certificate program only. In fact, it is an applied degree program with rigorous requirements, namely, 28 units of course work requiring various outputs such as research protocols, systematic review, and meta-analysis. Some of these outputs have been published in scientific journals.10,11 The name of the program, Diploma in Epidemiology (Clinical Epidemiology), was derived from the British System/Nomenclature of graduate programs. In the American system (the system generally implemented in the country), the Diploma program is equivalent to a professional graduate program like the Master of Public Health. Taking these factors into consideration, DCE is embarking on curricular changes for this program including a change of name to Master of Clinical Epidemiology.

Some challenging situations in finishing the MSc program were identified by the respondents, including clinicians starting their practice and students who are breadwinners of the family. These challenges are commonly reported for graduate students worldwide, since graduate students are more likely to juggle familial and professional obligations compared to undergraduate students.12 The programs, especially the MSc program, can be beneficial/advisable to those ‘for whom CE training will complement their long-term plans’ and ‘who have the determination to finish the program despite obstacles/stumbling blocks’.

Some respondents suggested to implement a smoother transition when students wish to shift from the Diploma to MSc program or from the MSc to Diploma. Currently, shifting from MSc to Diploma is not allowed. Shifting from Diploma to MSc is only allowed while the student is enrolled in the Diploma program. When the student already finished the Diploma program, he or she is considered a new applicant to the MSc program. Once accepted to the MSc program, units are credited according to the rules of the University. Given the findings of this survey, it seems more prudent to recommend that students finish their Diploma degree first rather than apply to shift to the MSc program. The Diploma degree is already a considerable, concrete accomplishment. Shifting to the MSc program, which has a markedly lower graduation rate, can be considered a gamble, with the chances of graduating depending on their determination and life situation.

Course content, offerings, and evaluation

Respondents generally found the course content very good and helpful, although some found the course load too heavy. Program content and materials must be streamlined and updated regularly, given the rapid advancements in epidemiology.

Respondents also suggested to provide qualitative assessments to students. Since DCE aims to hone skills such as protocol development, critical appraisal, conduct of systematic reviews, and manuscript writing, qualitative assessments would provide students more information on how to improve their work. Although UP grading system is quantitative, DCE can address this suggestion by giving qualitative assessments or comments during the formative part of the courses. This is currently being done in most DCE courses. For example, in CE 212, students present their draft protocols to the faculty for comments. In CE 215, a faculty team works together with the students to refine the protocols and manuscripts of systematic reviews written by the students. This suggestion can be a reminder to DCE faculty that such comments and feedback are important to students.

Some respondents suggested for DCE to offer non-degree capacity building programs for health professionals. DCE has been offering short courses on research methods, data analysis, and scientific paper writing as stand-alone workshops. Instead of offering such workshops, or in addition to these workshops, DCE can offer certificate programs that can include essential CE courses. These certificate programs will enable health professionals to better function as training coordinators in residency and fellowship programs, and improve the conduct of their own research.

Respondents suggested more choices for electives. A few students even suggested offering electives in partnership with other institutions, including international institutions. DCE can explore these possibilities. Its history of collaborative work with other CEUs and CERTCs all over the world makes this initiative realistic. Local and international partnerships would add value to the programs as well.

Curricular changes

DCE has been exploring the possibility of students to take ‘tracked’ programs. A student interested in health policy can take the CE programs majoring in health policy. Another can focus his/her training in health economics. This ‘tracking’ option was also suggested by some of the respondents. A few respondents suggested ‘tracking’ not just in existing disciplines at DCE, but also in newer fields such as health technology assessment. This course of action, however, may be difficult at this point given the current resources available.

These curricular changes are particularly relevant to meet national and international demands for experts on specific CE fields. With the passage of the UHC law in the Philippines, the need for CPGs to cover a variety of common conditions became urgent. Health economics and health technology assessment are also urgently needed locally and globally for priority setting and decision making in relation to several treatment, diagnostic, and preventative strategies.13 Clinical epidemiologists ‘specialized’ in these fields are greatly needed.

Program implementation

Respondents expressed appreciation to DCE faculty and staff for their support and expertise. They also appreciated the flexibility in schedule, administrative support, and mentoring sessions. To further improve the programs, respondents suggested offering all listed electives every year or every semester, closer mentoring of students, and continuation of the online learning platform.

During the pandemic, DCE implemented a 100% remote learning using Zoom, Canvas, University Virtual Learning Environment (UVLE) or Google facilities. This survey showed that this platform was found effective and convenient by the respondents. It was earlier noted that the interest in the programs increased (as evidenced by the increase in the number of applicants) during the pandemic. The online platform could be one of the major reasons for this increase. When remote learning was being implemented, DCE received applications from areas outside the National Capital Region and Southern Luzon, including Baguio, La Union, Zamboanga, Iloilo and Marawi. Remote learning, however, requires access to technology to be effective. A report on the experience of an online public health capacity building educational training directed to health professionals of Low-Middle-Income Countries (LMICs) found that poor information technology (IT) access, lack of preparation for self-directed adult learning, and language barriers reduce the effectivity of online learning.14 While language barrier may not be a big problem for our programs since the medium of instruction in our educational system is English, poor IT access and lack of self-directed learning may pose a challenge to some students.

As the pandemic wanes, learning institutions are going back to the face-to-face mode of learning. Aside from the challenges of remote learning mentioned above, another difficulty of remote learning for our students is the challenge of following instructions for hands-on sessions such as learning how to use a software (Review Manager in CE 215 and Stata in CE 205). Given the positive experience of the respondents with remote learning, DCE can offer a blended form of learning. Hands-on sessions may be done face-to-face in a Blended Learning Model. Such model will not deter applicants outside NCR since they can opt to go to Manila only for these sessions or learn the software on their own. There are a lot of online demonstrations on how to use such software. The faculty can curate these resources for the students. The preference for blended learning is similar to the experience of the clinical epidemiology program in Africa.15

Faculty and staff complement, library resources, and infrastructure

DCE is determined to address the weaknesses and suggestions for improvement obtained in this study. However, this would be hard to implement without faculty and staff development. Respondents similarly saw this need and recommended increase of faculty complement and expansion of the multi-disciplinary nature of the faculty. Since graduate students operate in a less structured environment and require more independent study compared to undergraduate students, the role of the faculty as a supervisor is important. The supportiveness, or lack thereof, of supervisors has been shown to be a predictor of graduate student satisfaction with the program.6 Thus, with the increasing number of applicants and students being accepted in the program, the faculty complement must be carefully assessed if the current number can adequately address the needs of the students. To expand the multi-disciplinary nature of the faculty, DCE can recruit not only clinical epidemiologists of various medical specialties, but also experts in global health, clinical trials, and health technology assessment.

Equally important is the maintenance of library facilities and other resources. DCE recommends that subscriptions to databases of the medical literature be undertaken by the library.

With respect to staff development, DCE continually supports the staff through the Foundation for the Advancement of Clinical Epidemiology, Inc. (FACE). Currently, DCE has one administrative officer, one administrative assistant (on job order) and one clerk messenger who is fully supported by FACE. At the minimum, DCE can function with three support staff who ideally should be fully supported by UP. With the potential increase in number of students and faculty, the number of support staff may also need to be re-evaluated.

Maintaining the faculty complement has always been a challenge for DCE. Currently, four faculty members are graduates of the DCE programs. There is no dearth of potential CE faculty. What is lacking are UP items to offer to applicants. The maintenance of faculty with CE-related disciplines is more challenging. DCE has been in search for potential biostatisticians, social scientists, and economists; however, a UP item is not automatically available for potential applicants.

Improvement of thesis policies and support

A lot of the suggestions of the respondents centered on thesis policies and support. Respondents consider this stage of their training as most challenging; thus, it is an area where they need much support. International studies that evaluated their graduate programs have similar findings that thesis completion was the largest barrier to program completion.13 An international study that evaluated the important competencies of epidemiology graduate students reported that doing and completing thesis work had substantial value. The epidemiology graduates reported that thesis work allowed application of research skills and created networking opportunities for them.16 Thus, strengthening thesis policies to provide a more conducive environment for students to complete their thesis work is of paramount importance.

Currently, each student is assigned a CE faculty/program adviser and a statistician adviser. Once they are on their thesis stage, they have a thesis panel headed by a Thesis Chair to supervise and monitor their progress. Apparently, this system is not adequate. Several respondents suggested closer monitoring and supervision especially during the thesis stage. One roadblock to provision of more intensive monitoring is the availability of enough faculty to supervise the growing number of students. International publications similarly report concerns about the capacity of their faculty to supervise many students amidst fulfilling other competing responsibilities, as well as difficulty in recruitment of additional supervisors.14

DCE is now conducting monthly sessions with students in CE 300 (the thesis course code). Aside from the orientation at the beginning of the program, DCE now conducts another orientation after the MSc students finish the course work. DCE also expanded the possible thesis topics, including a design thesis, that students can embark on. In addition, DCE is helping the students establish a student council as an additional avenue to keep communication between faculty and students open.

Alumni support

Respondents suggested refresher courses, invitation to attend lectures on new topics, and online access to contents of courses. This is now very feasible given the online platform. Other suggestions include holding get-togethers for alumni, reaching out to graduates and even those who only finished the course work, and support for conduct of research. To address these suggestions, DCE is assisting the alumni to organize themselves and plan to set up an alumni association.

Limitations

The main limitation of this survey is the low response rate of 55.4%. Despite our best efforts, a sizable proportion of our alumni did not answer the survey. The characteristics and sentiments of these non-responders may be significantly different compared to those who responded.

The survey was conducted by DCE faculty and staff. The email invites were sent by DCE. There was a possibility that socially desirable responses were obtained due to the student/alumni-faculty relationship. Despite this, the survey generated not only the strengths of the programs but weaknesses/areas of improvement as well.

The assessment of competency by the participants were subjective and thus do not necessarily translate to actual competency in clinical research. This should be complemented by objective measurement such as a validated survey to test competency of clinical research professionals that the department produces.11

The survey asked respondents to name research deemed to have considerable impact. However, “impact” was not operationally defined in the survey. Thus, the responses to this question were varied, with some responses pertaining to outputs and outcomes of the respondents’ research.

The results of this study are applicable only for the DCE graduate programs. While other graduate programs may have some similarities in strengths and limitations, the study respondents’ perspectives were particularly focused on the DCE graduate programs and the field of clinical epidemiology. Thus, the results of this study cannot be generalized to other graduate programs.

Future directions

The ability of graduate students to ‘thrive’ in their academic programs is a multi-dimensional construct involving six themes—1) achieving academic goals and milestones, 2) engaging and actively participating, 3) connecting with peers and faculty, 4) balancing work and life commitments, 5) enjoying, and 6) being confident, dedicated, and motivated.11 Student satisfaction is regarded as an important performance indicator of higher education institutions.17 The DCE will continue to improve its programs to ensure that our students and alumni thrive. We will regularly evaluate the strengths, weaknesses, and impact of our programs, especially after instituting the above recommendations, to ensure that our programs are relevant and effective in meeting the needs of our students and our country.

CONCLUSION

This survey reiterated the importance of clinical epidemiology graduate programs in research capacity building of health care professionals. Students and alumni occupied diverse positions in academic, research, clinical, and pharmaceutical setting, and majority accomplished research studies with considerable impact.

DCE graduate programs play a crucial role in the efforts to capacitate health care professionals in using and doing research. A major challenge identified in this study leading to a low graduation rate in the Master of Science program is the completion of thesis work. The survey identified several initiatives towards continuous quality improvement of clinical epidemiology programs, including improvement of thesis policies and support, updating the curriculum content and materials, increasing allotment of hours for hands-on activities, exploring possibilities of offering electives in partnership with other institutions, offering a blended learning platform, maintaining an efficient administrative support for students, and continuing education for alumni. Through the findings of this survey, DCE will continue strengthening and improving its programs to ensure that students receive the best training in clinical epidemiology that it can offer. Strong institutional support for personnel and infrastructure development is essential for these initiatives to succeed.

Acknowledgments

This initiative would not have been possible without the full support of the DCE Chair, Dr. Jacinto Blas Mantaring. We thank all students and alumni who participated in this survey. Special thanks to Dr. Teresa Tolosa (DCE Graduate Student) and Dr. Mantaring for their participation in the pre-test.

Statement of Authorship

All authors certified fulfillment of ICMJE authorship criteria.

Author Disclosure

All authors declared no conflicts of interest.

Funding Source

No funding was received for the conduct of this study.

This study was initiated by faculty members of the DCE.
==== Refs
REFERENCES

1 The INCLEN Trust International. Who We Are [Internet]. 2017 [cited 2023 Jul 4]. Available from: http://inclentrust.org/inclen/introduction/.
2 World Health Organization. Epidemiology and Related Training Capacity in Selected Countries in the South-East Asia Region: A Rapid Review [Internet]. 2009 [cited 2023 Jul 4]. Available from: https://apps.who.int/iris/bitstream/handle/10665/205013/B4257.pdf?sequence=1.
3 Sana EA, Atienza MA, Mojica JAP, Abarquez LF, Fajutagana NS. Evaluation of the Master of Science in Epidemiology (Clinical Epidemiology) curriculum. Acta Med Philipp. 2009;43 (4 ):35-42. doi: 10.47895/amp.v43i4.7390.
4 Ferreira JESM, de Oliveira LR, Cavalcante TF. Contribution of students in the evaluation of master's and doctoral programs in the field of nursing. Invest Educ Enferm. 2022 Mar;40 (1 ):e01. doi: 10.17533/udea.iee.v40n1e01. PMID: 35485614 PMCID: .35485614
5 Zuniga NC, Colbern A. The annual BUILD snapshot: tracking alumni outcomes. HCI Intern 2021 Late Break Pap (2021). 2021 Jul;13096 :345-60. doi: 10.1007/978-3-030-90328-2_22. PMID: 35281203; PMCID: 35281203
6 Samuels E, Ianni P, Chung H, Eakin B, Martina C, Murphy S, et al . Guidelines for Evaluating Clinical Research Training using Competency Assessments [version 2]. MedEdPublish. 2020;8 :202. doi: 10.15694/mep.2019.000202.1.38089266
7 Google Maps [Internet]. 2023 [cited 2023 Jun 27]. Available from: https://www.google.com/maps.
8 Arizona State University Graduate College. Best Practices for Successful Graduate Student Recruitment [Internet]. 2017 [cited 2022 Nov]. Available from: https://graduate.asu.edu/sites/default/files/gradrecruitmentbestpractices-feb2017.pdf.
9 Hanover Research. Best practices in graduate student recruitment [Internet]. 2017 [cited 2022 Nov]. Available from: https://www.winona.edu/gradstudies/media/best-practices-in-graduate-student-recruitment-final.pdf.
10 Tan-Lim CSC, Esteban-Ipac NAR. Probiotics as treatment for food allergies among pediatric patients: a meta-analysis. World Allergy Organ J. 2018 Nov;11 (1 ):25. doi: 10.1186/s40413-018-0204-5. PMID: 30425779; PMCID: .30425779
11 Quilendrino MIO, Castor MAR, Mendoza NRNP, Vea JR, Castillo-Carandang NT. Parents' perceptions of autism and their health-seeking behaviors. Clin Epidemiol Glob Health. 2015;3 (S1 ):S10–S15. doi: 10.1016/j.cegh.2015.11.003.
12 Coe-Nesbitt HA, Soleas EK, Moucessian AM, Arghash N, Kutsyuruba B. Conceptualizing thriving: an exploration of students’ perceptions of positive functioning within graduate education. Front Educ. 2021;6 :704135. doi:10.3389/feduc.2021.704135.
13 Uzochukwu BSC, Okeke C, O'Brien N, Ruiz F, Sombie I, Hollingworth S. Health technology assessment and priority setting for universal health coverage: a qualitative study of stakeholders' capacity, needs, policy areas of demand and perspectives in Nigeria. Global Health. 2020 Jul;16 (1 ):58. doi: 10.1186/s12992-020-00583-2. PMID: 32641066; PMCID: .32641066
14 Heller RF, Barrett A, Oaiya O, Heller J, Madhok R. Final report of a novel and successful online public healthy capacity building experiment. Open Praxis. 2022;14 (1 ):83-92. doi: 10.55982/openpraxis.14.1.150.
15 Young T, Naude C, Brodovcky T, Esterhuizen T. Building capacity in Clinical Epidemiology in Africa: experiences from Masters programmes. BMC Med Educ. 2017 Feb;17 (1 ):46. doi: 10.1186/s12909-017-0885-4. PMID: 28241762; PMCID: .28241762
16 DiSipio T, Protani MM, Finnane A, Johnson R, Hall L. Exploring postgraduate epidemiology competencies: Preparing our students for a post-COVID world. Aust N Z J Public Health. 2023 Apr;47 (2 ): 100026. doi: 10.1016/j.anzjph.2023.100026. PMID: 36906998.36906998
17 Wong WH, Chapman E. Student satisfaction and interaction in higher education. High Educ (Dordr). 2023;85 (5 ):957-978. doi: 10.1007/s10734-022-00874-0. PMID: 35669591; PMCID: .35669591
