
==== Front
Tunis Med
Tunis Med
Tunis Med
La Tunisie Médicale
0041-4131
2724-7031
Tunisian Society of Medical Sciences

Article
Patient-management problem dans l'évaluation de la médecine intensive
Patient-management problem in the evaluation of intensive medicineTrifi Ahlem 11
Benzarti Aida 22
Mehdi Asma 11
Seghir Eya 11
Messaoud Lynda 11
Abdellatif Sami 11
1. Medical intensive care unit, la Rabta hospital, Tunis, Tunisia
2. Occupational medicine department, la Rabta hospital, Tunis, Tunisia
4 2023
05 4 2023
102 4 194199
2022
https://creativecommons.org/licenses/by-nc-nd/4.0/ This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
RESUME

Introduction: En médecine aigue (MA), l'utilisation du Patient-Management Problem (PMP) reste limitée et pas de retour des étudiants. Objectif: Comparer la faisabilité de l'utilisation de PMP faisant référence à des cas cliniques (CC) comme outils d'évaluation des connaissances et compétences des étudiants en MA et recueillir la perception des étudiants concernant cette expérience.Méthodes: il s'agissait d'un essai randomisé transversal. Etaient inclus les étudiants externes en 3ème année du 2ème cycle d'études médicales (DCEM3) lors de leur externat en MA. Tous les participants ont eu deux tirages au sort (le 1er pour l'outil d'évaluation à démarrer (PMP ou CC) et le 2ème pour l'ordre de passage au PMP. Deux PMP versus deux CC-QCM groupés ont été préparés et un questionnaire de satisfaction a été distribué. Le principal critère de jugement était l'effet de chaque outil d'évaluation sur le processus de prise de décision des étudiants. Cela a intéressé la pertinence des éléments apportés par chaque technique, l'implication et la difficulté ressentie. Le critère d'évaluation secondaire était les scores obtenus pour chaque outil testé.Résultats: 20 étudiants ont été inclus. Tous les participants avaient une expérience préalable aux PMP et seuls neuf connaissaient les CC-QCM groupés. Les scores PMP étaient de 14,9 pour le 1er thème et de 15,8 pour le 2ème thème. La médiane des scores CC-QCM groupés était de 14 [12-16] pour les deux. Les moyennes ne différaient pas entre les deux techniques. Pour le 1er thème : les scores étaient négativement corrélés (r=-0,58 et p=0,007). Les étudiants ont ressenti une meilleure satisfaction pour l'évaluation par PMP (p<10-3), les éléments fournis par le PMP étaient plus pertinents pour le processus décisionnel (p<10-3), l'implication était plus ressentie avec le PMP (p<10-3) et la difficulté était plus ressentie avec les CC (p<10-3). Pour les 2 sessions, l'effet du PMP a été jugé significatif sur le raisonnement clinique (n=36), l'auto-évaluation (n=38), la résolution de problèmes (n=40) et la prise de décision (n=39). Les étudiants ont fortement recommandé le PMP comme outil d'évaluation en réanimation (p<10-3).Conclusion: les scores étaient comparables entre les deux techniques testées. La perception positive vis-à-vis du PMP encourage sa généralisation et la formation des enseignants doit être renforcée.

ABSTRACT

Introduction: In intensive care medicine (ICM), the use of Patient-Management Problem (PMP) remains limited and no feedback from students is available. Aim: To compare the feasibility of employing PMP referring to clinical cases (CC) as assessment tools for appraising the knowledge and competencies in ICM students; and to gather the students’ perception regarding this experience. Methods: it was a cross-sectional randomized trial. Were included, external students in the 3rd year of the 2nd cycle of medical studies (3rd-SCMS) during their ICM externship. All the participants underwent two random draws (the 1st one for assessment tool to be started (PMP or CC) and the 2nd for the passage order for PMP. Two PMPs versus two grouped QCMs-CC were prepared and a satisfaction questionnaire was distributed. The main judgment criterion was the effect of each assessment tool on the students' decision-making process. This focused on the relevance of the elements provided by each technique, the implication and the difficulty felt. The secondary endpoint was the scores taken for each tool tested. Results: 20 students were included. All participants had previous experience with PMPs and only nine were familiar with grouped MCQs-CC. PMP scores were 14.9 for the 1st theme and 15.8 for the 2nd theme. The median of the grouped MCQs-CC scores was 14 [12-16] for both. The scores didn’t differ between the two techniques. For the 1st theme: the scores were negatively correlated (r=-0.58 and p=0.007). Students felt a better satisfaction for PMP evaluation (p<10-3), the elements provided by PMP were more relevant for decision-making process (p<10-3), the involvement was more felt with PMP (p<10-3) and difficulty was more felt with CCs (p<10-3). The effect of PMP was found to be significant on clinical reasoning (n=36), self-assessment (n=38), problem solving (n=40) and decision making (n=39). Students recommended strongly PMP as a tool of evaluation in ICM (p<10-3). Conclusion: scores were comparable between the two tested techniques. The positive perception of students regarding PMP encourages its generalization and teacher training must be strengthened.

Mots clés

Evaluation
Pédagogie médicale
Questionnaire
Etudiant en médecine
Réanimation
Patient Management Problem CorrespondanceAhlem Trifi Medical intensive care unit
la Rabta hospital
Tunis
TunisiaEmail
trifiahlem
gmail
Keywords

Evaluation
Medical education
Questionnaire
Medical student
Management
Intensive care
Patient Management Problem résuméIntroduction
En médecine aigue
MA
l
utilisation du Patient
Management Problem
PMP
reste limitée et pas de retour des étudiants
==== Body
pmcIntroduction

Over the past few decades, the medical education described remarkable progression in teaching and assessment programs.

It is based on active and selfdirected pedagogical principles.

The problem-solving approach places learners in the active role that reflect the "" real" practice to them.

The Patient-Management Problem (PMP) represent an educational tool developed since the 1960s and consists of studying a clinical file cases where the information is provided in a sequential, linear or algorithmic way, at the request of the learner, after each of his answers (1).

The PMPs allowed the possibility to assess the student's ability to collect relevant information from the patient.

In addition, it incites to interpret the physical examination data’s, to select and interpret the appropriate complementary examinations and consequently to establish a management strategy (2).

In intensive medicine, environment full of complex and stressful critical situations, requires a rapid and efficient clinical reasoning (3 ).

In this context, the use of PMP as an assessment tool of ICM students finds all its interest.

Herein, we aimed to compare the feasibility of employing PMP versus CC as assessment tools for appraising the knowledge and competencies in ICM students.

This interested mainly the students' decision-making process (the relevance of the elements provided by each technique, the implication and the difficulty felt).

Also, we sought to gather the students’ perception regarding PMP referring to clinical cases (CC) with grouped MCQs.

Methods

Study design

This was a cross-sectional randomized trial conducted in a hospital externship site: the medical intensive care unit(ICU) of the teaching hospital center of la Rabta over five months (from January to May 2023).

Hence, it concerned 3 externship periods of the 2nd semester of 2022/2023 academic year.

Study participants and randomization

Were included, external students in the 3rd year of the 2nd cycle of medical studies (3rd-SCMS) during their ICM externship.

All the participants underwent two random draws (the 1st one for assessment tool to be started (PMP or CC) and the 2nd for the passage order for PMP.

The externs who gave their consent to participate were informed by the study protocol and the corresponding essay topics by asking them to read their mini-modules beforehand.

Study protocol

For the three externship period, two evaluation sessions using the two methods were carried out (a total of 6 sessions); as a cross-test form (i.e. in the same session, a student explored the 2 tests).

The evaluation session began with a reception phase of 10 to 15 minutes.

During this phase, the teacher presented the general framework, its realization steps, PMP procedure and ensures the prior reading of mini-modules as previously requested by asking them oral questions on the topic.

Participants were invited, each time, to make two random draws (the 1st one for the evaluation method to be started (PMP or clinical case: CC) and the 2nd one for the passage order of PMP.

The PMPs were carried out individually (15 mn per student).

For the MCQ-CC test, each student worked alone but at the same time as their colleagues and in the same room-exam.

The supervising teacher monitored the test in order to avoid the group work.

All the session ended with a 30-mn group debriefing phase collecting students' perception about strengths, weaknesses and difficulties encountered.

Also, the proofreading was made and a satisfaction questionnaire was distributed.

Materials

We prepared two PMPs versus two CCs and a perception/ satisfaction questionnaire. Each PMP and its matched CC were focused on a chapter of intensive care medicine: severe exacerbation of asthma (SEA) and septic shock.

PMPs were designed in electronic form (Microsoft Powerpoint® 2019) and interested both diagnostic and therapeutic management.

Each PMP contained 3 slides: clinical vignette (severe exacerbation of asthma with acute lobar pneumonia for PMP 1 and septic shock due to acute pyelonephritis for PMP 2), an instructional slide explaining the PMP grading scale, a list of 22 options to choose from.

Each proposal was linked by a hypertext link to a slide with the expected answer and the assigned rating.

Information was provided in a linear sequential way.

For the PMP score calculate, we used the operational grading scale at the Faculty of Medicine of Tunis (4): a positive rating (+1 or +2) if the proposal was useful or essential, and a rating null or negative if the proposal was useless and/or dangerous for the patient

The student was required to choose the correct answers and in the correct order.

Correct answers that were not chosen in the correct order were scored zero.

The final score obtained was converted into a mark out of twenty by taking into account the maximum possible scores for each PMP and by applying the rule of three.

Clinical cases (CCs) were designed in paper formats: for the 1st theme (SEA), two clinical cases were prepared, each of which included five multiple-choice questions (MCQs) with grouped complements and the score to be analyzed corresponded to the sum of the two scores.

For the second theme (septic shock), a CC was prepared comprising 10 MCQs with grouped complements.

Both diagnostic and therapeutic attitudes were examined and covered taxonomic level 2 and 3 objectives.

Each question was scored on 2 points (a total of 20 points for each CC).

Scoring was based on the all-or-nothing law (given that a single grouped complement is the correct answer).

Scoring details

For the PMP score calculate, we used the operational grading scale at the Faculty of Medicine of Tunis and the final score was converted into a mark out of twenty by taking into account the maximum possible scores for each PMP and by applying the rule of three:

• For PMP1: the maximum score is obtained when the student selects, among the 22 proposals, 8 essential propositions (equivalent to 16 points), 3 useful propositions (equivalent to 3 points) and none of the useless and/or dangerous propositions.

The appropriateness of the order choice of proposals was noted by the examiner's judgment on 5 points.

Thus, the maximum score to attribute to PMP 1 was 24 points.

• For PMP2: the maximum score is obtained when the student selects, among the 22 proposals, 7 essential propositions (equivalent to 14 points), 5 useful propositions (equivalent to 5 points) and none of the useless and/or dangerous order choice of proposals was noted by the examiner's judgment on 5 points.

Thus, the maximum score to attribute to PMP 2 was 24 points.

Clinical cases (CCs) were both noted on 20 points (each MCQ of 2 points) and the maximum score to attribute to every CC was at 20 points (additional material):

• For CC 1: two clinical cases were prepared, each one comprised 5 MCQs and marked on 10 points with a total of 20 points.

• For CC2: one CC was prepared including 10 MCQs and each question was scored on 2 points (a total of 20 points).

The satisfaction questionnaire (paper format) included 3 parts: the 1st one explored the general and logistic data, a 2nd one examined the previous experience regarding PMP and/or grouped MCQs-CC, and a 3rd part was focused on the perception/satisfaction of participants.

The last part used ratings on a Likert scale and examining: overall satisfaction (where 1= not at all satisfied to 5= very satisfied), contribution of the provided elements in the decision-making process (where 1= not at all sufficient to 5 = completely sufficient), student’s degree of involvement in the decision-making process (where 1=not at all involved to 5=full involvement requiring reflection), difficulty in choosing the decision (where 1= no perceived difficulty and 5= major difficulty), contribution of the PMP during this experience (where 1= no contribution and 5= major contribution) on the clinical reasoning, self-assessment, problem solving and decision-making decision, comparison between the two methods in terms of simplicity and perceived stress and finally their recommendation scale (where 1= totally disagree to 5= totally agree) as a tool of evaluation for each one (additional material).

Judgment criteria

we focused on the comparison of scores taken for each tool tested and the students’ perception on each evaluation tool.

Statistical analyses

participants’ answers were collected, recorded and analyzed on an electronic database using SPSS 20 software.

Quantitative variables were expressed as mean, standard deviations and extreme values or as median and interquartiles (25-75).

The Wilcoxon test was used to compare scores assigned in PMPs versus CCs from the same student, as considered matched samples.

Spearman test was used for correlation between the notes obtained in the two tests.

The usefulness of these statistical tests is designed for the comparability of the 2 techniques to distinguish the difference in assessment of the knowledge/competencies in ICM students through the scores assigned.

The categorical variables were expressed in absolute number and in percentages.

A p value < 0.05 was set for statistical significance.

Results

General characteristics

Twenty students were included with a sex ratio (F/M) = 4 (16/4) having a median age of 23 years [23-24].

All the participants had a prior experience with PMP (exclusively during the pediatric externship evaluation) and 9 (45%) were familiar with grouped MCQs-CCs.

Tests results

• By theme: For PMP 1 (interested in SEA): the median score was at 14.9 with a maximums at 9.1 and 20,one participant didn’t succeed it (9.1/20).

For PMP 2(interested in septic shock), all the participants succeeded it.

The two PMP scores didn’t differ.

Concerning the CCs, all the participants succeeded the two tests with a median at 14/20 for both.

• By assessment tool: The attributed scores were comparables.

A negative correlation was highlighted between the obtained scores at CC and those of PMP for the 1st theme (rho=-0.58, p=0.007) and no correlation was found for the 2nd theme results (r=0.16, p=0.49).

The scores details and comparisons are displayed in table 1.

• Per student: scores by PMP were better than those by grouped-MCQs CCs in 13 students for theme 1 and in 11 for theme 2.

Students’ Perception

This part was analyzed by combining the questionnaire’s answers of the two evaluation sessions equivalent to a sample size at n=40.

Overall, the global satisfaction for the evaluation by PMP was better than the grouped MCQ-CCs: Table 2.

Distribution of students’ number according to the investigated items is displayed in figure 1.

Figure1. Distribution of students according to their answers to the perception questionnairePMP: patient management problem, MCQ: Multiple choice question, CC: Clinical case, ICM: Intensive care medicine

Almost all of participants noted the impact of PMP from fairly significant to major (score=4-5) on the four questioned components without significant difference between them (p=0.36).

In terms of simplicity, PMP was considered to be simpler than grouped MCQs-CC in 31 times (77.5%).

PMP was considered equivalent to lower stress generator than grouped MCQs-CC in 22 times (55%).

In free comments(14 times), the most requested difficulty for CC was related to the grouped nature of the proposals (n=9) and for the PMP, it was about the order of choices priority(n=5).

Discussion

Our crossover trial included 20 students at the 3rdSCMS in ICM externship who passed two tests using two different assessment tools (study tool: PMP versus reference tool: grouped MCQ-CC).

The scores assigned to PMPs were 14.9 [12.9-16.9] for the 1st and 15.8 [12.7-16.4] for the 2nd.

For the CC scores, the median was 14 [12-16] for both.

By comparing the results obtained by PMP versus those obtained by CC, no difference was observed (neither by theme nor themes combined).

However, for the 1st theme, the scores were negatively correlated (r=-0.58 and p=0.007).

Per student: thirteen and eleven had better grades by the PMP for theme 1 and theme 2 respectively.

Overall, the participants experienced better satisfaction with PMP.

They considered that: the elements provided by PMP were more relevant for the decision-making process, the involvement was more felt with PMP, and the difficulty in the decision-making process was more felt with CCs.

This latter concerned the combined nature of proposals.

Finally, the students expressed a strong agreement in recommending the PMP as an assessment tool compared to the other tool

PMPs are specific to the content and their value in medical education has long been reported as well as it is appreciated by students (5 -8).

Here we opted to test the feasibility of PMP in evaluating the discipline of critical medicine for two reasons.

On the one hand, the absence of previous experience with PMPs in acute medicine in the Faculty of Medicine of Tunis neither as formative nor sanctioning evaluation during multiple station tests as is the case in pediatrics for example.

On the other hand we ignore the students' opinion on this technique in ICM: their readiness, the strengths and weaknesses that they think, etc.

The choice of grouped MCQ-CC as a reference tool for the comparison was justified by the fact that this technique aims to improve reasoning by integrating the different components of the situation.

In addition, we often use it with our advanced level external students

Here we can raise a self-criticism.

That concerns the appropriateness of comparing these 2 assessment tools.

Indeed, PMP is a pedagogical tool used to assess competencies of the trainees concerning mainly the clinical reasoning and can be applied at a daily practice.

MCQ-CCs are an assessment tool used to assess the cognitive knowledge of the trainees during the teaching lesson in the faculty.

That’s why; comparing a method used in training departments to a method used in the faculty for the cognitive knowledge might cause confusion.

This can be remedied by the fact that the propositions in the CC were combined in a way to measure higher levels of thinking like problem solving.

On the other hand, the clinical case tool is frequently used during the ICM externship course as assessment technique.

We showed that all the participants passed all tests apart from one student in PMP1 and the scores for the two assessment tools were similar indicating coherence between the theoretical knowledge background and the competence in the management of the clinical situation.

A different result was showed in the Iranian crosssectional study in third-year dental students when the authors tested the same methods that we used (PMP and MCQ-CC) (9).

Indeed, the test scores of students were significantly lower in PMP than in MCQ (43.07±13.18 versus 75.68±12.36, p < 0.001).

The authors concluded that despite good practical information, the students were weak at clinical judgment.

Using assessing methods such as clinical reasoning tests for dental students can be a good way to measure higher abilities (9).

Also, we point out an ''"intriguing'' result: the negative correlation between the CC notes and those of PMP for the 1st theme (r=-0, 58 and p=0.007).

This could reflect the huge gap in data entry; influencing clinical reasoning and the decision-making process between the two techniques.

This perception was clearly felt through the responses to the questionnaire, which greatly favored the PMPs in terms of overall satisfaction, the relevance of the elements provided, the feeling of involvement and the preference for evaluation method.

This underlines the particularity of learning of critical medicine which requires a multidisciplinary approach and the PMP can meet all these expectations.

Similarly to our results, Marquis et al, concluded that PMPs represent motivating and effective technique of continuing medical education for general practitioners (5).

The knowledge acquired was transferred to practice as reported by the participants on a questionnaire (5).

Additionally, PMPs have been shown to facilitate learning when used for assessment purposes (5).

In another study (6), 50 students were divided into two groups: the 1st was taught in a traditional way and the 2nd group was trained using PMP.

Authors concluded that all students in the PMP group agreed that the PMPs focused on common pediatric problems, were clear to them, challenging, encouraged them to develop problem-solving skills, and also helped them with planning for management of common pediatric problems (6).

The study of Ben Abdelaziz R, et al(10) evaluated the contribution of learning by PMP in pediatrics and the perception of students (n=44) for learning and assessment. Authors found that post-test scores statistically improved than pre-test scores and more than 90% of participants thought that PMP was a useful way to learn and would change the way they think (10).

Our results are similar on the impact felt on the learner’s clinical reasoning, which was considered major in 36/40 of cases by combining the two sessions.

We recognize that the impact we examined is instantaneous and cannot be extrapolated to subsequent practice. Esmaeili S and colleagues reassessed their dental students two weeks after the PMP test (11).

The impact was rather marked for the MCQ test with a statistical significant correlation between the student’s educational progress and the score of MCQ test (P < 0.001).

Meanwhile, there were no significant relationship between the educational progress and the scores of PMP- test (11).

This result indicates that the students were weak at reasoning and clinical judgments (11).

In our series, an approval for an assessment method in intensive medicine was significantly in favor of PMP (97.5% versus 35% for the grouped MCQs-CC with p< 10- 3).

PMP was considered more stressful in 68% among participants in the study cited above (10) while it was only in 18/40 (45%) in our study.

In 79% of cases, PMP was a thought as reliable tool of assessment (10).

The major limitation mentioned by the students concerning the grouped MCQs-CC which was effectively the grouped nature of proposals that might influence answers.

The MCQs are among the objective written tests and include several varieties: single complement, multiple complements, simple association type, compound association type, cause and effect type and finally association type of grouped complements (used as reference technique in our study).

To guarantee the success of the CC type association of grouped complements, certain rules must be applied, namely: proposing five supplements, relevant and attractive, homogeneous in form, without mutual exclusions, independent of each other and without upturn (12).

We supposed that all of these requirements were met and verified in our proposed grouped MCQs-CC and the “" negative” feedback perceived by the students was linked to their lack of experience with this tool.

Another study (13) designed to assess improvement in students' ability to respond to consecutive PMP exams, and its relationship to academic progress in 67 3rd-year nutrition students.

The authors concluded that the use of PMP exams in reasoning-based clinical education may be an appropriate approach for the clinical assessment of undergraduate students and may improve students' clinical reasoning (13).

Furthermore, it is important to point out that a new trend is gaining momentum.

These are online PMPs designed with awards for winners to obtain publication credits, registration in journals, etc [14].

This constitutes even more motivation on the part of the students towards this method.

Strengthens of our study consisted on 3 points: firstly, on its originality; given that the use of PMP as a tool of evaluation is lacking in intensive medicine both in formative or summative way.

Secondly, the cross-test design allowed a more objective comparison between the two tested tools.

Thirdly, the choice of 2 critical situations posing major challenges of reflections and decisions which required methodical clinical reasoning with adequate decision-making.

Nevertheless, we recognize two weaknesses: the relatively small number and the degree of subjectivity in the responses to the perception questionnaire

We concluded that the use of PMP in acute medicine discipline was especially appreciated by students during evaluation process.

This positive assessment is linked to the relevance of the elements provided; by giving them a feeling of involvement in the decision-making process and the debriefing (crucial step in the success of any pedagogy centered on student and based on interactivity).

The practice of PMP, in our discipline as an evaluation method, within the Faculty of Medicine of Tunis remains very limited.

This can be explained by the constraints imposed to succeed a PMP: the required time to design it, validate it, a lack of experience and training of certain teachers, etc.

Thus, we suggest the generalization of PMP as a tool of evaluation in acute/intensive medicine discipline both formative during the externship or to integrate it as a station in the objective clinical evaluation test guided by predefined educational objectives and standardized planning between the different intensive care training sites.

Also, the training of teachers for the design and implementation of PMPs through seminars at the section level should be consolidated
==== Refs
References

Jouquan J L’évaluation des apprentissages des étudiants en formation médicale initiale. Pédagogie Médicale 2002 3 1 38 52
McCarthy WH Gonnella JS The simulated Patient Management Problem: a technique for evaluating and teaching clinical competence. Br J Med Educ 1967 1 5 348 52 5583313
Jackson M Cairns T Care of the critically ill patient. Surgery (Oxf) 2021 39 1 29 36 10.1016/j.mpsur.2020.11.002 33776181
Tabbane C Conception et Construction d'un P.M.P (Patient Management Problem). Eléments d'Introduction aux Ateliers de Pédagogie Médicale 2000 135 44
Marquis Y Chaoulli J Bordage G Chabot JM Leclere H Patient-management problems as a learning tool for the continuing medical education of general practitioners. Med Educ 1984 18 2 117 124 10.1111/j.1365-2923.1984.tb00984.x 6700446
Afroza S Use of a PMP manual as a teaching tool to accelerate paediatric teaching in Bangladesh. Med Teach 2000 22 4 365 369
Eisenberg H Gordon S Computer assisted teaching program: a pulmonary patient management problem. Int J Clin Monit Comput 1987 4 4 195 197 3316452
Euvrard GC Poteaux N Le débriefing comme clé de l’accompagnement réflexif en formation et en pratique de l’éducation thérapeutique du patient Pédagogie Médicale 2023 (on line) 10.1051/pmed/20230049
Darmiani S Ebrahimipour S Comparison of Two Methods of Dental Students Assessment (MCQ and PMP) and their correlation with the total grade-point average Journal of Dentomaxillofacial Radiology, Pathology and Surgery 2022 11 1 14 18 http://dx.doi.org/
Ben Abdelaziz R Hajji H Boudabous H Ben Chehida A MradMazigh S Azzouz H Patient-management Problem (PMP) for paediatrics learning: Value and students perceptions Tunis Med 2018 96 1 1 5 30324984
Esmaeili S Sajadi FS Mahmoodi MR Zangiabadi P Comparison of Dental Students’ Ability to Answer PMP-MEQ and MCQ Tests and Its Association with Educational Progress Sch. J. Dent. Sci. 2015 2 5 330 335
Reyna J WRITING EFFECTIVE MULTIPLE-CHOICE QUESTIONS IN MEDICAL EDUCATION INTED Proceedings 2023 10.21125/inted.2023
Mahmoudi MR Do consecutive Patient Management Problem (PMP) and Modified Essay Question (MEQ) Examinations Improve Clinical Reasoning in Students? Strides in Development of Medical Education 2019 In Press In Press e86566 10.5812/sdme.86566
Bruna J Wen PY Patient Management Problem: Self-Assessment and CME Continuum (Minneap Minn) 2015 21 2 535 540 25837913
