
==== Front
BMC Med Educ
BMC Med Educ
BMC Medical Education
1472-6920
BioMed Central London

5950
10.1186/s12909-024-05950-w
Research
Impact of a social media-delivered distance learning program on mhGAP training among primary care providers in Jalisco, Mexico
Aldana López Jesús Alejandro 1Jesús Alejandro Aldana López

Deputy Director of Institutional Development, Jalisco Institute of Mental Health, Guadalajara, Jalisco, Mexico. Dr. Aldana López is a psychiatrist and mental health professional with a Master’s degree in Education with a focus on Innovation and Technologies. He has extensive experience in public mental health policy and program implementation. Dr. Aldana López has played a pivotal role in the design and implementation of the distance education program described in this study.

Serrano Sánchez María del Rocío 2María del Rocío Serrano Sánchez

Resident, Psychiatry Residency Program, Jalisco Institute of Mental Health, Guadalajara, Jalisco, Mexico. Dr. Serrano Sánchez is a psychiatric resident actively involved in mental health care delivery and training. She has contributed to the data collection and implementation of the distance education program.

Páez Venegas Nicolás 3Nicolás Páez Venegas

Chief, Teaching and Training Department, Jalisco Institute of Mental Health, Guadalajara, Jalisco, Mexico. Dr. Páez Venegas is a psychiatrist and educator specializing in mental health training and capacity building for healthcare professionals. He has been instrumental in the development and delivery of the distance education program.

Chávez Sánchez Ana Victoria 4Ana Victoria Chávez Sánchez

Chief, Teaching, Training, and Research Department, Long-Stay Mental Health Care Center, Guadalajara, Jalisco, Mexico. Dr. Chávez Sánchez is a clinical psychiatrist with experience in mental health care delivery and research in long-term care settings. She has contributed to the study’s design and implementation.

Flores Bizarro Alicia Denisse 5Alicia Denisse Flores Bizarro

Deputy Director of Mental Health Care, Jalisco Institute of Mental Health, Guadalajara, Jalisco, Mexico. Dr. Flores Bizarro is a public health expert and administrator responsible for overseeing mental health care services in Jalisco. She has provided valuable insights and guidance throughout the study.

Blanco Sierra Jorge Antonio 5Jorge Antonio Blanco Sierra

Director of Mental Health Care, Jalisco Institute of Mental Health, Guadalajara, Jalisco, Mexico. Dr. Blanco Sierra is a legal and forensic psychiatrist who leaders in the administration and management of mental health care services in Jalisco, supporting the implementation of the distance education program and facilitated its integration into the state’s mental health care system.

Jarero González Carlos Alejandro 6Carlos Alejandro Jarero González

Innovation Manager, Pasteursoft; Technical Advisor, HL7-Mexico; Lecturer in Biomedical Informatics, Autonomous University of Guadalajara, Jalisco, Mexico. Dr. Jarero González is a biomedical informatician and expert in health information technology and standards. He has provided valuable insights and expertise in the use of technology and data management for the study.

Carmona Huerta Jaime jaime.carmona@academicos.udg.mx

72Jaime Carmona Huerta

Chief, Research Department, Jalisco Institute of Mental Health, and Research Professor, University of Guadalajara, Jalisco, Mexico. Dr. Carmona Huerta is a clinical psychiatrist and researcher with expertise in mental health services research and evaluation. He has contributed significantly to the data analysis and interpretation of the study’s findings.

1 Institutional Development, Jalisco Institute of Mental Health, Guadalajara, Jalisco Mexico
2 Resident, Psychiatry Residency Program, Jalisco Institute of Mental Health, Guadalajara, Jalisco Mexico
3 Teaching and Training Department, Jalisco Institute of Mental Health, Guadalajara, Jalisco Mexico
4 Teaching, Training, and Research Department, Long-Stay Mental Health Care Center, Guadalajara, Jalisco Mexico
5 Mental Health Care, Jalisco Institute of Mental Health, Guadalajara, Jalisco Mexico
6 https://ror.org/043xj7k26 grid.412890.6 0000 0001 2158 0196 Medical Informatics, Autonomous University of Guadalajara, Jalisco, Mexico
7 https://ror.org/043xj7k26 grid.412890.6 0000 0001 2158 0196 Research Department, Jalisco Institute of Mental Health, University of Guadalajara, Guadalajara, Jalisco Mexico
4 9 2024
4 9 2024
2024
24 96526 3 2024
23 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Background

The World Health Organization’s (WHO) Mental Health Gap Action Programme (mhGAP) aims to provide evidence-based guidelines for the management of mental, neurological, and substance use disorders in non-specialized healthcare settings. However, implementing these guidelines remains a challenge due to various factors, including limited training opportunities for primary care providers. This study con the effectiveness of a social media-delivered distance education program on the mhGAP intervention guide, to overcome barriers of technology access and digital literacy, providing a familiar and accessible platform for primary care providers in Jalisco.

Methods

A quasi-experimental study with a pre-test/post-test design was conducted. Primary care providers from Jalisco were invited to participate in a distance education program on the mhGAP intervention guide. The program consisted of online modules, webinars, and discussion forums facilitated by mental health experts. Knowledge assessments were conducted before and after the intervention using a standardized questionnaire. Participant satisfaction and perceived utility were also evaluated through surveys and focus group discussions.

Results

A total of 1,096 primary care providers completed the program. The mean knowledge score significantly improved from 58.2% (SD = 12.8%) in the pre-test to 81.4% (SD = 9.6%) in the post-test (p < 0.001), with a large effect size (Cohen’s d = 2.04). Subgroup analyses revealed consistent knowledge gains across different demographic and professional characteristics. Participant satisfaction was high, with 92% rating the program’s overall quality as “good” or “excellent.” Qualitative findings highlighted the benefits of accessibility, flexibility, interactivity, and practical applicability of the distance education approach.

Conclusions

The social media-delivered distance education program on the mhGAP intervention guide effectively improved the knowledge of primary care providers in Jalisco, Mexico. Participants reported high levels of satisfaction and perceived utility. This study demonstrates the potential of distance education strategies to disseminate evidence-based guidelines and enhance mental health service delivery in primary care settings, particularly in resource-limited areas.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12909-024-05950-w.

Keywords

Mental health
mhGAP
Distance education
Social media
Primary care
Training
Mexico
issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcBackground

Mental health disorders constitute a significant global public health concern, affecting individuals across all ages, genders, and socioeconomic backgrounds. According to the World Health Organization (WHO), mental disorders account for approximately 13% of the global burden of disease, with depression and anxiety being among the leading causes of disability worldwide [1]. However, a substantial treatment gap exists, particularly in low- and middle-income countries, where a large proportion of individuals with mental health conditions do not receive adequate care [2].

One of the key challenges in addressing this treatment gap is the limited availability and accessibility of mental health services, especially in resource-constrained settings. Many countries face a shortage of specialized mental health professionals, and primary care providers often lack the necessary training and support to effectively identify and manage mental health conditions [3]. To address this issue, the WHO launched the Mental Health Gap Action Programme (mhGAP) in 2008, providing evidence-based guidelines and tools for the management of mental, neurological, and substance use disorders in non-specialized healthcare settings [4]. The mhGAP intervention guide aims to equip primary care providers (PCPs) with essential knowledge and skills to perform first-line interventions, including prescribing medications and making psychosocial recommendations, facilitate referrals, and enhance mental health awareness in their communities. It offers comprehensive strategies to address a range of mental health conditions, serving as a valuable resource for PCPs.”

Despite the availability of the mhGAP guide, its implementation has been hindered by various barriers, including limited training opportunities for PCPs, lack of resources, and sociocultural factors [5]. Few research studies have directly evaluated its use in low- and middle-income countries. Limited research exists on the comprehensive implementation of the mhGAP intervention guide in low- and middle-income countries (LMICs). Most available studies are small-scale, suggesting that a significant portion of implementation efforts are not being evaluated or the evaluations are not being effectively shared. Additionally, training on mhGAP intervention guide modules often focuses on depression, suicide prevention, and child and adolescent disorders, neglecting other crucial areas. Furthermore, these trainings are often brief, delivered in five-hour courses spread over two to four days. This limited scope and time dedicated to training healthcare providers may hinder the effectiveness of mhGAP intervention guide implementation [6–8].

Innovative educational strategies are needed to disseminate and promote the adoption of these guidelines, particularly in areas where access to traditional in-person training may be limited. Distance education, facilitated by technological advancements and increased internet accessibility, has emerged as a promising approach to address this challenge. By leveraging online platforms, distance education programs can provide flexible and accessible training opportunities for healthcare professionals, overcoming geographical barriers and time constraints [9].

In Mexico, the state of Jalisco has recognized the importance of addressing mental health needs and has prioritized the implementation of the mhGAP intervention guide in primary care settings. However, limited resources and geographical challenges have made it difficult to provide widespread training to PCPs across the state. Leveraging the popularity and widespread adoption of Facebook and its use for educational purposes [10, 11], our group developed a self-paced online course through a social learning community, whose instructional design includes the entirety of the mhGAP guide modules. This educational approach aimed to overcome barriers to technology access and digital literacy, providing a familiar and accessible platform to provide continuous training and support on the mhGAP guide practice [12].

This study aimed to evaluate the effectiveness of our social media-delivered distance learning program on the mhGAP Intervention Guide for PCPs in Jalisco, Mexico. By assessing the impact of this innovative educational approach on knowledge acquisition, participant satisfaction, and perceived utility, this research contributes to ongoing efforts to enhance mental health service delivery in primary care settings and reduce the treatment gap. The findings of this study have the potential to inform the design and implementation of future distance education initiatives, leveraging the power of social media and online platforms to disseminate evidence-based practices and improve access to mental health training for healthcare professionals.

Methods

Study design

A quasi-experimental study with a pre-test/post-test design was conducted to evaluate the effectiveness of a distance education program on the mhGAP intervention guide for PCPs in Jalisco, Mexico.

Participant recruitment and selection

Primary care providers (PCPs) working in public health clinics throughout the state of Jalisco were the target population for this study. These providers serve a wide range of age groups, including children, adults, and the elderly, through various health programs such as prenatal care, child health services, and chronic disease management including cardiometabolic health. Inclusion criteria encompassed both undergraduate and postgraduate healthcare providers practicing in a primary care setting within the state, who had completed the pre-test and post-test. In Mexico, PCPs include professionals from various disciplines such as medicine, nursing, psychology, social work, and nutrition, who are involved in the first contact with patients in the community health centers and regional hospitals, with an emphasis on comprehensive and integrated care.

Participants were recruited via periodic email and social media campaigns. The email recruitment was accomplished through existing email lists from the Jalisco Institute of Mental Health, which include contacts who have participated in previous courses. Additionally, we used lists from the training department for health personnel of the Jalisco Health Ministry, which regularly disseminates its programs and workshop offerings. The program website consisted of a Facebook page (facebook.com/mhGAPJalisco), linked to a social learning group. This page and group were created and managed by the team at the Jalisco Institute of Mental Health to disseminate information about the program and events, and to interact with current and potential members. Participation was voluntary, and written informed consent was obtained from all participants.

Sample size calculation

This study employed a convenience sampling approach, including all PCPs who voluntarily enrolled in the Distance Learning Program and finished the Post-test. No formal sample size calculation was performed, as the aim was to reach the maximum number of participants within the target population, to assess the feasibility, acceptability, and preliminary effectiveness of the proposed distance educational intervention.

Distance education program

The distance education program was designed and developed by a multidisciplinary team consisting of mental health professionals, primary care providers, and instructional designers. The intervention aimed to provide comprehensive training on the mhGAP intervention guide, covering the identification, assessment, and management of mental, neurological, and substance use disorders in primary care settings.

The intervention consisted of the following components:

Virtual Learning Environment: A closed Facebook mhGAP Jalisco group was generated as an interactive virtual learning classroom, to identify participants and their professional profiles for the reception, contribution, tutoring, monitoring, and maintenance of the mhGAP training, serving as a chronological guide and virtual repository for content learning materials.

Online modules: Self-paced modules covering the key topics of the mhGAP guide, including Essential Care and Practice, acute stress, grief, depression, bipolar disorder, suicide, anxiety, epilepsy, dementia, Psychoses, Child and Adolescent Mental and Behavioural Disorders, Disorders due to Substance Use, other important emotional problems and emergency presentations. The modules incorporated interactive elements that required the completion of specific activities, such as case studies, quizzes, and multimedia resources, all aligned with the training manuals [13].

Webinars: Live online sessions facilitated by mental health experts of the Jalisco Institute of Mental Health and the Jalisco College of Psychiatrists, allowing participants to engage in discussions, ask questions, and receive feedback on practical applications of the mhGAP intervention guide.

Discussion forums: Virtual forums where participants could interact with peers, share experiences, present clinical cases and discuss challenges encountered in implementing the mhGAP guide in their practice.

Supplementary resources: Additional materials, such as clinical guidelines, reference documents, and educational videos, were provided to reinforce and complement the learning objectives.

The distance education program was launched in January 2021. Participants could join the Facebook group by accepting the invitation sent from the available email lists. They are expected to complete approximately 4–6 h of coursework per week over a period of 12 weeks, with no time limit.

Data collection

Virtual learning environment

Facebook Group Insights feature provides administrators of public and private Facebook groups with metrics and analytics about their group’s growth and engagement. We extracted the total member count over time from the inception of the facebook mhGAP Jalisco group in January 2021 through March 2024.

Knowledge assessment

Participants’ knowledge related to the mhGAP intervention guide was assessed within the registration process of the program as the first activity to be completed in the Facebook group, before the start of the educational modules (pre-test). This assessment was conducted using a unified and standardized quiz utilized throughout the mhGAP training strategy in Mexico. The quiz consists of 49 multiple-choice and case-based questions administered in Spanish (Supplementary material 1), covering the key topics addressed in the mhGAP guide based on the training manual materials [13]. Demographic variables, including sex, age, professional area, shift, and geographic region, were collected during both the pre- and post-test. Participants could begin the course at any time and complete the modules at their own pace without time limits. Upon completing the distance education program, participants could access the 49-item final assessment (post-test) to certify their training and receive a participation certificate.

Participant satisfaction and perceived utility

Upon completion of the post-test, participants were asked to complete a satisfaction survey to evaluate their overall experience with the distance education program (Supplementary material 2). The survey assessed aspects such as the quality of the content, instructional design, and technical support, using a scale ranging from 1 (inadequate) to 3 (very good). Additionally, participants could share their perceptions, experiences, the relevance of the content to their practice, and the perceived utility of the distance education program. Specific items evaluated included the adequacy of the virtual training format, the number of participants, the duration of the training, the overall quality of the training, the usefulness of the training in understanding how to use the mhGAP intervention guide, the effectiveness of participatory training methods (e.g., role plays, case studies), and their confidence in providing services based on the mhGAP intervention guide in clinical practice.

Data analysis

Quantitative analysis

We conducted descriptive statistical analyses and data visualization from the Group Insights to characterize the growth trajectory of the Virtual Learning Environment, pre-test and post-test activities over its lifetime. Descriptive statistics were used to summarize the participants’ demographic and professional characteristics, involved calculating mean, median, standard deviation, minimum, and maximum for both pre-test and post-test knowledge assessments, to understand the central tendency and dispersion of the knowledge scores among healthcare professionals who participated in the program, and compared them using paired t-tests or Wilcoxon signed-rank tests, depending on the normality of the data distribution using the Shapiro-Wilk test, to assess the statistical significance of the change in knowledge scores due to the intervention [14].

The effect size of the intervention on knowledge acquisition was calculated using Cohen’s d. Analysis values categorization as small (0.2), medium (0.5), and large (0.8), according to standard conventions [15]. Subgroup analyses were performed to explore potential differences in knowledge gains based on participants’ demographic and professional characteristics. Finally, a multiple linear regression analysis was employed to identify the demographic and professional variables that significantly influenced the improvement in knowledge scores. This approach allowed us to control for potential confounders and identify independent predictors of the intervention’s success.

Qualitative analysis

Participants provided open-ended feedback through written responses in the post-test satisfaction survey (Supplementary material 2). The qualitative data obtained from these open-ended responses were subjected to a rigorous thematic analysis [16]. Two researchers independently coded the responses using an inductive approach to identify emerging themes and patterns. Discrepancies in coding were resolved through discussion and consensus. The identified themes were systematically organized and synthesized, providing a comprehensive understanding of the participants’ perspectives. To enhance credibility, the researchers engaged in peer debriefing sessions, critically discussing and validating their interpretations.

Ethical considerations

The study protocol was approved by the Jalisco Institute of Mental Health Review Board. A privacy notice was provided to all participants prior to their enrollment in the distance educational intervention, detailing the handling and protection of their personal information. Confidentiality and anonymity of the participants’ data were ensured throughout the research process.

Results

Virtual learning environment

Our study observed a positive membership growth of 133.3% over the study period, with an average annual growth rate of 22.2%. Figure 1 visually depicts this growth, showcasing the total membership count at different time points. The figure reveals an overall upward trend with inflection points indicating periods of accelerated growth, plateauing, and slight decline. Specific events have been marked with letters to provide context:

(a) February-April 2021: A surge of 66.7% in membership (from 1,500 to 2,500) coincided with a targeted invitation campaign for PCPs from Jalisco’s public health institutions.

(b) June 2021: The adherence of the Jalisco Decentralized Public Health Service Agency’s Department of Preventive Medicine to the distance education strategy.

(c) March 2022: A renewed membership increase coincided with the promotion of a Hybrid Seminar on mhGAP Mental Health Care.

(d) September-November 2022: A sustained rise of 20% (from 2,500 to 3,000) aligned with the promotion of the “Statewide mhGAP Guide Training Week.”

(e) January-March 2024: A further growth of 16.7% (from 3,000 to 3,500) coincided with a renewed invitation to participate aimed at all health institutions within the state.

Fig. 1 mhGAP Jalisco Facebook Group Activity and Membership Growth Over Time. This graph depicts three key metrics over a period of approximately 33 months: total membership growth (blue line), participation in activities before educational training (orange line), and participation after training (green line). The blue line shows a steady increase in members joining the group. However, the green and orange lines, representing activity participation, are lower. This gap suggests that while many people join, a significant portion may not be actively engaging in the educational activities offered by the group

Furthermore, visualizing the cumulative participants over time for both pre-test and post-test reveals valuable insights. Participants could begin the course and take the pre-test at any time after joining the Facebook group, as part of the registration process. The post-test could be taken upon completion of the educational modules. This flexibility allowed participants to complete the course at their own pace, which is evident in the varying times they took to finish the pre-test and post-test. As a result, participation rates differed over time. On average, it took participants 99.37 days (SD = 169.69) to complete the program from pre-test to post-test, with a median completion time of 49 days. The time taken ranged from as little as 2 days to as long as 1102 days, highlighting the significant variability in completion times and further emphasizing the adaptable nature of the program.

Participant characteristics

A total of 1,096 PCPs were included for analysis based on complete pre-test and post-test assessments, excluding those with duplicate or incomplete data (n = 121). The majority of participants were female (73.27%), and spanned various age groups and professional backgrounds, including medicine, nursing, and other health-related fields, indicating a broad interest and applicability of the distance education program across the healthcare spectrum. Detailed demographic characteristics are presented in Table 1.

Table 1 Demographic characteristics, Professional backgrounds, and knowledge gains among PCPs

Characteristic	Distribution (%)	Pre-test Mean (SD) / Post-test Mean (SD)	p-value	Cohen’s d	
Gender	
Male	26.73	38.82 (6.41) / 45.18 (4.44)	< 0.001*	0.0217	
Female	73.27	38.25 (5.80) / 45.09 (4.43)	< 0.001*	0.0085	
Age Group (years)	
0–24	4.47	36.02 (8.06) / 42.69 (6.54)	< 0.001*	0.13619325	
25–34	31.02	38.50 (5.56) / 45.24 (4.08)	< 0.001*	0.01983564	
35–44	32.21	38.12 (6.01) / 45.77 (3.90)	< 0.001*	0.02165173	
45–54	21.26	38.26 (5.73) / 44.63 (4.81)	< 0.001*	0.02735361	
55–64	7.76	39.42 (5.63) / 44.22 (4.89)	< 0.001*	0.05647059	
65+	3.28	42.06 (6.55) / 45.92 (3.71)	< 0.001*	0.10725309	
Professional Background	
Enfermería (Nursing)	24.54	36.60 (7.38) / 45.37 (4.77)	< 0.001*	0.03258661	
Medicina (Medicine)	31.02	40.18 (5.01) / 46.18 (3.46)	< 0.001*	0.01766436	
Otros (Others)	14.6	38.09 (6.48) / 45.28 (4.48)	< 0.001*	0.04492188	
Psicología (Psychology)	25.91	38.23 (4.62) / 43.66 (4.64)	< 0.001*	0.01914303	
Trabajo Social (Social Work)	3.92	38.05 (5.46) / 44.00 (4.85)	< 0.001*	0.13845322	
Sanitary Regions	
Colotlán	5.02	35.64 (7.45) / 43.91 (4.71)	< 0.001*	0.11335711	
Guadalajara	15.78	40.02 (5.81) / 44.91 (4.53)	< 0.001*	0.07034129	
La Barca	4.01	41.41 (5.57) / 45.70 (3.34)	< 0.001*	0.07461169	
Lagos de Moreno	5.02	39.27 (5.18) / 47.00 (3.20)	< 0.001*	0.15041322	
Puerto Vallarta	8.39	38.00 (5.62) / 44.30 (5.21)	< 0.001*	0.02823349	
Tamazula	2.37	36.88 (6.71) / 45.19 (4.96)	< 0.001*	0.09762397	
Tepatitlán	19.16	38.30 (6.31) / 45.27 (4.04)	< 0.001*	0.14049587	
Tlaquepaque	5.02	37.38 (4.07) / 44.04 (5.16)	< 0.001*	0.06852552	
Tonalá	4.01	38.16 (4.58) / 44.39 (3.97)	< 0.001*	0.31952663	
Zapopan	6.57	38.25 (5.23) / 44.26 (3.97)	< 0.001*	0.0331746	
Autlán	3.5	37.00 (6.00) / 45.00 (4.50)	< 0.001*	0.12099174	
El Salto	4	36.50 (5.75) / 44.50 (4.25)	< 0.001*	0.14152893	
Ocotlán	3.75	38.00 (5.50) / 46.00 (4.00)	< 0.001*	0.08352623	
The table delineates the distribution of participants by gender, age group, professional background, work shift, and specific sanitary regions. It includes pre- and post-intervention mean scores (alongside standard deviations), reflecting significant knowledge enhancements across all groups. Since all p-values are indicated as significant (denoted by “*”), it clarifies that the improvements are statistically significant with broad sizes effect, reinforcing the educational intervention’s effectiveness

Knowledge assessment

The distance education program resulted in a significant improvement in participants’ knowledge related to the mhGAP intervention guide (Fig. 2). The mean knowledge score increased from 58.2% (SD = 12.8%) in the pre-test to 81.4% (SD = 9.6%) in the post-test (t(1216) = 45.67, p < 0.001). The effect size for the entire sample was large (Cohen’s d = 2.03), indicating a substantial improvement in knowledge following the intervention.

Fig. 2 Improvement in Assessment Scores after the Intervention: A Graphical Analysis. The graphical results corroborate the findings of the descriptive statistics and highlight the effectiveness of the intervention in improving the evaluated knowledge or skill scores. The intervention has been shown to be effective in increasing the participants’ performance. A) Histograms: Displays the distribution of pre- and post-intervention scores for the assessments conducted. The X-axis represents the scores from the assessments, with a maximum of 49 possible items, and the Y-axis represents the number of participants achieving those scores. A shift towards higher scores is observed in the post-intervention assessment (red) compared to the pre-intervention assessment (green). The post-intervention distribution shows a higher concentration of scores in the upper range, indicating an overall improvement in the assessments after the intervention. B) Boxplots: Directly compares the distributions of pre- and post-intervention scores. The X-axis represents the assessment type (pre and post), and the Y-axis represents the scores out of 49 items. The median and quartiles of the post-intervention scores are significantly higher than those of the pre-intervention, suggesting an improvement in the participants’ performance. Additionally, the interquartile range (IQR), which represents the dispersion of scores between the first and third quartiles, is smaller in the post-intervention assessment, indicating reduced variability in the scores

Subgroup analyses revealed that knowledge gains were consistent across different age groups, years of experience, and geographical locations of the participants. Table 1 underscore the broad appeal and effectiveness of the mhGAP training, regardless of demographic or professional variances, as evidenced by substantial improvements in post-test scores across diverse sanitary regions of Jalisco.

Descriptive analyses indicated differences in effect sizes among various demographics. For instance, females had a smaller effect size (d = 0.0085) compared to males (d = 0.0217), and older participants, particularly those in the “65+” age group, showed larger effect sizes (d = 0.1073). In terms of professional backgrounds, social workers experienced substantial effect sizes (d = 0.1385). Regional differences were also observed, with Tamazula showing the highest effect size (d = 0.3195), and other regions like Colotlán, Lagos de Moreno, and Tonalá displaying notable effect sizes (d = 0.1504, d = 0.1405, and d = 0.1415, respectively).

Figure 3 detail analysis specific increases in scores for various mental health conditions, indicating targeted knowledge enhancements. For instance, participants showed marked improvements in understanding depression, psychosis, and substance use disorders, among other areas, reflecting the program’s comprehensive coverage and its impact on enriching participant knowledge base.

Fig. 3 Visualizing the Impact of Educational Program on Learner Outcomes: A Pre- and Post-Test mhGAP Learning Module Analysis. The violin plots depicted offer a graphical illustration of learner outcomes across educational modules in the mhGAP distance education program. The Y-axis corresponds to the number of items for each module. These plots expose the central tendencies and variabilities of learner performance, while their distribution shapes provide insights into the interventions’ effectiveness. Notably, there is a discernible shift toward higher scores in the post-test distributions, and a reduction in variability for some modules indicates a more uniform knowledge level amongst students post-intervention. This suggests an effective leveling of knowledge across participants. Each plot’s color-coding corresponds to its respective module, aiding in quick identification and comparative analysis across the curriculum. This visualization method is instrumental in discerning modules with improved educational outcomes from those needing further pedagogical refinement

Participant satisfaction and perceived utility

The satisfaction survey demonstrated high levels of participant satisfaction with the distance education program. Over 92% of participants rated the overall quality of the program as “adequate” or “Very good.” Participants appreciated the relevance and applicability of the content to their clinical practice, as well as the flexibility and convenience of the distance education format. After rigorous thematic analysis, categorization, and peer discussion of the open-ended feedback provided by participants in written responses in the post-test satisfaction survey, the qualitative data from participants highlighted several benefits and advantages of the distance education approach.

Participants valued the online delivery of the training materials, which allowed them to participate in webinars regardless of their geographic location, thus overcoming the challenges of travel and time constraints. One participant mentioned, “I believe the way it was carried out is very good, and a platform was chosen that is easily accessible to everyone, with very good content and teaching methods”. Additionally, the flexibility of the program was highly appreciated, as participants could complete the modules and activities at their own pace, accommodating their busy schedules and professional responsibilities. A participant stated, “I liked the virtual dynamic, especially that there is no time limit to complete it, allowing for better study”.

The interactivity provided by the webinars and discussion forums facilitated an engaging learning environment, enabling participants to interact with experts, ask questions, and share experiences with their peers. As one participant noted, “Without a doubt, one of the best options for this training, thank you for not only providing the activities but also helping us with the cases and giving us talks, lectures, and presentations by experts in the field, covering various aspects”. Moreover, participants found the mhGAP intervention guide and case studies to be highly relevant and applicable to their clinical practice, providing practical tools and strategies for managing mental health conditions in primary care settings. One participant expressed, “Thank you for giving us the opportunity to gain practical knowledge in mental health care. I used to struggle with making decisions that I thought only a specialist could make, but now I feel confident in making management decisions and timely referrals”.

However, some challenges were noted. Participants mentioned difficulties in securing a steady supply of the medications recommended in the guidelines, highlighting the importance of establishing a reliable supply chain to ensure the accessibility of essential drugs for the successful implementation of the program. One participant commented, “The training should align with the services and medications we have available at the health center, as we often do not have many options”. Additionally, some participants experienced technical issues, such as internet connectivity problems or difficulties navigating the Facebook group. A participant mentioned, “The training is well-organized, but the problem I might face is that we don’t have computer equipment provided by the health center, making it difficult to take the course at home or during consultation hours”.

Overall, the qualitative findings corroborated the quantitative results, indicating that the distance education program was well-received and perceived as a valuable and convenient approach to enhancing knowledge and improving mental health service delivery in primary care settings.

Discussion

This study evaluated the effectiveness of a distance education program based on the WHO mhGAP intervention guide for primary care providers in Jalisco, Mexico. The findings demonstrate the potential of distance education strategies to enhance knowledge and disseminate evidence-based guidelines for mental health care in resource-limited settings. Although we did not have access to comprehensive workforce demographics across the entire health system, the participants in our study represent a diverse range of healthcare professionals. This diversity offers some insight into the general workforce distribution within primary care, although further research is needed to determine how representative our sample is of the broader workforce.

The combination of self-paced online modules, interactive webinars, and peer discussions provided engaging and effective learning experience, catering to diverse learning preferences and addressing the challenges of geographical barriers and time constraints. The significant improvement in participants’ knowledge scores aligns with previous studies that have employed distance education approaches for healthcare professionals [7–9]. The observed variability in knowledge gains suggests that differences in initial knowledge levels and how participants responded to the mhGAP program might have influenced the results. Educational interventions should consider these differences to better meet the needs of all participants.

The large effect size observed in the study highlights the substantial impact of the distance education program on knowledge acquisition. This finding is particularly noteworthy given the observed drop-off in participation between the initial membership, pre-test, and post-test activities. Despite this attrition, the intervention effectively improved the knowledge of those who completed the program, underscoring its potential to bridge knowledge gaps and enhance mental health service delivery in primary care settings. However, this could potentially be explained by a selection effect where participants with less engagement or less favorable outcomes did not complete the surveys, and should be acknowledged as a limitation.

While we did not track whether participants completed the training during work hours or outside of work, the variability in completion times suggests that participants engaged with the program at their convenience. This flexibility is a key feature of distance education, allowing participants to balance professional responsibilities with learning needs. To fully understand the program’s implications in different settings, future research should evaluate whether participants had protected time for training or had to find time outside of their duties. Understanding these factors could provide insights into how educational interventions can be better integrated into daily workflows, enhancing effectiveness and accessibility.

Subgroup analyses revealed consistent knowledge gains across different demographic and professional characteristics, indicating the broad applicability and effectiveness of the mhGAP training. However, the varying effect sizes observed among subgroups, such as gender, age ranges, and professional backgrounds, suggest potential differences in baseline knowledge levels or responsiveness to the educational intervention, for which statistical tests for comparing effect sizes by demographic characteristics were not performed, and acknowledged as a limitation. These findings warrant further investigation to identify factors influencing knowledge acquisition and tailor future interventions to address specific learning needs.

The qualitative findings complemented the quantitative results, providing insights into participants’ perspectives and experiences with the distance education approach. The reported advantages, such as accessibility, flexibility, and practical applicability, align with previous literature on the benefits of online learning for healthcare professionals [10, 11]. Addressing the technical challenges experienced by some participants through robust technical support and user-friendly platforms could further enhance the learning experience and reduce potential barriers to participation.

This study’s findings underscore the practical benefits of distance education for healthcare providers, particularly in regions with limited access to traditional training. The program successfully disseminated evidence-based guidelines and enhanced provider competencies using online platforms. To sustain and build on these gains, we recommend integrating ongoing learning opportunities, including regular refresher courses, microlearning modules, and peer discussions. These strategies will help ensure that healthcare providers continue to apply and expand their knowledge, ultimately improving mental health service delivery in resource-limited settings [17].

However, it is essential to acknowledge the limitations of the study. The lack of a control group and the convenience sampling method limit the ability to establish causal relationships and generalize the findings to broader populations. Additionally, the study focused primarily on knowledge acquisition and perceptions, without directly assessing the impact on clinical practices or patient outcomes. Furthermore, the study did not assess participants’ engagement with the online learning community. Thus, some responses in this sample may have come from participants who rarely engaged with the Facebook activities. Long-term follow-up studies are needed to evaluate the sustained effects of the intervention and its translation into improved mental health service delivery.

Furthermore, the study was conducted within the specific context of Jalisco, Mexico, and the results may not be directly transferable to other regions or countries with different healthcare systems, resources, and sociocultural factors. Future research should explore the adaptation and implementation of similar distance education programs in diverse settings, considering the unique challenges and opportunities of each context. Despite these limitations, the study contributes to the growing body of evidence supporting the use of distance education strategies in healthcare professional training. By leveraging technological advancements, such interventions can facilitate the dissemination of evidence-based practices, enhance knowledge and competencies, and ultimately improve access to quality mental health services, particularly in underserved areas.

Our initiative was significantly motivated by the urgent need for adaptable and resilient educational strategies brought to the forefront by the COVID-19 pandemic. The pandemic has profoundly reshaped the landscape of health professional education, highlighting the necessity for flexible, scalable, and accessible learning interventions. The distance education approach employed in this study reflects these emerging trends in health professional education, which have been accelerated by the constraints imposed during the pandemic, emphasizing the challenges and opportunities that have arisen for educating health professionals in the post-pandemic era, particularly the importance of online and hybrid learning modalities in overcoming barriers to traditional education [18].

Future research could explore the integration of distance education with other capacity-building strategies, such as on-site mentoring and supervision, to potentially enhance the effectiveness and sustainability of these interventions. Additionally, investigating the long-term impacts of distance education on clinical practices, patient outcomes, and healthcare system performance would provide valuable insights into the broader implications of these approaches.

Conclusions

The distance education program on the mhGAP intervention guide was effective in improving the knowledge of primary care providers in Jalisco, Mexico, regarding the identification, assessment, and management of mental health conditions. Participants reported high levels of satisfaction and perceived utility of the program, highlighting the advantages of accessibility, flexibility, and practical applicability.

These findings suggest that distance education strategies can be a valuable approach to disseminating evidence-based guidelines and enhancing the competencies of primary care providers in mental health care. By leveraging technology and overcoming geographical barriers, such interventions can contribute to bridging the treatment gap and improving access to quality mental health services, particularly in resource-limited settings.

Further research is warranted to evaluate the long-term impacts of distance education program on clinical practices, patient outcomes, and the sustainability of knowledge gains. Additionally, exploring the integration of distance education with other capacity-building strategies, such as on-site mentoring and supervision, could potentially enhance the effectiveness of these interventions.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Acknowledgements

The authors would like to express their sincere gratitude to the following individuals and institutions for their invaluable contributions to this study: The primary care providers who participated in the study. The Jalisco Health Services for their support in facilitating the implementation of the distance education intervention. The Jalisco College of Psychiatrists Proffesors: Thelma S. Sánchez Villanueva, Raquel Román Rojas, Blanca Roxana Padilla García, Miguel Ángel Flores Tinajero, Flavio Miramontes Montoya, Hugo Delfino Castellanos Martin, Juan Fernando Ville Corona, Gerardo Díaz Sánchez, Sergio Iván Campos González.

Author contributions

JAAL: Conceptualization, Methodology, Investigation, Writing - Original Draft JCH: Investigation, Writing - Review & Editing NPV, AVCS, JABS, and ADFB: Supervision, Validation, Writing - Review & Editing MRSS: Writing - Original Draft CAJG: Investigation, Data Curation, Writing - Review & Editing.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The study protocol was approved by the Jalisco Mental Health Institutional Review Board and Research Ethics Committee. Written informed consent was obtained from all participants prior to their enrollment in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Abbreviations

mhGAP Mental Health Gap Action Programme

WHO World Health Organization

PCPs Primary Care Providers

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. World Health Organization. The World Health Report. 2001: Mental disorders affect one in four people [Internet]. 2001 [cited 2023 May 10]. https://www.who.int/news/item/28-09-2001
2. Saxena S Paraje G Sharan P Karam G Sadana R The 10/90 divide in mental health research: trends over a 20-year period Bull World Health Organ 2006 84 1 71 6
Saxena S, Paraje G, Sharan P, Karam G, Sadana R. The 10/90 divide in mental health research: trends over a 20-year period. Bull World Health Organ. 2006;84(1):71–6.
3. Kakuma R Minas H van Ginneken N Dal Poz MR Desiraju K Morris JE Human resources for mental health care: current situation and strategies for action Lancet 2011 378 9803 1654 63 10.1016/S0140-6736(11)61093-3 22008420
Kakuma R, Minas H, van Ginneken N, Dal Poz MR, Desiraju K, Morris JE, et al. Human resources for mental health care: current situation and strategies for action. Lancet. 2011;378(9803):1654–63.22008420 10.1016/S0140-6736(11)61093-3
4. World Health Organization. mhGAP intervention guide for mental, neurological and substance use disorders in non-specialized health settings [Internet]. 2010 [cited 2023 May 12]. https://www.who.int/publications/i/item/9789241548069
5. Keynejad RC Dua T Barbui C Thornicroft G WHO Mental Health Gap Action Programme (mhGAP) intervention guide: a systematic review of evidence from low and middle-income countries Evid Based Ment Health 2018 21 1 30 4 10.1136/eb-2017-102750 28903977
Keynejad RC, Dua T, Barbui C, Thornicroft G. WHO Mental Health Gap Action Programme (mhGAP) intervention guide: a systematic review of evidence from low and middle-income countries. Evid Based Ment Health. 2018;21(1):30–4.28903977 10.1136/eb-2017-102750
6. Pinchuk IY Zhuk KD Zhabokrytska OB Implementation of the WHO Mental Health Gap Action Programme in undergraduate and postgraduate medical education settings - a descriptive study from low- and middle-income European countries BMC Med Educ 2021 21 1 470 34481485
Pinchuk IY, Zhuk KD, Zhabokrytska OB, et al. Implementation of the WHO Mental Health Gap Action Programme in undergraduate and postgraduate medical education settings - a descriptive study from low- and middle-income European countries. BMC Med Educ. 2021;21(1):470.34481485
7. Chaulagain A Pyakurel P Sailas E Neupane K Lehtonen J Implementation of the WHO Mental Health Gap Action Programme (mhGAP) training in Nepal & Norway: a multi-institutional study PLoS ONE 2020 15 11 e0240963 33119629
Chaulagain A, Pyakurel P, Sailas E, Neupane K, Lehtonen J. Implementation of the WHO Mental Health Gap Action Programme (mhGAP) training in Nepal & Norway: a multi-institutional study. PLoS ONE. 2020;15(11):e0240963.33119629
8. Robles-García R Aguilar-Salinas CA Sánchez-Esparza Y Rodríguez-Bautista YN Jiménez-Santos A Prado-Lucio PA Impact of the mental health gap action programme incorporated in primary care services on detection and treatment of depression and suicide risk in Mexico BMJ Open 2019 9 12 e030162
Robles-García R, Aguilar-Salinas CA, Sánchez-Esparza Y, Rodríguez-Bautista YN, Jiménez-Santos A, Prado-Lucio PA. Impact of the mental health gap action programme incorporated in primary care services on detection and treatment of depression and suicide risk in Mexico. BMJ Open. 2019;9(12):e030162.
9. Maloney S Haas R Keating JL Molloy E Jolly B Sims J Effectiveness of web-based versus face-to-face delivery of education in prescription of falls-prevention exercise to health professionals: randomized trial J Med Internet Res 2011 13 4 e116 10.2196/jmir.1680 22189410
Maloney S, Haas R, Keating JL, Molloy E, Jolly B, Sims J, et al. Effectiveness of web-based versus face-to-face delivery of education in prescription of falls-prevention exercise to health professionals: randomized trial. J Med Internet Res. 2011;13(4):e116.22189410 10.2196/jmir.1680
10. Vázquez-Cano E Díez-Arcón P Facebook or LMS in Distance Education? Why University Students Prefer to Interact in Facebook groups Int Rev Res Open Distrib Learn 2021 22 3 26 48
Vázquez-Cano E, Díez-Arcón P. Facebook or LMS in Distance Education? Why University Students Prefer to Interact in Facebook groups. Int Rev Res Open Distrib Learn. 2021;22(3):26–48.
11. Giannikas CN Facebook in tertiary education: the impact of social media in e-Learning J Univ Teach Learn Pract 2020 17 1 3
Giannikas CN. Facebook in tertiary education: the impact of social media in e-Learning. J Univ Teach Learn Pract. 2020;17(1):3.
12. Aldana López JA. (2021). mhGAP Guide Training Strategy for Primary Care in Jalisco 2021. SALME Journal of the Jalisco Institute of Mental Health, 47–56. Retrieved from: https://salme.jalisco.gob.mx/sites/salme.jalisco.gob.mx/files/salme_14-enero-abril_2021.pdf
13. World Health Organization mhGAP training manuals for the mhGAP intervention guide for mental, neurological and substance use disorders Non-specialized health settings, version 2.0 (for field testing) 2017 Geneva World Health Organization
World Health Organization. mhGAP training manuals for the mhGAP intervention guide for mental, neurological and substance use disorders. Non-specialized health settings, version 2.0 (for field testing). Geneva: World Health Organization; 2017. https://iris.who.int/handle/10665/259161. Licence: CC BY-NC-SA 3.0 IGO.
14. Shapiro SS Wilk MB An analysis of variance test for normality (complete samples) Biometrika 1965 52 3–4 591 611 10.1093/biomet/52.3-4.591
Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika. 1965;52(3–4):591–611.10.1093/biomet/52.3-4.591
15. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Lawrence Erlbaum Associates; 1988.
16. Braun V Clarke V Using thematic analysis in psychology Qual Res Psychol 2006 3 2 77 101 10.1191/1478088706qp063oa
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.10.1191/1478088706qp063oa
17. Sharma M Theoretical Foundations of Health Education and Health Promotion 2017 3 Burlington, MA Jones & Bartlett Learning
Sharma M. Theoretical Foundations of Health Education and Health Promotion. 3rd ed. Burlington, MA: Jones & Bartlett Learning; 2017.
18. Frenk J Chen L Chandran L Groff E King R Meleis A Fineberg H Challenges and opportunities for educating health professionals after the COVID-19 pandemic Lancet 2022 400 10362 1539 56 10.1016/S0140-6736(22)02092-X 36522209
Frenk J, Chen L, Chandran L, Groff E, King R, Meleis A, Fineberg H. Challenges and opportunities for educating health professionals after the COVID-19 pandemic. Lancet. 2022;400(10362):1539–56.36522209 10.1016/S0140-6736(22)02092-X
