
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39209502
10.1136/bmjopen-2024-084916
bmjopen-2024-084916
Protocol
Mental Health
1712
1506
Addressing mental illness stigma in German higher education: study protocol for a mixed-methods evaluation of a psychosocial setting-based intervention
http://orcid.org/0009-0009-8800-069X
Nething Emily 1emily.nething@uni-greifswald.de

Stoll Elena 1elena.stoll@uni-greifswald.de

Dobson Keith S. 2ksdobson@ucalgary.ca

Szeto Andrew C. H. 2aszeto@ucalgary.ca

http://orcid.org/0000-0002-2846-5489
Tomczyk Samuel 1samuel.tomczyk@uni-greifswald.de

1 Department of Health and Prevention, Institute of Psychology, University of Greifswald, Greifswald, Germany
2 Department of Psychology, Faculty of Arts, University of Calgary, Calgary, Alberta, Canada
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

None declared.

Ms; emily.nething@uni-greifswald.de
2024
28 8 2024
14 8 e08491631 1 2024
19 7 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Introduction

Mental illness stigma is associated with a range of negative consequences, such as reduced help-seeking for mental health problems. Since stigma affects individual, social, and structural aspects, multilevel interventions such as the Canadian programme The Working Mind have been proven to be the most effective. Given the solid evidence base for The Working Mind, it is our aim to implement and evaluate culturally adapted versions of the programme in German higher education, targeting students, employees and managers.

Methods and analysis

We will evaluate the programme with regard to its effect on mental illness stigma, openness to mental health problems, willingness to seek help, and positive mental health outcomes. Further, we will investigate the programme’s effectiveness dependent on gender and personal values, various mechanisms of change, and factors facilitating and hindering implementation. The study uses a sequential explanatory mixed-methods evaluation design (QUAN → qual) that consists of three steps: (1) quasi-experimental online survey with programme participants, (2) focus groups with programme participants, and (3) qualitative interviews with programme stakeholders. The quantitative data collected in step 1 will be analysed using 2×3 analysis of variances and a parallel multiple mediation analysis. The results will inform the qualitative data to be collected in steps 2 and 3, which will be analysed using qualitative content analysis.

Ethics and dissemination

The study was approved by the local Ethics Committee (Ethics Committee of University Medicine Greifswald; BB 098/23). Participants have to provide written consent before taking part in a focus group or interview. As for the online survey, participants have to give their consent by agreeing to an online data protection form before they can start completing the survey. We will publish central results and the anonymised data in an Open Access Journal. Further, the statistical code will be included as a supplement to the paper(s) documenting the results of the study.

Trial registration number

DRKS00033523.

MENTAL HEALTH
Psychosocial Intervention
PUBLIC HEALTH
Stereotyping
Psychological Stress
Federal Ministry of Health of Germany 2523FSB214 The sponsor was not involved in any activities pertaining to the study or the writing and submission of this study protocol.
==== Body
pmcSTRENGTHS AND LIMITATIONS OF THIS STUDY

The intervention (The Working Mind) addresses workplace mental illness stigma on multiple levels.

A sequential (QUAN → qual) mixed-methods design is used to evaluate the intervention.

The intervention will also be evaluated with regard to a behavioural outcome measure, the utilisation of support offers.

The study is monocultural.

The results may be affected by self-report bias.

Introduction

Background

Mental illness stigma is associated with various negative consequences for those affected.1 Stigmatisation is commonly defined as a complex process in which an individual is labelled as different, stereotyped, separated, and can thus be affected by status loss and discriminated against.2 Crucially, a power differential is required for stigmatisation to unfold.2 Stigma contains cognitive, affective, and/or behavioural components.3 It can manifest on different levels, for example, the societal level (structural stigma), the general population level (public stigma) or the intrapersonal level (personal or self-stigma).4 Structural mental illness stigma is reflected, for example, in the lower allocation of resources for mental healthcare compared with physical healthcare.5 Public mental illness stigma may be reflected in denying housing opportunities to someone with a mental illness.6 Self-stigma is a form of personal stigma.7 It consists of being aware of and agreeing with public stigma, applying it to oneself and experiencing harm as a consequence,8 such as reduced help-seeking.1 In addition to mental health (self-)stigma, the specific (self-)stigma of seeking help constitutes a major barrier to seeking support offers.1 Finally, stigma can be experienced in different ways4: while, for instance, the housing example describes the experience of endorsed stigma, the stigma of seeking help may often refer to anticipated stigma.

In Germany, more than a quarter of the population experiences mental illness over the course of 1 year.9 Despite this prevalence, mental illness remains stigmatised, in particular certain disorders such as schizophrenia spectrum disorders or substance use disorders.10 One factor that may contribute to society’s negative perception of individuals living with mental illness may be the potential impact on work performance during acute phases of these conditions. Given the high value German culture ascribes to work,11 12 an assumed decrease in productivity could have a negative influence on how people living with mental illness are perceived in the workplace. For instance, individuals living with depression are often unfairly labelled as lazy and disorganised, decreasing their chances of being hired or put forward for promotions.13 14 This underscores the importance of interventions that also have the potential to change structural factors in the work context or other settings where performance is salient, for example, in higher education.

Evidence-based interventions targeting mental illness stigma

In order to reduce mental illness stigma, current research and practice rely primarily on interventions that combine elements of psychoeducation and/or contact with people with lived experience of mental illness.15 The aim is to educate people about the background, symptoms, progression, and support options, and also to show that anybody can be affected by these illnesses.61518 Interventions that combine several approaches and work on several levels (relationship and behavioural prevention) are considered particularly effective; even short interventions (1–2 sessions) are sufficient to achieve significant effects.1517 However, the number of high-quality studies on multicomponent interventions addressing multiple levels remains low.15 16

Moreover, many interventions target specific disorders (eg, schizophrenia), which can be helpful for people living with this disorder but might not work for others (eg, people living with addiction or depression). From a societal perspective, a shift towards destigmatising mental illness needs to consider the entire spectrum of mental illness. Consequently, the mental health continuum model describes positive to negative mental health across mental illnesses via gradually increasing impairment.1923 Mental health is viewed functionally—positive mental health means well-being and satisfaction, whereas negative mental health means social withdrawal and low quality of life. Research shows that the continuum concept is significantly associated with the reduction of stigma in correlative and interventional studies19 20—even for various highly stigmatised illnesses (eg, eating disorders, addiction and schizophrenia). Importantly, the mental health continuum model posits that a diagnosis of mental illness can coexist with positive mental health, which aligns well with the two continua model of positive mental health and mental illness.24

The mental health continuum model also constitutes the centrepiece of the intervention The Working Mind (TWM). TWM is aimed at reducing stigma in the work context, encouraging open exchange and supporting the use of help.22 23 25 In addition to its psychoeducational approach, the programme relies on the stigma reduction strategy of contact. This is done in the form of video clips in which people with lived experience describe aspects of their mental health journey, including experiences with stigmatisation.25 Beyond the content aimed primarily at reducing mental illness stigma, TWM fosters positive mental health by teaching coping skills and thus strengthening mental health literacy25 (for more detailed information on the programme, see the ‘Intervention’ section).

Importantly, TWM not only uses several approaches to stigma reduction but also addresses several levels: public stigma, self-stigma, and, to some extent, structural stigma. Since TWM is aimed at employees and managers in general, that is, regardless of their mental health state, public stigma constitutes the programme’s primary focus. When internalised by people living with a mental illness, public stigma turns into self-stigma, as described above. Therefore, TWM may also—indirectly—reduce self-stigma in participants living with a mental illness, particularly because participants are encouraged to actively reflect on their own mental health throughout the programme. Reduced structural stigma at the level of the organisation may be a distal outcome of TWM, fostered by participants’ exchange around workplace structures and practices beneficial to mental health. An adaptation for students (The Inquiring Mind)26 has also been developed, which takes into account the special circumstances of student life, for example, the campus environment and coping with study-specific stressors such as exam periods. Both programmes have already been successfully evaluated several times: meta-analyses point to moderate reductions in stigma and increases in resilience23 25 26 across 3 months following the programme. Following the programme, participants also reported increasing openness to discussing mental health problems and willingness to seek help for mental health problems when needed.23 25 26 However, the programme has not yet been evaluated using a control group and outside the Canadian context.23 Given both the prevalence of mental illness stigma in Germany and the central role of work in German culture described above, interventions such as TWM seem necessary.

Differential effectiveness of interventions targeting mental illness stigma

Further, it is widely acknowledged that a given intervention may be more beneficial for some participants than for others.27 For instance, interventions aimed at reducing public stigma appear to be more effective for specific target groups, such as students and employees, than for community members.3 17

Gender

Participants’ gender may also exert an influence on programme efficacy: More specifically, traditional (Western) gender norms demand that men restrain their emotions and do not express vulnerability,28 resulting in higher levels of stigmatisation of mental health problems and help-seeking.29 Both a meta-analysis and a scoping review on the effectiveness of antistigma interventions also point to differential effectiveness depending on participants’ gender.15 16 In addition to the differential endorsement of stigma, antistigma interventions’ effectiveness may be impacted by gender-specific manifestations of mental distress and illness30 31 as well as gender-specific use of coping strategies such as help-seeking,32 the latter being of particular relevance to TWM due to its focus on promoting help-seeking behaviour. While no gender differences emerged regarding the effectiveness of TWM in the Canadian context,23 25 26 the question remains whether the same applies to the German context. Due to the traditional gender norms for men described above, it seems possible that men in Germany could be more prone to stigmatisation processes (eg, public stigma, public stigma of seeking help) than women, and that the workshop, therefore, has a stronger effect on them.

Personal values

Recently, research on mental illness stigma has also taken into account that the process of stigmatisation is influenced by personal values. Personal values, such as benevolence, can become more salient or decrease in importance depending on ‘what matters most’ in a specific situation.33 34 Using well-established questionnaires based on the Theory of Human Values developed by Schwartz,35 36 hypotheses on connections between mental illness stigma and personal values have been tested. Lannin et al37 found an effect of Schwartz’ self-transcendence values on reducing public stigma of help-seeking behaviour and thus reducing self-stigma of help-seeking behaviour (public stigma of help-seeking behaviour functioned as a mediator). Rieckhof et al38 go a step further, developing a new questionnaire (Value-based Stigma Inventory, VASI) that involves aspects of personal values and mental illness stigma and shows negative correlations of stigma with self-transcendence values. Therefore, it can be assumed that participants with a higher endorsement of self-transcendence report lower stigmatising attitudes. Moreover, research shows that contextual cues can increase the salience of specific values, and if there is a fit between personal values and context, they can facilitate value-oriented behaviour and increase well-being.39 40 These findings underline that personal values can represent intraindividual differences and could influence the effects of the workshop. TWM aims to increase interpersonal dialogue and support regarding mental health and may, therefore, be connected to liberal values such as self-transcendence and benevolence. Because of the person-programme-environment fit, it may thus be more effective for people who endorse said values.

Mechanisms of change in interventions targeting mental illness stigma

Knowing for whom a programme works is crucial—as is knowing how it works.41 Over the last years, the study of possible mechanisms of change (or mechanisms of action) has increasingly made its way into evaluation research, both in the field of clinical42 43 and behaviour change interventions.44 45 Chen46 47 has proposed a conceptual framework for studying such mechanisms: a so-called change model. A change model encompasses three causally linked components: the intervention, intervention determinants, and intervention outcomes.47 Determinants are mechanisms that are influenced by the intervention and in turn influence the outcomes,47 that is, they correspond to the aforementioned mechanisms of change.

In the field of interventions targeting mental illness stigma, research into mechanisms of change is scarce, as a recent meta-analysis points out.3 Existing basic and interventional research suggests that contact may reduce public stigma by increasing empathy towards people with a mental illness, by reducing intergroup anxiety, and, to a smaller extent, by increasing knowledge in the sense of mental health literacy.48 49 In the case of TWM, more specifically, Szeto et al26 point to the central role of the continuum model of mental health. Therefore, increasing continuum beliefs may be another mechanism through which the programme could bring about change.

A comprehensive evidence base shows that programmes such as TWM are effective in reducing stigma and promoting positive mental health in multiple settings.15 17 23 25 26 However, the programme has not yet been transferred to and evaluated in other cultural contexts. In addition, the field benefits from an evaluation of the programme’s theoretical underpinnings in the sense of a change model46 47 to examine mechanisms of change.

Study objectives and research questions

Against this background, our study aims to implement and evaluate a culturally adapted version of TWM in Germany. In order to adequately investigate the different study objectives, we will pursue a comprehensive mixed-methods evaluation strategy. First, we will evaluate the programme regarding its effect on mental illness stigma, openness to mental health problems, and the willingness to seek help. We will also examine the programme’s effect on positive mental health outcomes (resilience and subjective well-being). Second, we will investigate whether the programme’s efficacy depends on participants’ gender and personal values. Third, we will test and explore possible mechanisms of change regarding public stigma, which may serve as a starting point for developing a change model according to Chen.46 47 Lastly, we will explore factors facilitating or hindering the programme’s implementation in the higher education setting to be able to improve the implementation process and enable continuity. This leads us to the following research questions and hypotheses:

Primary research questions

1. Does TWM reduce mental illness stigma?

H1. Participation in TWM leads to reduced mental illness stigma.

2. Does TWM foster openness to mental health problems?

H2. Participation in TWM leads to increased openness to mental health problems.

3. Does TWM increase the willingness to seek help for mental health problems?

H3. Participation in TWM leads to increased willingness to seek help and, if affected personally, to increased utilisation of support offers.

Secondary research questions

4. Does TWM promote positive mental health?

H4. Participation in TWM leads to higher resilience.

H5. Participation in TWM leads to higher subjective well-being.

5. Does the effect of TWM on mental illness stigma differ according to participants’ gender and personal values?

H6. The effect of participating in TWM on mental illness stigma is moderated by gender.

H7. The effect of participating in TWM on mental illness stigma is moderated by personal values.

6. What mechanisms in TWM bring about change regarding mental illness stigma?

H8. The effect of participating in TWM on mental illness stigma is mediated by mental health literacy.

H9. The effect of participating in TWM on mental illness stigma is mediated by agreement with continuum beliefs.

H10. The effect of participating in TWM on mental illness stigma is mediated by empathy towards people with mental illness.

H11. The effect of participating in TWM on mental illness stigma is mediated by intergroup anxiety towards people with mental illness.

Exploratory research question

What factors promote and hinder the (sustainable) implementation of TWM in the higher education setting?

Methods and analysis

Trial design

Our choice to use a mixed-methods trial design is guided by the principle of pragmatism,50 our overarching goal being to comprehensively evaluate different facets of TWM. While our study primarily focuses on the programme’s efficacy, as reflected in the primary research questions, we also seek to elaborate on these results and expand the study’s focus by investigating questions of a more exploratory type. Therefore, the two main purposes of our mixed-methods design are complementarity and expansion.51 In line with this rationale, we have chosen a sequential exploratory design (QUAN → qual), which encompasses three steps: (1) a quasi-experimental online survey, (2) focus groups, and (3) qualitative interviews with stakeholders. While step 1 pertains to the ‘QUAN’ phase, steps 2 and 3 form the ‘qual’ phase. The findings from the quantitative and qualitative phases will be connected in the intermediate stage of the study52, such that the findings of step 1 will influence both steps 2 and 3. A visual model depicting the trial design can be found in figure 1.

Figure 1 Visual model depicting the mixed-methods trial design. ANOVA: analysis of variance. ANCOVA: analysis of covariance.

In the online survey, we will collect quantitative data in three waves: preintervention, postintervention and at 6-month follow-up. As the intervention is being carried out in a different cultural context for the first time, our focus at this stage will be to evaluate the intervention as such rather than contrasting it with other active treatments. Consequently, we have chosen a quasi-experimental design with a passive control group.

In step 1, we will focus on answering the efficacy-focused research questions, that is, questions 1–5. Additionally, this step will help to identify ‘candidate mediators’ and mechanisms (of change)41 targeted in question 6.

The focus groups with intervention participants will take place approximately 1 month after the intervention. Participants will be stratified according to target group (students, employees and managers) and, if possible, gender (women, men; other gender identities will not be used for stratification purposes).

In step 2, we will focus on both elaborating and expanding the quantitative results obtained in step 1. The goal of the elaboration process will be to develop a deeper understanding of the quantitative data, for example, by obtaining a more fine-grained understanding of mechanisms of change. The goal of the expansion process will be to develop a broader understanding of the quantitative data, for example, by giving participants the opportunity to freely share their experiences with the programme (thus expanding the range of programme ‘outcomes’).

The qualitative interviews with stakeholders will take place after the first implementation round, that is, after the first 6-month follow-up assessment. Programme stakeholders will include people working in mental health and psychosocial services at the German universities in which the programme will be implemented. Further, representatives of the universities’ personnel department and the staff council will be contacted for an interview.

In step 3, our sole focus will be stakeholders’ perspectives on factors facilitating and hindering the sustainable implementation of the programme, that is, research question 7. Although the target group of this step is different from that of steps 1 and 2, the previously obtained results provide an important basis for answering question 7. Therefore, development51 may be considered the third function of our mixed-methods design.

Patient and public involvement

The programme’s target groups, that is, students, employees, and managers, have been involved in the entire research process. They were included in the development of the study design, and the study was designed based on their needs and priorities. The research questions regarding stigmatising attitudes, mental health literacy and openness were derived from panel discussions and expert interviews. They were also part of the process of culturally adapting the programme (including focus group discussions, translations, etc). Further, the target groups were involved in the adaptation process of two of the outcome measurement instruments used in this study (Opening Minds Scale–Workplace Attitudes (OMS-WA) and Intergroup Anxiety Scale (IAS); see the ‘Data collection’ section). Programme participants will also be involved in the choice of outcome measures in the qualitative parts of the study. As described above, they will be asked to reflect on the ways in which the programme may be beneficial to them, either personally (step 2) or as seen from their professional perspective (step 3). Moreover, they will be part of the recruitment of participants by forwarding invitations and presenting the study to their peers, and in a similar way, they will also be part of the dissemination, as leaflets and factsheets will be created that summarise the main findings of the study in lay language to be disseminated to the public. To ensure their continued involvement in the research, the study was connected to an internal advisory board at the university (consisting of representatives of all groups) that reflects on and discusses the progress of the study.

Sample size

We have calculated the sample size based on the anticipated mean effect for research question 1. An a priori power analysis for a 2×3 analysis of variance (ANOVA) (intervention vs control; preintervention, postintervention, and follow-up assessment) with repeated measures with a power of 0.80, α=0.05 and an anticipated medium effect size (d=0.38; see meta-analysis on TWM23; indicates a required total sample of n=50, which means approximately n=25 per group (ie, control and intervention groups) if the groups are equally populated. As we will carry out the intervention in different target groups, the required number of participants increases: For the three target groups, this results in a total planned sample of 25×2×3=150 people, of whom around a third are to be reached again for follow-up.

For the focus groups, participants of the intervention group will be stratified according to target group (students, employees, management) and, if possible, gender (male, female), thus ideally resulting in six groups of six people each (n=36).

Participant eligibility criteria

To be included in the study, participants have to be aged 18 years or older, to be members of a German higher education institution, and to belong to either of the intervention target groups: students, employees or managers. Given the focus of our intervention, managers are defined as having personnel responsibility. Individuals who do not meet these criteria will be excluded from the study.

Participant timeline

In the following, we will describe the different steps of the study procedure. Figure 2 provides a schematic overview of the schedule.

Figure 2 Schedule of participant acquisition, enrolment, interventions and assessments. IAS, Intergroup Anxiety Scale; IASMHS, Inventory of Attitudes to Seeking Mental Health Services; MHL-W-G, Mental Health Literacy Tool for the Workplace; OMS-WA, Opening Minds Scale–Workplace Attitudes; PHQ-9, Patient Health Questionnaire-9; SSMIS-SF, Self-Stigma of Mental Illness Scale–Short Form; SSOSH, Self-Stigma of Seeking Help; SSRPH, Stigma Scale for Receiving Psychological Help; VASI, Value-based Stigma Inventory.

Participant acquisition and enrolment

Since the intervention is aimed at students and staff of German universities and piloted at the University of Greifswald (a medium-sized town in the northeast of Germany), participants for the intervention group are currently being acquired via the university’s channels. The intervention has been included in the catalogue of health offers available to all members of the University of Greifswald. In addition, the intervention dates are being included in the university calendar, and flyers are being distributed across the campus. The employee and manager versions of the programme are also being advertised via central university mailing lists, whereas the student version will be advertised via various student Instagram channels. University members interested in taking part can enrol via the catalogue. Control group participants will be acquired via other German universities’ channels, for example, mailing lists.

Assessments and interventions

Participants of both groups will be invited to a quantitative preintervention assessment (t1) a few days prior to the intervention. As described above, the student and employee versions of the programme will take place on 1 day, whereas the manager version of the intervention will be spread over 2 days (max. 2 weeks apart). At the end of the intervention, only the intervention participants will be informed about the focus groups and asked to indicate whether they consent to the research team contacting them about participating. Quantitative postintervention assessment (t2QUAN) will take place shortly after the intervention for both groups; qualitative postintervention assessment (t2QUAL) will take place approximately a month after the intervention. Finally, a quantitative follow-up assessment (t3) will take place around 6 months postintervention for both groups. The study, that is, first participant enrolment, started on 1 February 2024, after trial registration (31 January 2024) and submission of the first version of this protocol (31 January 2024). The study is planned to end on 30 June 2025.

Intervention

TWM23 is a Canadian group-based intervention with the primary aims of reducing stigmatisation of mental illnesses and people affected by these illnesses and promoting help-seeking. Its secondary aim is to strengthen positive mental health by providing adaptive coping strategies and relevant help options so that stress can be dealt with more effectively. As a result, mental illnesses may develop less frequently and take a less severe course. The programme is based on a combination of psychoeducational, contact-based, and coping-oriented elements and thus reflects the current state of research (see the ‘Introduction’ section). As described above, there are different versions of the intervention. Given the target groups of our study, we will implement the culturally adapted student, employee, and manager programme versions. Table 1 lists the programme modules for each target group.

Table 1 The Working Mind (TWM) programme modules for the different target groups

Module #	Students	Employees	Managers	
1	Mental health and stigma	Mental health and stigma	Mental health and stigma	
2	The mental health continuum tool	Mental health in the workplace	Mental health in the workplace	
3	Self-care and building resilience	Self-care and building resilience	Self-care and building resilience	
4	Creating a supportive campus		Supporting your team	

The information will be delivered using a PowerPoint presentation along with a facilitator guide. Furthermore, participants will receive a handout and additional print information on specific topics (eg, a resilience guide). All the original materials have been translated into German and adapted culturally. Intervention facilitators have a professional background in mental health and have successfully completed a week-long facilitator training for the intervention.

The intervention will be provided in small face-to-face groups of up to 15 people. The intervention will take place on the premises of the University of Greifswald; a first round is planned from February 2024 onwards.

Each module is around 1 hour long, resulting in a total duration of around 4 hours. The programmes for students and employees are carried out in 1 day while the programme for managers is divided into two sessions of 4 hours each.

Measures

In the following, we will provide an overview of the constructs to be examined in our study. All the constructs listed below can also be found in figure 2.

Outcomes

The study’s primary outcomes, examined in research questions 1–3, encompass mental illness stigma (H1), openness to mental health problems (H2), and willingness to seek help/utilisation of support offers (H3).

Secondary outcomes relate to research questions 4 and 6. Research question 4 comprises positive mental health outcomes: resilience (H4) and subjective well-being (H5). Research question 6 comprises possible intermediary outcomes (ie, mechanisms of change): mental health literacy (H8), mental health continuum beliefs (H9), empathy towards people with mental illness (H10), and intergroup anxiety towards people with mental illness (H11).

We will aggregate individual participant data by calculating mean values. Due to our aim to measure changes in outcomes, we will examine the variables listed above:preintervention (quantitative), postintervention (quantitative and qualitative), and at 6-month follow-up (quantitative). The only exception will be the utilisation of support offers, which we will only collect at follow-up.

Covariates

Several measures will be collected and examined as potential moderators of programme outcomes. As stated in research question 5, we will examine gender (H6) and personal values (H7) as potential moderators. Further, we will collect other sociodemographic and contextual variables identified as relevant factors of influence in the literature151719 23 26 53: age, education, professional situation, field of study/work, current level of mental distress, as well as direct and indirect experience with mental illness and treatment options.

Again, we will aggregate individual participant data by calculating mean values. All covariates will be collected preintervention. Mental distress will additionally be measured at quantitative postintervention and 6-month follow-up, personal values will additionally be measured at 6-month follow-up. Gender, personal values, and experience with mental illness (treatment) will additionally be collected at qualitative postintervention. An overview can be found in figure 2.

Data collection methods

Quantitative data will be collected via online questionnaires, using the online platform SoSci Survey. Qualitative data will be collected in focus groups and interviews using semistructured interview guidelines. As the findings of step 1 will inform the subsequent steps of data collection, the interview guidelines will be finalised after analysis of these data. Both the online questionnaire and the interview guidelines will be uploaded to the OSF (https://osf.io/qrjce/?view_only=562269481229499c9467d750b7021e4c). The interview guidelines can also be found in onlinesupplemental files 12. Student research assistants supporting both quantitative and qualitative assessment will be trained for the different tasks involved. Participants in the intervention group will receive a voucher worth €10 for each completed questionnaire and €30 for taking part in a focus group to increase participation rates.

In the following, we will briefly describe the psychometric features of the quantitative measurement instruments used. If not stated otherwise, we will calculate the sum scores for each of the instruments.

Primary outcomes (questions 1–3)

(H1) Mental illness stigma will be examined with various questionnaires to adequately capture the construct’s multifaceted nature.

Public mental illness stigma in the workplace/study context will be assessed with the OMS-WA.23 25 26 54 55 The OMS-WA exists in different versions tailored to different contexts; given our target groups, we will use the student,26 employee23 25 55, and manager version.54 While the student version measures stigmatising attitudes related to the study context (and the phase of life associated with it), the employee and manager versions measure stigmatising attitudes in the workplace. The student version comprises 23 items, the employee version 22, and the manager version 11, such as ‘people/employees with a mental illness could snap out of it if they wanted to.’ Participants are asked to rate the items on a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. Since the OMS-WA is central to our study and the versions we will use have not yet been translated into German, we translated them using a forward-back translation procedure, consulting an expert panel and pretesting them with the target group. Internal consistency is good to excellent for the English student version (0.88≤α≤0.92)26 and excellent for the employee version (α=0.90).54 Validation of the manager version is reported to be in progress.54 A comprehensive validation study on the employee version is currently under review56; the student version will be validated subsequently.

Public mental illness stigma (context-unspecific) will be assessed with the German version of the Self-Stigma of Mental Illness Scale–Short Form57 58 Agreement subscale. Whereas the scale as a whole was developed to measure self-stigma in people living with a mental illness, the Agreement subscale captures public stigma (which represents a necessary component of self-stigma). Participants are asked to rate the five items, such as ‘I think most persons with mental illness are dangerous.’ on a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. Studies across multiple clinical samples and languages indicate good construct validity of the scale; internal consistencies for the Agreement subscale are acceptable (72≤α≤0.7957). The use of the subscale for a general workplace/student sample will be discussed in the context of the study’s limitations.

In addition, we will examine prognostic pessimism, that is, the view that people with mental illness are unlikely to recover, which represents another important component of mental illness stigma.59 Prognostic pessimism will be assessed with a single item measure based on the one used by Lebowitz and Ahn.60 It asks participants to rate, on a 7-point Likert scale from 1=‘not at all permanent’ to 7=‘very permanent’, how permanent they consider a mental illness to be.

Value-sensitive mental illness stigma will be assessed with the VASI.38 Participants are asked to rate the 15 items, such as ‘It damages my reputation if a mental illness becomes known in my family.’ on a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. The items form five subscales: Self-Realisation, Personal Enrichment, Reputation, Meritocratic Values, Security. Rieckhof et al38 report good internal consistency (α=0.88) and good convergent and construct validity of their questionnaire.

Public stigma of seeking help will be assessed with the German version of the Stigma Scale for Receiving Psychological Help.61 Participants are asked to rate the five items, such as ‘Seeing a psychologist for emotional or interpersonal problems carries social stigma.’ on a 4-point Likert scale from 0=‘strongly disagree’ to 3=‘strongly agree’. Internal consistency for the scale is good (α=0.81)61 and the original study indicated good construct validity62; however, information on its validity remains sparse.

Self-stigma of seeking help will be assessed with the German version of the Self-Stigma of Seeking Help scale.61 Participants are asked to rate the 10 items, such as ‘I would feel worse about myself if I could not solve my own problems.’ on a 5-point Likert scale from 1=‘strongly disagree’ to 5 ‘strongly agree’. Internal consistency for the scale is acceptable to good (0.80≤α≤0.84).61 63 Information on the validity of the German version is not yet available, but the original version has proven valid in terms of construct, criterion, and predictive validity.64

(H2) Openness towards mental health problems will be assessed with the German version of the Inventory of Attitudes to Seeking Mental Health Services65 Psychological Openness subscale: Participants are asked to rate the eight items, such as ‘There are certain problems which should not be discussed outside of one’s immediate family.’ on a 5-point Likert scale from 0=‘disagree’ to 4=‘agree’. Internal consistency for the Psychological Openness subscale is acceptable (α=0.70).61 Convergent validity for the original scale was demonstrated by Mackenzie et al.66

(H3) Willingness to seek help will be assessed by asking participants to rate how likely it would be for them to use different support offers, using a 6-point Likert scale from 1=‘not at all likely’ to 7=‘very likely’. They will be provided with a list of 8–10 (depending on the target group) support offers discussed during the intervention and asked to rate the likelihood for each of the offers.

Utilisation of support offers will be assessed by asking participants if they have used one or more support offers (0=‘no’, 1=‘yes’). Participants will be able to choose from the options provided when assessing their willingness to seek support, and they will also be able to state any other support offers they have sought out in a free text box.

Secondary outcomes

(H4) Resilience will be assessed with the German version of the Brief Resilience Scale (BRS)67 Participants are asked to rate the six items, such as ‘I tend to bounce back quickly after hard times.’ on a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. Internal consistency for the scale is good (α=0.85).67 The BRS is moderately correlated with optimism and social support, indicating convergent validity.67

(H5) Subjective well-being will be assessed with the German version of the WHO-5 Well-being Index (WHO-5).68 Participants are asked to rate five items, such as ‘I have felt calm and relaxed’. Participants are instructed to refer to the last 2 weeks, answering the items using a 6-point Likert scale from 0=‘all of the time’ to 5=‘at no time’. Internal consistency for the scale is good to excellent (0.89≤α=0.92).68 The WHO-5 is moderately correlated with somatic well-being, indicating convergent validity.68

(H8) Mental health literacy will be assessed using the Mental Health Literacy Scale (MHLS)69 for students and the German version of the Mental Health Literacy Tool for the Workplace (MHL-W-G)70 for employees and managers. As regards the MHLS, there is currently no validated German translation available; therefore, we will use the German translation developed and piloted in an unpublished bachelor’s thesis.71 Depending on the item, participants are asked to rate the 35 items on either a 4-point Likert scale from 1=very unlikely/unhelpful to 4=‘very likely/helpful’ or a 5-point Likert scale from 1=‘strongly disagree/definitely unwilling’ to 5=‘strongly agree/definitely willing’. They are, for instance, asked to rate statements such as ‘To what extent do you think it is likely that the diagnosis of Drug Dependence includes physical and psychological tolerance of the drug (ie, require more of the drug to get the same effect)?’. Internal consistency for the English version of the scale is good (α=0.83), and correlations with help-seeking intentions—although in the low range—point to construct validity.69 The MHL-W-G assesses workplace-related mental health literacy based on four vignettes. Participants are asked to rate their competence on the basis of four items per vignette, that is, 16 items in total, using a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. Internal consistency of the scale is good to excellent (0.88≤α≤0.92).67 The scale is moderately correlated with general health literacy and related variables; therefore, construct validity can be assumed.70

(H9) Continuum beliefs will be assessed with the newly developed (German) Continuum Beliefs Scale.72 Participants are asked to rate the nine items, such as ‘Now and again most of us have symptoms of a mental illness.’ on a 5-point Likert scale from 1=‘strongly disagree’ to 5=‘strongly agree’. As the scale has been developed as part of an ongoing project,72 psychometric validation is still outstanding.

(H10) Empathy will be assessed with a three-item scale used in previous stigma research73 74 which we have translated into German. Participants are asked to rate items such as ‘If a person with a mental health problem I knew was feeling sad, I think that I would also feel sad.’ on a 7-point Likert scale from 1=‘strongly disagree’ to 7=‘strongly agree’. Internal consistency of the empathy scale is good (ω=0.87). The scale has been shown to correlate with behavioural facets of stigma,73 which could be interpreted as a first hint to criterion validity.

(H11) Intergroup anxiety will be assessed with the IAS.49 75 The IAS instructs participants to imagine a situation in which they are to interact with people with a mental illness, and where they themselves are the only person without a mental illness. Subsequently, they are asked to rate to what extent they experienced certain emotions compared with a situation in which they interacted with people without mental illness. As regards the specific items, we will rely on Potts et al,49 whose version we have translated into German (for a detailed description of the translation process, see OMS-WA): Participants will be presented with a list of 13 adjectives (ie, items) to be rated on a 5-point Likert scale from 0=‘not at all’ to 4=‘extremely’. Positive emotions will then be reverse scored, and all items will be averaged. According to Potts et al,49 the scale shows good internal consistency (α=0.86) and associations with related constructs such as intergroup contact, which they interpret as evidence for construct validity. Individuals reporting personal experience with mental illness treatment and elevated levels of mental distress will not be asked about intergroup anxiety.

Assessment of potential covariates

(H6) Gender will be assessed by asking participants which gender they identify with, the response options being 1=‘female’, 2=‘male’, 3=‘non-binary’, 4=‘gender-fluid’, 5=‘agender’. Alternatively, participants can also use a text box to fill in the gender that they identify with.

(H7) Personal values will be assessed with the Portrait Values Questionnaire 21 (PVQ21, ESS21)66, adapted in a gender-neutral form.38 There is some evidence76 77 that partial scalar invariance of this short form exists for seven of the ten values postulated by Schwartz35 in his Theory of Human Values. Higher order measures of values (self-transcendence, self-enhancement, conservation, and openness to change) have shown more acceptable model fits.77 Personal values measured with PVQ21 have been shown to be stable over a 3-year period, similar to personal traits.78 The PVQ21 consists of 21 items presented in the form of short verbal portraits that describe a person’s goals, aspirations, or desires that point explicitly to each value, for example: ‘It’s important to them to be rich’. The respondents are asked to rate the similarity on a scale from 1=‘very similar’ to 6 = ‘not similar at all’.

Age, education, professional situation, and field of study or work will each be assessed with single items. Age will be assessed by asking participants to indicate their age in years. Education will be assessed by asking participants to indicate their highest school-leaving certification, with the response options ranging from 1=‘none’ to 11=‘doctorate’, including a free text box for certifications/qualifications not listed (not all response options listed as they are specific to the German education system). Professional situation will be assessed by asking employees and managers to indicate their current professional situation, with various nominally scaled response options provided: 1=‘full-time employed’, 2=‘part-time employed’, 3=‘partially retired’, 4=‘marginally employed’, 5=‘occasionally employed’, 6=‘in vocational training/an apprenticeship’, 7=‘in voluntary service’, 8=‘on parental leave/leave of absence’, including a free text box. Students will be asked whether they pursue gainful employment alongside their studies, with the following nominally scaled response options provided: 1=‘no’, 2=‘yes, full-time employed’, 3=‘yes, part-time employed’, 4=‘yes, self-employed’, 5=‘yes, marginally employed’. Field of work will be assessed by asking employees to indicate their field of work, with the following response options provided: 1=‘administration’, 2=‘academic staff’, 3=‘other’. Students will be asked to indicate their field of study by choosing one of the following response options: 1=‘social sciences’, 2=‘arts’, 3=‘teacher training’, 4=‘mathematics, sciences’, 5=‘medicine, health sciences’, 6=‘humanities’, 7=‘economics, law’.

Mental distress will be assessed with the German version of the Patient Health Questionnaire-9 (PHQ-9),79 a screening instrument for depression. Participants are asked to rate nine items describing different symptoms of depression, such as ‘Feeling down, depressed or hopeless’ on a 4-point Likert scale with the response options 1=‘not at all’, 2=‘several days’, 3=‘more than half the days’, 4=‘nearly every day’. The internal consistency of the scale is good (α = 0.89). The PHQ-9 has been shown to negatively correlate with various indicators of positive mental health, which can be considered an indicator for convergent validity.80

Experience with mental illness (treatment) will be assessed with four questions capturing different types of experience. Participants are asked (1) whether they are currently in treatment for mental illness, (2) whether they have been in treatment for mental illness at some point in their life, (3) whether someone from their immediate social environment has been in treatment for mental illness, and (4) whether they have already worked with someone with a mental illness. The items are to be answered in a yes-no response format (0=‘no’, 1=‘yes’); for questions 3 and 4, the option 2 = ‘I don’t know’ is also provided. Participants who answer questions one and/or two with ‘yes’ will additionally be asked about the type of treatment they have sought, with the response options (multiple answers possible) being 1=‘medical treatment (psychiatric treatment, eg, psychotropic drugs’), 2=‘psychotherapeutic treatment (eg, talk therapy)’, 3=‘art, music and/or sports therapy’, 4=‘self-help group’, 5=‘counselling and/or coaching services (eg, educational counselling, life counselling)’, 6=‘online therapy and/or telephone therapy’.

Data management

Participants will enter the quantitative survey data via the platform SoSci Survey. SoSci Survey works with SSL encryption (HTTPS) of the data when filling out the questionnaire and when retrieving the collected data. These data will then be stored pseudonymously on the file servers of the University of Greifswald. Plausibility checks of the data (eg, range checks) will be carried out before data analysis. The statistical analysis of the quantitative data will be carried out using the software R. After collection of the 6-month follow-up data, the quantitative data will be completely anonymised.

The qualitative focus group and interview data will be audiotaped, stored on the file servers of the University of Greifswald and transcribed manually. The data will be anonymised in the transcription process, and the audio files will be deleted once the transcription is completed. The qualitative data will be analysed using MAXQDA.

The data collected will be compiled in the data processing facility and stored on the secured server of University of Greifswald’s data centre. The anonymised data will be stored beyond the end of the project, with the sole purpose of evaluating and reporting them in scientific publications. Once analysed, the anonymised dataset, aggregated at the group level, will be made accessible via the OSF.

Data analysis

The quantitative data collected in step 1 will serve to answer research questions 1–6. Hypotheses H1–5 will be analysed with a 2×3 ANOVA with repeated measures to take into account the two groups (intervention vs waitlist) and the three time points of quantitative assessment. Given a significant main effect, moderation hypotheses H6 will be tested using a 2×3×2 ANOVA with group, time point, and gender as predictors, testing the interaction effect of all three predictors. H7 will be tested using a 2×2 ANCOVA with group, time and personal values (continuous) as predictors. If the assumptions of normality and heterogeneity cannot be met, an ordinal logistic regression will be calculated. Mediation hypotheses H8–11 will be tested using parallel multiple mediation analyses.

To minimise the risk of missing values, participants are compensated for taking part in the survey. Further, the survey is set up so that participants are reminded whenever they have left any items unanswered. Before proceeding to the next question, they are asked to confirm they wish to proceed without answering said item(s). By reducing inadvertent omissions, this strategy may particularly reduce one mechanism under which missing values occur: values missing completely at random. In order to manage data missing at random, attrition will be analysed, examining selective dropout across the three time points. This procedure allows for the identification of potential auxiliary variables (eg, mental distress), which may then enable the use of the full information maximum likelihood method, provided that an appropriate analytical model can be specified with the available data.81 Otherwise, alternative methods such as multiple imputation, using mixed-effect models, will be considered.81

The qualitative data collected in the focus groups (step 2) and the stakeholder interviews (step 3) will be analysed using qualitative content analysis, according to Kuckartz.82 A combination of inductive and deductive approaches to analysis will be used.83 More specifically, we will follow Kuckartz’82 content-structuring approach, which encompasses seven steps: (1) initial textual work, including writing memos and case summaries; (2) deductive development of main categories based on the focus group or interview guideline; (3) coding the data based on the main categories; (4) inductive development of subcategories; (5) coding the data based on the subcategories; (6) analysing the data along the main and subcategories, including associations between categories and (7) putting into writing the findings and documenting the process. The steps described may be repeated over several cycles, thus resulting in an iterative process of data analysis.81 We will analyse the fully transcribed focus group or interview transcripts, using the group as the unit of analysis for the focus groups. As regards the coding units, no minimum or maximum length will be specified since a coding unit equals a unit of meaning in Kuckartz’ qualitative content analysis.82 The transcripts will be coded and analysed by two researchers (ES and EN) as well as trained student assistants, such that each transcript will be coded by two coders, at least one of them ES or EN. To ensure a shared understanding of both the research questions and the method, we will organise a dedicated meeting with all German team members (ES, ST, EN and the student assistants) prior to starting qualitative data analysis. ES and EN both hold a postgraduate degree in psychology; the research assistants are undergraduate students of psychology. ES is a licensed psychotherapist and has extensive expertise regarding the systematic analysis of interview data and qualitative content analysis; EN is a psychologist and has no prior experience with qualitative content analysis but has familiarised herself thoroughly with the method on the basis of the relevant literature and under supervision of ST, psychologist, who has led and conducted several qualitative research projects.

The findings from the quantitative and qualitative phases will be integrated at the data interpretation stage52 (see figure 1).

Data monitoring

Data monitoring committee

A data monitoring committee will not be installed, as there is no blinding to experimental manipulations, and adverse events can directly be reported to those responsible for the intervention and the study. The study information includes contact details of the trial investigators, with explicit instructions to contact them at any time.

Harms

No harm is anticipated since participation is voluntary and since previous studies have not found any iatrogenic or non-intended effects. As mentioned above, participants will be able to report any adverse events during or after the intervention or study to the intervention facilitators or trial investigators. Potential adverse events would thus be collected non-systematically. They would be assessed conjointly with the person, as would appropriate harm reduction strategies and post-trial care. Finally, they would be quantified and reported on in scientific publications.

Ethics and dissemination

Research ethics approval

The project is based on the guidelines of Good Scientific Practice of the German Research Foundation and Good Clinical Practice of the German Society for Epidemiology. For the implementation of the empirical studies in the project, a vote was obtained from the local ethics committee (Ethics Committee of University Medicine Greifswald; BB 098/23).

Consent

Participants are informed and educated about both the intervention and the study prior to data collection (consent form, see Supplement 3); informed consent as defined by the Declaration of Helsinki is a condition for participation in the study. Participants have to provide written consent before taking part in a focus group or interview. As for the online survey, participants have to give their consent by agreeing to an online data protection form before they can start completing their first survey.

Confidentiality and dissemination policy

A detailed data protection concept based on the European Union's General Data Protection Regulation (EU GDPR) regulates the administrative, organisational and technical measures for processing and protecting the data and is reviewed by the University of Greifswald’s data protection officer (for more information, see the ‘Data management’ section). This includes the open-access publication of a study protocol and central results as well as the anonymised data aggregated at the group level in the sense of open science. Further, the statistical code will be included as a supplement to the paper(s) documenting the results of the study. The project thereby follows the recommendations of the German Psychological Society on Open Science practices.

Protocol amendments

The necessity to modify the protocol will most likely arise from low participation rates. In that case, participants would be informed before taking part in the programme. Deviations from the original trial design and, accordingly, the statistical methods originally planned will be documented in both the trial registry and the journal in which the study or studies will be published.

supplementary material

10.1136/bmjopen-2024-084916 online supplemental file 1

10.1136/bmjopen-2024-084916 online supplemental file 2

Review Process File
28 08 2024

Funding: This work was supported by the Federal Ministry of Health of Germany, based on a resolution by the German Federal Parliament, grant no. 2523FSB214.

Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-084916).

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication: Not applicable.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
==== Refs
References

1 Sickel AE Seacat JD Nabors NA Mental health stigma update: a review of consequences Adv Ment Health 2014 12 202 15 10.1080/18374905.2014.11081898
2 Link BG Phelan JC Conceptualizing stigma Annu Rev Sociol 2001 27 363 85 10.1146/annurev.soc.27.1.363
3 Na JJ Park JL LKhagva T et al The efficacy of interventions on cognitive, behavioral, and affective public stigma around mental illness: a systematic meta-analytic review Stigma Health 2022 7 127 41 10.1037/sah0000372
4 Pescosolido BA Martin JK The stigma complex Annu Rev Sociol 2015 41 87 116 10.1146/annurev-soc-071312-145702 26855471
5 Ungar T Knaak S Measuring structural stigma The stigma of mental illness: models and methods of stigma reduction 1st edn Oxford University Press 2021 39 54 Available 10.1093/med/9780197572597.003.0004
6 Rüsch N Angermeyer MC Corrigan PW Mental illness stigma: concepts, consequences, and initiatives to reduce stigma Eur psychiatr 2005 20 529 39 10.1016/j.eurpsy.2005.04.004
7 Gerlinger G Hauser M De Hert M et al Personal stigma in schizophrenia spectrum disorders: a systematic review of prevalence rates, correlates, impact and interventions World Psychiatry 2013 12 155 64 10.1002/wps.20040 23737425
8 Corrigan PW Watson AC The paradox of self-stigma and mental illness Clin Psychol Sci Pract 2002 9 35 53 10.1093/clipsy.9.1.35
9 Jacobi F Höfler M Strehle J et al Psychische Störungen in der Allgemeinbevölkerung: Studie zur Gesundheit Erwachsener in Deutschland und ihr Zusatzmodul psychische Gesundheit (DEGS1MH) Der Nervenarzt 2014 85 77 87 10.1007/s00115-013-3961-y 24441882
10 Schomerus G Schindler S Sander C et al Changes in mental illness stigma over 30years - improvement, persistence, or deterioration? Eur Psychiatry 2022 65 e78 10.1192/j.eurpsy.2022.2337 36328960
11 Borchert M Landherr G The changing meanings of work in Germany Adv Dev Hum Resour 2009 11 204 17 10.1177/1523422309333762
12 Matousian N Otto K How to measure mental illness stigma at work: development and validation of the workplace mental illness stigma scale Front Psychiatry 2023 14 1225838 10.3389/fpsyt.2023.1225838 37502810
13 Kapoor A Depressed people need not apply: mental health stigma decreases perceptions of employability of applicants with depression Undergraduate Research Toronto University of Toronto 2017
14 O’Hagan S The effect of depression behaviours and diagnostic label on hiring managers’ stigma Honours Thesis Brisbane The University of Queensland 2011
15 Morgan AJ Reavley NJ Ross A et al Interventions to reduce stigma towards people with severe mental illness: systematic review and meta-analysis J Psychiatr Res 2018 103 120 33 10.1016/j.jpsychires.2018.05.017 29843003
16 Kerkemeyer L Achtert K Entstigmatisierung psychischer Erkrankungen. Scoping Review zu Interventionen und Bestandsaufnahme von Best-Practice-Beispielen. Ergebnisbericht GKV-Spitzenverband 2021
17 Maunder RD White FA Intergroup contact and mental health stigma: a comparative effectiveness meta-analysis Clin Psychol Rev 2019 72 101749 10.1016/j.cpr.2019.101749 31254936
18 Corrigan P Michaels PJ Morris S Do the effects of antistigma programs persist over time? Findings from a meta-analysis Psychiatr Serv 2015 66 543 6 10.1176/appi.ps.201400291 25686817
19 Peter L-J Schindler S Sander C et al Continuum beliefs and mental illness stigma: a systematic review and meta-analysis of correlation and intervention studies Psychol Med 2021 51 716 26 10.1017/S0033291721000854 33827725
20 Tomczyk S Schlick S Gansler T et al Continuum beliefs of mental illness: a systematic review of measures Soc Psychiatry Psychiatr Epidemiol 2023 58 1 16 10.1007/s00127-022-02345-4 35927343
21 Persson L Dobson KS Frampton NMA 2022 Evaluation of a mental health continuum model in two samples Can J Behav Sci 54 206 12 10.1037/cbs0000273
22 Dobson KS Szeto A Knaak S et al Mental health initiatives in the workplace: models, methods and results from the mental health commission of Canada World Psychiatry 2018 17 370 1 10.1002/wps.20574 30192093
23 Dobson KS Szeto A Knaak S The working mind: a meta-analysis of a workplace mental health and stigma reduction program Can J Psychiatry 2019 64 39S 47S 10.1177/0706743719842559 31122049
24 Westerhof GJ Keyes CLM Mental illness and mental health: the two continua model across the lifespan J Adult Dev 2010 17 110 9 10.1007/s10804-009-9082-y 20502508
25 Szeto A Dobson KS Knaak S The road to mental readiness for first responders: a meta-analysis of program outcomes Can J Psychiatry 2019 64 18S 29S 10.1177/0706743719842562 31010293
26 Szeto ACH Henderson L Lindsay BL et al Increasing resiliency and reducing mental illness stigma in post-secondary students: a meta-analytic evaluation of the inquiring mind program J Am Coll Health 2023 71 2909 19 10.1080/07448481.2021.2007112 34875206
27 Belsky J van Ijzendoorn MH What works for whom? Genetic moderation of intervention efficacy Dev Psychopathol 2015 27 1 6 10.1017/S0954579414001254 25640826
28 Heise L Greene ME Opper N et al Gender inequality and restrictive gender norms: framing the challenges to health Lancet 2019 393 2440 54 10.1016/S0140-6736(19)30652-X 31155275
29 Rice S Oliffe J Seidler Z et al Gender norms and the mental health of boys and young men Lancet Public Health 2021 6 e541 2 10.1016/S2468-2667(21)00138-9 34332667
30 Lucht M Schaub RT Meyer C et al Gender differences in unipolar depression: a general population survey of adults between age 18 to 64 of German nationality J Affect Disord 2003 77 203 11 10.1016/s0165-0327(02)00121-0 14612220
31 Walther A Grub J Ehlert U et al Male depression risk, psychological distress, and psychotherapy uptake: validation of the German version of the male depression risk scale J Affect Disord Rep 2021 4 100107 10.1016/j.jadr.2021.100107
32 Seidler ZE Dawes AJ Rice SM et al The role of masculinity in men’s help-seeking for depression: a systematic review Clin Psychol Rev 2016 49 106 18 10.1016/j.cpr.2016.09.002 27664823
33 Schomerus G Angermeyer MC Blind spots in stigma research? Broadening our perspective on mental illness stigma by exploring “what matters most” in modern Western societies Epidemiol Psychiatr Sci 2021 30 e26 10.1017/S2045796021000111 33729113
34 Yang LH Kleinman A Link BG et al Culture and stigma: adding moral experience to stigma theory Soc Sci Med 2007 64 1524 35 10.1016/j.socscimed.2006.11.013 17188411
35 Schwartz SH Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries Zanna MP Advances in experimental social psychology New York Academic Press 1992 1 65 Available 10.1016/S0065-2601(08)60281-6
36 Schwartz SH An overview of the schwartz theory of basic values Online Read Psychol Cult 2012 2 11 10.9707/2307-0919.1116
37 Lannin DG Ludwikowski WMA Heath PJ et al How are personal values linked to help-seeking stigma? Couns Psychol 2020 48 249 76 10.1177/0011000019884815
38 Rieckhof S Sander C Speerforck S et al Development and validity of the value-based stigma inventory (VASI): a value-sensitive questionnaire for the assessment of mental health stigma BMC Psychiatry 2021 21 570 10.1186/s12888-021-03427-4 34781933
39 Boer D Values and affective well-being: how culture and environmental threat influence their association Roccas S Sagiv L Values and behavior Cham Springer 2017 Available 10.1007/978-3-319-56352-7_927
40 Bojanowska AB Kaczmarek ŁD How healthy and unhealthy values predict hedonic and eudaimonic well-being: dissecting value-related beliefs and behaviours J Happiness Stud 2022 23 211 31 10.1007/s10902-021-00396-z
41 Kazdin AE Mediators and mechanisms of change in psychotherapy research Annu Rev Clin Psychol 2007 3 1 27 10.1146/annurev.clinpsy.3.022806.091432 17716046
42 Gu J Cavanagh K Strauss C Investigating the specific effects of an online mindfulness-based self-help intervention on stress and underlying mechanisms Mindfulness (N Y) 2018 9 1245 57 10.1007/s12671-017-0867-y 30100935
43 Schmidt SJ Schimmelmann BG Mechanisms of change in psychotherapy for children and adolescents: current state, clinical implications, and methodological and conceptual recommendations for mediation analysis Eur Child Adolesc Psychiatry 2015 24 249 53 10.1007/s00787-015-0698-0 25711288
44 Borek AJ Smith JR Greaves CJ et al Developing and applying a framework to understand mechanisms of action in group-based, behaviour change interventions: the MAGI mixed-methods study Efficacy Mech Eval 2019 6 1 162 10.3310/eme06030
45 Hagger MS Moyers S McAnally K et al Known knowns and known unknowns on behavior change interventions and mechanisms of action Health Psychol Rev 2020 14 199 212 10.1080/17437199.2020.1719184 31964227
46 Chen H-T Practical program evaluation: assessing and improving planning, implementation, and effectiveness Thousand Oaks, California SAGE Publications, Inc 2005 Available 10.4135/9781412985444
47 Chen H-T A theory-driven evaluation perspective on mixed methods research Acad J 2006 13 75
48 Pettigrew TF Tropp LR How does intergroup contact reduce prejudice? Meta‐analytic tests of three mediators Euro J Social Psych 2008 38 922 34 10.1002/ejsp.504
49 Potts LC Bakolis I Deb T et al Anti-stigma training and positive changes in mental illness stigma outcomes in medical students in ten countries: a mediation analysis on pathways via empathy development and anxiety reduction Soc Psychiatry Psychiatr Epidemiol 2022 57 1861 73 10.1007/s00127-022-02284-0 35451604
50 Bishop FL Using mixed methods research designs in health psychology: an illustrated discussion from a pragmatist perspective Br J Health Psychol 2015 20 5 20 10.1111/bjhp.12122 25405549
51 Palinkas LA Aarons GA Horwitz S et al Mixed method designs in implementation research Adm Policy Ment Health 2011 38 44 53 10.1007/s10488-010-0314-z 20967495
52 Ivankova NV Creswell JW Stick SL Using mixed-methods sequential explanatory design: from theory to practice Field methods 2006 18 3 20 10.1177/1525822X05282260
53 Schnyder N Panczak R Groth N et al Association between mental health-related stigma and active help-seeking: systematic review and meta-analysis Br J Psychiatry 2017 210 261 8 10.1192/bjp.bp.116.189464 28153928
54 Dobson KS Szeto A The assessment of mental health stigma in the workplace Dobson KS Stuart H The stigma of mental illness: models and methods of stigma reduction New York Oxford University Press 2021 Available 10.1093/med/9780197572597.001.0001
55 Szeto ACH Luong D Dobson KS Does labeling matter? An examination of attitudes and perceptions of labels for mental disorders Soc Psychiatry Psychiatr Epidemiol 2013 48 659 71 10.1007/s00127-012-0532-7 22711063
56 Lindsay BL Dobson K Krupa T et al Under review. A psychometric evaluation of the Opening Minds Scale for Workplace Attitudes (OMS-WA): A measure of public stigma towards mental illness in the workplace
57 Corrigan PW Michaels PJ Vega E et al Self-stigma of mental illness scale--short form: reliability and validity Psychiatry Res 2012 199 65 9 10.1016/j.psychres.2012.04.009 22578819
58 Rüsch N Hölzer A Hermann C et al Self-stigma in women with borderline personality disorder and women with social phobia J Nerv Ment Dis 2006 194 766 73 10.1097/01.nmd.0000239898.48701.dc 17041289
59 Haslam N Kvaale EP Biogenetic explanations of mental disorder: the mixed-blessings model Curr Dir Psychol Sci 2015 24 399 404 10.1177/0963721415588082
60 Lebowitz MS Ahn W Emphasizing malleability in the biology of depression: durable effects on perceived agency and prognostic pessimism Behav Res Ther 2015 71 125 30 10.1016/j.brat.2015.06.005 26112398
61 Zhou Y Lemmer G Xu J et al Cross-cultural measurement invariance of scales assessing stigma and attitude to seeking professional psychological help Front Psychol 2019 10 1249 10.3389/fpsyg.2019.01249 31214074
62 Komiya N Good GE Sherrod NB Emotional openness as a predictor of college students’ attitudes toward seeking psychological help J Couns Psychol 2000 47 138 43 10.1037/0022-0167.47.1.138
63 Apolinário-Hagen J Trachse Dugo A Anhorn L et al Exploring individual differences in online and face-to-face help-seeking intentions in case of impending mental health problems: the role of adult attachment, perceived social support, psychological distress and self-stigma J Health Soc Sci 2016 1 223 40 10.19204/2016/xplr24
64 Vogel DL Wade NG Haake S Measuring the self-stigma associated with seeking psychological help J Couns Psychol 2006 53 325 37 10.1037/0022-0167.53.3.325
65 Kessler EM Agines S Bowen CE Attitudes towards seeking mental health services among older adults: personal and contextual correlates Aging Ment Health 2015 19 182 91 10.1080/13607863.2014.920300 24898327
66 Mackenzie CS Gekoski WL Knox VJ Age, gender, and the underutilization of mental health services: the influence of help-seeking attitudes Aging Ment Health 2006 10 574 82 10.1080/13607860600641200 17050086
67 Chmitorz A Wenzel M Stieglitz R-D et al Population-based validation of a German version of the brief resilience scale PLoS One 2018 13 e0192761 10.1371/journal.pone.0192761 29438435
68 Brähler E Mühlan H Albani C et al Teststatistische Prüfung und Normierung der deutschen Versionen des EUROHIS-QOL Lebensqualität-Index und des WHO-5 Wohlbefindens-Index Diagn 2007 53 83 96 10.1026/0012-1924.53.2.83
69 O’Connor M Casey L The Mental Health Literacy Scale (MHLS): a new scale-based measure of mental health literacy Psychiatry Res 2015 229 511 6 10.1016/j.psychres.2015.05.064 26228163
70 Wulf IC Ruhle SA Psychische Gesundheitskompetenz am Arbeitsplatz: Validierung und Anpassung einer deutschsprachigen Version des Mental Health Literacy tool for the Workplace Diagn 2020 66 50 61 Available 10.1026/0012-1924/a000237
71 Appel AJ Ausprägung und Zusammenhänge psychischer Gesundheitskompetenz mit der Intention zur Hilfesuche bei Studierenden unpublished bachelor’s thesis Greifswald University of Greifswald 2022
72 McLaren T Peter L-J Tomczyk S et al How can the utilisation of help for mental disorders be improved? A quasi-experimental online study on the changeability of stigmatising attitudes and intermediate variables in the process of utilisation BMC Public Health 2021 21 2124 10.1186/s12889-021-12125-5 34798860
73 Birtel MD Oldfield G Affective, cognitive, and behavioral mental illness stigma in health care: a comparison between general ward nurses and the general population Stigma Health 2022 7 380 8 10.1037/sah0000416
74 Swart H Hewstone M Christ O et al Affective mediators of intergroup contact: a three-wave longitudinal study in South Africa J Pers Soc Psychol 2011 101 1221 38 10.1037/a0024450 21728450
75 Stephan WG Stephan CW Intergroup anxiety J Soc Issues 1985 41 157 75 10.1111/j.1540-4560.1985.tb01134.x
76 Davidov E Schmidt P Schwartz SH Bringing values back in: the adequacy of the European social survey to measure values in 20 countries Public Opin Q 2008 72 420 45 10.1093/poq/nfn035
77 Cieciuch J Davidov E A comparison of the invariance properties of the PVQ-40 and the PVQ-21 to measure human values across German and Polish samples Surv Res Methods 2012 6 37 48 Available 10.18148/srm/2012.v6i1.5091
78 Dobewall H Aavik T Rank-order consistency and profile stability of self- and informant-reports of personal values in comparison to personality traits J Individ Differ 2016 37 40 8 10.1027/1614-0001/a000186
79 Kroenke K Spitzer RL Williams JB The PHQ-9: validity of a brief depression severity measure J Gen Intern Med 2001 16 606 13 10.1046/j.1525-1497.2001.016009606.x 11556941
80 Kocalevent RD Hinz A Brähler E Standardization of the depression screener patient health questionnaire (PHQ-9) in the general population Gen Hosp Psychiatry 2013 35 551 5 10.1016/j.genhosppsych.2013.04.006 23664569
81 Rioux C Little TD Missing data treatments in intervention studies: what was, what is, and what should be Int J Behav Dev 2021 45 51 8 10.1177/0165025419880609
82 Kuckartz U Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung Weinheim Beltz Juventa 2016
83 Lagunes-Cordoba E Alcala-Lozano R Lagunes-Cordoba R et al Evaluation of an anti-stigma intervention for Mexican psychiatric trainees Pilot Feasibility Stud 2022 8 5 10.1186/s40814-021-00958-1 35031066
