
==== Front
Trials
Trials
Trials
1745-6215
BioMed Central London

8424
10.1186/s13063-024-08424-y
Study Protocol
Promoting mental health and wellbeing among post-secondary students with the JoyPop™ app: study protocol for a randomized controlled trial
MacIsaac Angela 1
Mann Vamika 1
Toombs Elaine 1
Schmidt Fred 12
Olthuis Janine V. 3
Stewart Sherry H. 4
Newton Amanda 5
Ohinmaa Arto 6
http://orcid.org/0000-0002-4494-1326
Mushquash Aislin R. aislin.mushquash@lakeheadu.ca

1
1 https://ror.org/023p7mg82 grid.258900.6 0000 0001 0687 7127 Department of Psychology, Lakehead University, Thunder Bay, Canada
2 Children’s Centre Thunder Bay, Thunder Bay, Canada
3 https://ror.org/05nkf0n29 grid.266820.8 0000 0004 0402 6152 Department of Psychology, University of New Brunswick, Fredericton, Canada
4 https://ror.org/01e6qks80 grid.55602.34 0000 0004 1936 8200 Departments of Psychiatry and Psychology & Neuroscience, Dalhousie University, Halifax, Canada
5 https://ror.org/0160cpw27 grid.17089.37 Department of Pediatrics, University of Alberta, Edmonton, Canada
6 https://ror.org/0160cpw27 grid.17089.37 School of Public Health, University of Alberta, Edmonton, Canada
2 9 2024
2 9 2024
2024
25 57610 4 2024
23 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Background

Technology use may be one strategy to promote mental health and wellbeing among young adults in post-secondary education settings experiencing increasing distress and mental health difficulties. The JoyPop™ app is mobile mental health tool with a growing evidence base. The objectives of this research are to (1) evaluate the effectiveness of the JoyPop™ app in improving emotion regulation skills (primary outcome), as well as mental health, wellbeing, and resilience (secondary outcomes); (2) evaluate sustained app use once users are no longer reminded and determine whether sustained use is associated with maintained improvements in primary and secondary outcomes; (3) determine whether those in the intervention condition have lower mental health service usage and associated costs compared to those in the control condition; and (4) assess users’ perspectives on the quality of the JoyPop™ app.

Methods

A pragmatic, parallel arm randomized controlled trial will be used. Participants will be randomly allocated using stratified block randomization in a 1:1 ratio to the intervention (JoyPop™) or control (no intervention) condition. Participants allocated to the intervention condition will be asked to use the JoyPop™ app at least twice daily for 4 weeks. Participants will complete outcome measures at four assessment time-points (first [baseline], second [after 2 weeks], third [after 4 weeks], fourth [after 8 weeks; follow-up]). Participants in the control condition will be offered access to the app after the fourth assessment time-point.

Discussion

Results will determine the effectiveness of the JoyPop™ app for promoting mental health and wellbeing among post-secondary students. If effective, this may encourage more widespread adoption of the JoyPop™ app by post-secondary institutions as part of their response to student mental health needs.

Trial registration

ClinicalTrials.gov NCT06154369. Registered on November 23, 2023.

Keywords

Emotion regulation
Mental health
Students
Post-secondary
eHealth
mHealth
Mental health promotion
issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcAdministrative information

Note: the numbers in curly brackets in this protocol refer to SPIRIT checklist item numbers. The order of the items has been modified to group similar items (see http://www.equator-network.org/reporting-guidelines/spirit-2013-statement-defining-standard-protocol-items-for-clinical-trials/). Title {1}	Promoting mental health and wellbeing among post-secondary students with the JoyPop™ app: study protocol for a randomized controlled trial	
Trial registration {2a and 2b}	ClinicalTrials.gov NCT06154369. Registered on November 23, 2023.	
Protocol version {3}	August 9, 2024; version 2	
Funding {4}	Brain Canada—Future Leaders in Canadian Brain Research Program	
Author details {5a}	AM: Department of Psychology, Lakehead University

VM: Department of Psychology, Lakehead University

ET: Department of Psychology, Lakehead University

FS: Department of Psychology, Lakehead University; Children’s Centre Thunder Bay

JO: Department of Psychology, University of New Brunswick

SHS: Departments of Psychiatry and Psychology & Neuroscience, Dalhousie University

AN: Department of Pediatrics, University of Alberta

AO: School of Public Health, University of Alberta

ARM: Department of Psychology, Lakehead University

	
Name and contact information for the trial sponsor {5b}	Aislin R. Mushquash

Department of Psychology, Lakehead University

955 Oliver Road, Thunder Bay, Ontario, P7B5E1

Email: aislin.mushquash@lakeheadu.ca

	
Role of sponsor {5c}	The funder has no role in the study design; data collection, management, or analysis; or in the writing and publishing of reports.	

Introduction

Background and rationale {6a}

Canadian youth and young adults have experienced an increase in mental health difficulties over the past several years [1, 2]. Post-secondary students in this age group are no exception, with the number of students with mental health symptoms and diagnoses significantly increasing between 2013 and 2019 according to population-level data [3]. Further, symptoms of anxiety, depression, and stress rose among Canadian post-secondary students compared to other young adults during the COVID-19 pandemic [4–7], though understanding the longer-term outcomes is still evolving. Service utilization has also increased among youth and young adults, including counseling services and hospital visits [3, 8]. While most Canadian post-secondary institutions offer campus mental health supports to address these concerns [9], specific services vary by institution, and formal assessment, service evaluation, and early identification procedures are sometimes lacking [10, 11]. Many students believe their institution should increase mental health resources [12], while some young adults perceive barriers to using mental health services including perceived time constraints or concerns about privacy and stigma [13, 14].

Mobile apps can increase access to and engagement in mental health-related interventions [15]. Many apps exist, with meta-analyses and systematic reviews suggesting mental health-focused apps improve mental health-related outcomes such as depression, anxiety, stress, and quality of life, with small to large effects [16–18]. Findings also generally suggest that post-secondary students are open to using apps to address their mental health concerns [19]. While promising in their impact, mobile mental health apps are heterogenous in terms of their specific interventions (e.g., mindfulness techniques, stress reduction), and there are many apps that have not been adequately evaluated [20]. For these reasons, it is imperative to build the evidence base for any new app prior to widespread implementation.

The JoyPop™ app was co-designed for youth and young adults by researchers, youth, and service providers [21]. It contains features (see Fig. 1) that promote wellbeing and resilience by targeting emotion regulation [22]. Considered a transdiagnostic risk factor, emotion regulation is linked to many mental health difficulties such as anxiety and depression and is directly related to one’s ability to cope with stress [23], thus rendering it a relevant target for intervention among post-secondary students. Understanding and managing one’s emotions is also linked to better academic performance [24, 25], further emphasizing the importance of emotion regulation for students.Fig. 1 The JoyPop app

Initial study of the JoyPop™ app was conducted with first-year university students to understand whether the app would support them in terms of the stressors and challenges associated with transitioning to university [22]. Results demonstrated positive changes in emotion regulation and depressive symptoms with each additional day of app usage over a 4-week period, suggesting those who used the app for more days experienced greater benefit [22]. In qualitative studies examining the acceptability and utility of the app, users expressed that the app had several positive outcomes such as increasing self-monitoring and awareness, improving emotion regulation, and providing helpful distraction [26, 27].

While promising, prior research has lacked a control group of participants who do not receive access to the app to more rigorously evaluate mental health and wellbeing outcomes associated with the JoyPop™ app. Study of whether app benefits are sustained when users are no longer reminded to use the JoyPop™ app is also needed. Additionally, while smaller samples have qualitatively provided feedback on the acceptability and quality of the app, quantitative metrics assessing perceptions of app quality from a larger sample would provide a more robust understanding of the user experience. Lastly, while the JoyPop™ app is theorized to support student mental health needs, formal evaluations of whether the app reduces the need for, and cost associated with, other mental health services have not been conducted. The present study aims to address these gaps via a randomized controlled trial design.

Objectives {7}

The first objective of this research is to evaluate the effectiveness of the JoyPop™ app in improving post-secondary students’ emotion regulation skills (primary outcome), as well as their mental health, wellbeing, and resilience (secondary outcomes). Prior research suggests that the JoyPop™ app will be associated with positive change in emotion regulation and mental health over time as compared to no intervention. We also expect app use to result in improved resilience and wellbeing as the app was designed to support adaptation and perseverance in the face of stress [21, 22, 28]. The second, third, and fourth objectives are more exploratory. The second objective is to evaluate sustained app use once users are no longer reminded and determine whether sustained use is associated with maintained benefit in outcomes. The third objective is to determine whether those using the app have lower mental health service usage and associated costs. The fourth objective is to assess users’ perspectives on the quality of the JoyPop™ app.

Trial design {8}

This study design is a pragmatic, parallel arm randomized controlled superiority trial. Stratified block randomization will be used to randomize participants in a 1:1 ratio to either the intervention (JoyPop™) or control (no intervention) condition. Those in the intervention condition will be asked to use the JoyPop™ app at least twice per day for 4 weeks. Primary (emotion regulation) and secondary (mental health, wellbeing, and resilience) outcomes and mental health service usage (number of visits to various service providers) will be assessed among participants at four assessments time-points (first [baseline], second [after 2 weeks], third [after 4 weeks], fourth [after 8 weeks; follow-up]). Participants in the intervention condition will complete measures of app quality at the third assessment time-point. The Standard Protocol Items: Recommendations for Intervention Trials (SPIRIT) guidelines are being followed. The trial was registered with ClinicalTrials.gov (NCT06154369). All procedures have been reviewed and approved by the Research Ethics Board at the Thunder Bay Regional Health Sciences Centre (acting as the Board of Record for Lakehead University clinical research projects).

Methods: participants, interventions, and outcomes

Study setting {9}

All data will be collected at Lakehead University, a public university with a main campus in Thunder Bay, Ontario, Canada, and a smaller campus in Orillia, Ontario, Canada. Participants will be drawn from the undergraduate student population of approximately 7000 students across both campuses. The first (baseline) assessment time-point will be on-campus in a laboratory setting or virtually via Zoom depending on student preference, whether the student requires use of an iPhone to take part in the study (which would necessitate an on-campus visit to secure the phone), and depending on their location (i.e., those in Orillia will occur via Zoom). Subsequent assessment time-points will be administered online.

Eligibility criteria {10}

Participants are required to be between the ages of 18–25 years, enrolled as an undergraduate student at Lakehead University, and not have participated in any other study that evaluated the JoyPop™ app. Students from the Orillia campus will require access to their own iOS device.

Who will take informed consent? {26a}

Students will attend a study orientation session either in-person or virtually, during which time a research assistant will explain the study and answer questions. Students will also receive an information letter and consent form and will be given time to review and ask any questions of the research assistant before proceeding. Undergraduate research assistants will have undergone training including shadowing the graduate student researcher and being observed to ensure consistency in protocol administration. All research team members will also have completed the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans training.

Additional consent provisions for collection and use of participant data and biological specimens {26b}

N/A. No additional consent provisions are needed, and no biological samples will be collected.

Interventions

Explanation for the choice of comparators {6b}

The control condition (no intervention) was chosen to account for natural changes over time and regression to the mean expected in outcomes such as emotion regulation [29]. Comparing use of the JoyPop™ app to receiving no intervention creates the opportunity to test whether those who receive the app experience improved outcomes compared to those who do not receive it.

Intervention description {11a}

Participants in the intervention group will download and use the JoyPop™ app. Participants will be asked to use the app twice daily for 4 weeks, with no other requirements specifying their usage. The JoyPop™ app features are Rate My Mood, SleepEase, Breathing Exercises, SquareMoves, Journaling, Art, Circle of Trust, and Call for Help (see Fig. 1). Participants from the Thunder Bay campus who do not have an iOS device will be given a restricted-access iPhone to borrow for the study, with only the JoyPop™ app enabled.

Individuals in the control condition will not receive any intervention.

Criteria for discontinuing or modifying allocated interventions {11b}

Participants may request to withdraw from the study at any time by contacting the research team via email; otherwise, there are no pre-specified criteria for modifying or discontinuing the intervention.

Strategies to improve adherence to interventions {11c}

Those in the intervention condition will receive a text message each morning and evening (8 am and 8 pm) reminding them to use the app.

Relevant concomitant care permitted or prohibited during the trial {11d}

Participants will not be prohibited from accessing any concomitant care or other interventions throughout the study. The use of other mental health services will be evaluated as part of the third study objective to determine whether those who received the app have lower mental health service usage and associated costs.

Provisions for post-trial care {30}

While no harm associated with study participation is expected, there is a risk participants may feel distressed answering questions related to emotion regulation, mental health, and wellbeing. The information letter will include contact information for mental health support services, including local counseling services and telephone mental health hotlines.

Outcomes {12}

Primary outcomes

The primary outcome is change in emotion regulation over time. Emotion regulation will be measured at each of the four assessment time-points using the Difficulties in Emotion Regulation Scale—Short Form (DERS-SF) [30, 31]. We will examine change in overall emotion regulation via the total DERS-SF score and change in specific domains via the DERS-SF subscales (i.e., strategy use related to emotion regulation, non-acceptance of emotion, impulsivity, focus on goals, awareness of emotion, and clarity of emotion). Evaluating emotion regulation is an important outcome as it is a transdiagnostic risk factor relevant across many mental health difficulties [23] and as it is related to academic success [24, 25].

Secondary outcomes

Secondary outcomes are change in mental health, wellbeing, and resilience over time. Each of these variables will be measured at each of the four assessment time-points. Mental health, consisting of psychological distress, depressive symptoms, anxious symptoms, and stress, will be measured with the Depression, Anxiety, and Stress Scale 21 (DASS-21) [32]. Psychological distress will be assessed by the total score on the DASS-21, while depression, anxiety, and stress symptoms are measured by their respective subscale scores. General mental health will be measured using the total score from the General Health Questionnaire 12 (GHQ-12) [33]. These aspects of mental health are important to assess as many students face mental health difficulties and prior research suggests the JoyPop™ app may impact mental health symptoms [22].

Wellbeing will be measured using the total score from the Warwick Edinburgh Mental Wellbeing Scale [34, 35]. It is important to measure the impact of the JoyPop™ app on wellbeing as it is theorized to represent an element of mental functioning that is partially independent from the presence/absence of mental illness among Canadian students [28, 36]. Resilience will be measured using the Connor-Davidson Resilience Scale-10 (CD-RISC-10) [37]. Resilience is important as the ability to adapt and thrive in the face of stress and adversity [38] is relevant to students dealing with stressors. The JoyPop™ app was designed with resilience theory in mind [21].

Exploratory data

Primary and secondary outcomes will be assessed at the fourth assessment time-point as part of the exploratory objective of evaluating sustained app use and determining whether this use is associated with maintained benefit in primary and secondary outcomes. Self-reported mental health service utilization will be measured at each of the four time-points using a brief measure developed by the research team. Change in both total frequency of health service use and individual service provider visits will be calculated.

App quality will be evaluated during the third assessment time-points among those using the intervention. Quality will be assessed using the User Version of the Mobile Application Rating Scale (UMARS) [39, 40]. Total scores and subscale scores (i.e., engagement, functionality, esthetics, information, and perceived impact) will be calculated. App quality is relevant because a smartphone intervention is effective to the extent that the user is willing to engage with it and continues to use the app over time [41]. Thus, the target population for the JoyPop™ app must experience it being good enough quality to use and benefit from. Using the UMARS will allow us to compare app quality for the JoyPop™ app relative to other similar apps for which UMARS data have been published (e.g., Thabrew et al. [42]).

Participant timeline {13}

See Fig. 2.Fig. 2 SPIRIT schedule of enrollment, intervention, and assessments. *Week 0 (enrollment) includes any time prior to starting the study. **Intervention group only

Sample size {14}

Our past research suggests a medium effect for the primary outcome [22]. A sample size of 120 will be required to achieve power to detect a small to medium effect in the primary outcome when using a 2 × 4 (group by assessment time-point) design [43]. We had good retention in our past research, with 79% of participants completing outcome measures at baseline, after 2 weeks of app use, and after 4 weeks of app use [22]. However, our prior research did not involve a control group or a follow-up assessment after 8 weeks, both of which are likely to impact retention. Other app-based trials show retention rates between 65 and 80% [44]. Consistency and long-term engagement with apps outside of a research context is low, with 25% of users abandoning apps after one use [45]. Considering this information, we conservatively estimated 60% retention throughout the study (40% attrition). As such, we will recruit an initial sample of 160.

Recruitment {15}

Potential participants will be recruited through advertisements on social media (e.g., Facebook, Instagram), emails to students sent by course instructors, announcements made in classes, flyers distributed to students on campus, and posters displayed around campus. Advertisements will contain a QR code directing interested participants to an online initial interest survey which collects their basic contact information and eligibility information so that the research team can initiate contact over email or phone for the eligibility screen. Interested students can alternatively email the research team expressing interest in participating. Students enrolled in psychology classes at Lakehead University can obtain bonus points for participating in research studies; as such, we will advertise on the university’s online research sign-up system.

Assignment of interventions: allocation

Sequence generation {16a}

Stratified block randomization will be used to randomly assign participants to either the intervention condition or control condition in a 1:1 ratio.

Concealment mechanism {16b}

Random assignments based on the allocation sequence will be placed into numbered, opaque, sealed envelopes that will be kept in a locked filing cabinet. After a student consents to participate and completes the measures, the research assistant will access the next sealed envelope.

Implementation {16c}

A researcher not involved in the trial will use a computer-generated sequencing tool to create the allocation sequence [46]. During the first assessment time-point, the research assistant will access the allocation envelope from the locked filing cabinet, open it to determine the allocation, inform the participant, and proceed with associated study tasks.

Assignment of interventions: blinding

Who will be blinded {17a}

Trial participants themselves cannot be blinded to their condition due to the nature of the intervention and control conditions (i.e., using an app vs. no intervention). Research assistants who directly interact with participants will also not be blinded. The principal investigator and co-investigators will be blinded to condition assignment.

Procedure for unblinding if needed {17b}

If an adverse event were to occur, the participant’s allocation may be revealed to the principal investigator by the research assistant and/or by the participant.

Data collection and management

Plans for assessment and collection of outcomes {18a}

Assessment data will be collected online through SurveyMonkey [47]. During the orientation session, the research assistant will explain how to complete the assessments and participants can ask questions if they are unsure of how to answer a question. The research assistant will also explain that participants can expect to see similar measures at each subsequent assessment time-point. In addition to data collection through SurveyMonkey, the JoyPop™ app records usage data indicating when and for how long the app was accessed by participants. This will be used to measure sustained app use for the exploratory analysis of whether benefits in primary and secondary outcomes are maintained during the follow-up period.

Demographic information will be collected at the first assessment time-point using five items assessing age, race/ethnicity, gender identity, biological sex, and current year of study in university.

Emotion regulation will be measured with the DERS-SF [30, 31], an 18-item measure that asks participants to rate how much various statements regarding difficulty regulating emotion have applied to them over the last 2 weeks. Items are rated on a scale from 1—“Almost Never” to 5—“Almost Always.” Total scores range from 18 to 90, with higher scores indicating greater difficulty regulating one’s emotions. Subscales measure strategies (e.g., “When I was upset, it took me a long time to feel better”), non-acceptance (e.g., “When I was upset, I felt guilty for feeling that way”), impulse (e.g., “When I was upset, I lost control over my behaviour”), goals (e.g., “When I was upset, I had difficulty focusing on other things”), awareness (e.g., “I cared about what I was feeling”), and clarity (e.g., “I was confused about how I feel”). The DERS-SF is positively correlated with symptoms of emotional disorders and demonstrates high internal consistency [31].

Mental health data will be collected using the DASS-21 [32], a 21-item measure that asks participants to rate how much various statements have been relevant for them in the last week. Items are rated on a scale from 0—“Did not apply to me at all (never)” to 3—“Applied to me very much, or most of the time (almost always).” Total scores range from 0 to 63, with higher scores indicating greater psychological distress. In addition to psychological distress, subscales include depressive symptoms (e.g., “I felt that I had nothing to look forward to”), anxiety symptoms (e.g., “I felt I was close to panic”), and stress (e.g., “I found it difficult to relax”). The DASS-21 is positively correlated with other measures of depression, anxiety, and stress in university students and demonstrates high internal consistency [48]. The GHQ-12 [26] is a 12-item measure that asks participants to rate how much they have recently had various feelings or experiences related to general mental health difficulties (e.g., “Felt constantly under strain”). Items are rated on a scale from 0—“Less than usual” to 3—“Much more than usual.” It is correlated with measures of depression and mental health and demonstrates high internal consistency [49].

Wellbeing data will be collected via the Warwick Edinburgh Mental Wellbeing Scale [34, 35], a 14-item measure that asks individuals to rate how often they have had certain feelings and experiences related to wellbeing over the last 2 weeks. Items (e.g., “I’ve been dealing with problems well”) are rated on a scale from 1—“None of the time” to 5—“All of the time.” Total scores range from 14 to 70 with higher scores indicating greater wellbeing. This measure is positively correlated with other measures of wellbeing and related constructs of happiness and life satisfaction; it also demonstrates high internal consistency [34].

Resilience data will be collected using the CD-RISC-10 [37], a 10-item measure that asks individuals to rate their agreement with statements related to resilience reflecting on the last 2 weeks. Items (e.g., “I am able to adapt when changes occur”) are rated on a scale from 0—“Not at all true” to 4—“True nearly all the time.” Total scores range from 0 to 40 with higher scores indicating greater resilience. The CD-RISC-10 is positively correlated with social support and hardiness, and negatively with stress vulnerability [37] and has high internal consistency [22].

Mental health service usage will be collected using five questions asking the number of visits to various service providers for mental health reasons over the last 2 weeks, including walk-in clinic, family doctor or nurse practitioner, emergency department, mental health counselor, and mental health hotline/phone support. App quality data will be collected via the UMARS [39, 40], a 27-item measure that asks individuals to rate app quality within various domains. Items are rated on a 5-point scale with 5 generally representing the most positive quality rating. Total scores range from 22 to 130, with higher scores indicating higher perceived app quality. Measure subscales assess engagement (e.g., “Is the app interesting to use? Does it present its information in an interesting way compared to other similar apps?”), functionality (e.g., “How easy is it to learn how to use the app; how clear are the menu labels, icons and instructions?”), esthetics (e.g., “Is arrangement and size of buttons, icons, menus and content on the screen appropriate?”), information (e.g., “Is the information within the app comprehensive but concise?”), subjective quality (e.g., “What is your overall (star) rating of the app?”), and perceived impact (e.g., “This app has increased my knowledge/understanding of mental health”). The UMARS demonstrates high internal consistency and acceptable test–retest reliability [39].

Plans to promote participant retention and complete follow-up {18b}

Email reminders will be sent to participants to promote retention and completion of the measures at the four assessment time-points. Additional reminders will be sent each day for up to 2 days following the initial email if the participant has not completed the corresponding assessment. To incentivize completion, participants will receive cash compensation ($20 for first assessment time-point, $25 for second assessment time-point, $30 for third assessment time-point, $25 for forth assessment time-point) or one bonus point per assessment time-point on the university’s online research sign-up system.

Data management {19}

Data will be stored online using SurveyMonkey, which has its server located in the USA. SurveyMonkey’s data centers are SOC-2 accredited and are protected by continuous security monitoring, cameras, visitor logs, and entry limitations. Further information on storage and security is available on their website [50]. Data in transit are encrypted with secure TLS cryptographic protocols. App usage data will be stored within a password-protected Canadian server and encrypted during transmission. Assessment data and app usage data will be downloaded only by research team members ARM or VM onto a password-protected computer. Data will be stored for at least 5 years following data collection as specified by Lakehead University policy.

Confidentiality {27}

Names and contact information for interested and enrolled participants will be gathered through recruitment methods and will be kept in a tracking list accessed only by the research assistants. Those enrolled will be given an ID number and this will be added to the tracking list. Assessment and app usage data will be kept in a separate spreadsheet, organized by ID number. The spreadsheet will not contain any identifying information. Names and contact information will be deleted from the tracking list upon completion of data collection. All procedures pertaining to confidentiality will be conducted in accordance with the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans and in accordance with the protocols approved through the Thunder Bay Regional Health Sciences Centre and Lakehead University Research Ethics Boards.

Plans for collection, laboratory evaluation, and storage of biological specimens for genetic or molecular analysis in this trial/future use {33}

N/A. Biological specimens will not be collected.

Statistical methods

Statistical methods for primary and secondary outcomes {20a}

Linear mixed modeling will be conducted to determine whether the change over time in the primary outcome (emotion regulation) and secondary outcomes (mental health, wellbeing, and resilience) is greater for those in the intervention condition. This analytic approach is appropriate for longitudinal data as it effectively models the variation in an outcome attributed to both within-individual factors and between-individual differences (i.e., attributed to group assignment) [51].

Interim analyses {21b}

N/A. There are no interim analyses or stopping guidelines planned.

Methods for additional analyses (e.g., subgroup analyses) {20b}

There are no planned subgroup analyses. Subgroup analyses may be conducted based on evolving research questions of interest and pending adequate power.

Methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data {20c}

Analyses will be conducted using an intention-to-treat approach. An attrition analysis will be conducted to compare participants who provided data at all assessment time-points to those who did not complete measures at one or more assessment time-points. Missing data will be handled using full information maximum likelihood estimation [52, 53].

Plans to give access to the full protocol, participant-level data, and statistical code {31c}

The protocol is registered on clinicaltrials.gov.

Oversight and monitoring

Composition of the coordinating center and trial steering committee {5d}

Principal investigator ARM will provide oversight on all trial matters. The graduate student research assistant (VM) will lead data collection and coordinate participant enrolment and will support and train undergraduate research assistants. Meetings involving the principal investigator and research assistants will occur weekly to discuss trial progress and matters related to recruitment and retention. ARM will also meet with co-investigators (JO, ET, FS, AO, AN, and SHS) on a quarterly basis to consult about ongoing implementation and plan for future trial activities. ARM also meets with a local group of healthcare administrators, frontline staff, youth, and caregivers (functioning as a Stakeholder and Public Involvement Group) quarterly to discuss local implementation and evaluation of the JoyPop™ app across settings.

Composition of the data monitoring committee, its role and reporting structure {21a}

A formal data monitoring committee was not deemed necessary as minimal risk is involved in participation. Determination of minimal risk of harm is supported by prior study results which did not suggest any unfavorable impact on participants or relevant outcomes [22]. Aspects of data collection will be informally monitored by the research team, including recruitment and participants’ completion of assessment time-points.

Adverse event reporting and harms {22}

Participation is associated with minimal risk, particularly considering prior findings which did not indicate any unfavorable changes in outcomes most likely to be impacted by the intervention [22]. Adverse events identified among or reported by participants will be reported by the research assistants to the principal investigator immediately. The principal investigator will also inquire about any adverse events during weekly meetings with the research assistants. If an adverse event were to occur, the principal investigator would report it immediately to the Thunder Bay Regional Health Sciences Centre Research Ethics Board using the Research Ethics Local Serious Adverse Event Reporting Form.

Frequency and plans for auditing trial conduct {23}

There are no formal plans for auditing by an independent party. During weekly research team meetings, the principal investigator will ensure the protocol is being followed during the running of the trial. Research assistants will be trained on the protocol and quality and ethical standard of clinical trials [54]. They will be observed conducting study tasks prior to independent administration to ensure the protocol is followed.

Plans for communicating important protocol amendments to relevant parties (e.g., trial participants, ethical committees) {25}

If needed, modifications to the protocol would be discussed with co-investigators during quarterly meetings or over email. Changes would be submitted to the Research Ethics Board at the Thunder Bay Regional Health Sciences Centre and would be implemented once approved. The protocol registration would also be revised, and participants would be informed by phone or email if needed.

Dissemination plans {31a}

All participants will be asked if they would like to opt in to receive a summary of the research findings following study completion. This information will be kept in a separate document that is not linked to their assessment data, app usage data, or ID numbers. The research summary will be sent to interested participants via email. Results from this trial will be published in peer-reviewed, open-access journals, presented at relevant conferences, and shared with the Lakehead University community. Media interviews may also be used as a form of dissemination.

Discussion

Canadian young adults who are completing post-secondary education are facing increasing mental health concerns [3]. Barriers exist with respect to student mental health needs being fully addressed by institutional resources [11, 16], suggesting novel, accessible, and low-cost approaches to proactively promote mental health may be of benefit to students [12, 19]. This trial will evaluate the effectiveness of the JoyPop™ app as a tool to promote improved emotion regulation, mental health, wellbeing, and resilience. Results may support wider uptake of the app by post-secondary institutions in supporting students, particularly if the app is also associated with lower health service utilization and if users view the app as high-quality and are willing to adopt it.

Trial status

The trial study protocol was approved by the Research Ethics Board at the Thunder Bay Regional Health Sciences Centre (acting as the Board of Record for Lakehead University clinical research projects; #100,251).

Protocol version 2; August 9, 2024.

Trial start date: November 22, 2023.

Anticipated recruitment completion: December 2024.

Abbreviations

iOS device IPad, iPod, iPhone

DERS-SF Difficulties in Emotion Regulation Scale—Short Form

DASS-21 Depression, Anxiety, and Stress Scale 21

GHQ-12 General Health Questionnaire

UMARS User Version of the Mobile Application Rating Scale

CD-RISC-10 Connor-Davidson Resilience Scale-10

Acknowledgements

We would like to thank the youth who informed the initial design of the JoyPop™ app and those who continue to inform its evolution through participation in past and current research. We would also like to acknowledge the undergraduate student research assistants involved in data collection for their integral role in this study.

Authors’ contributions {31b}

ARM is the principal investigator and with VM initially conceived the study and led its design. ET, JO, FS, AN, AO, and SHS also provided design contributions. ARM, ET, JO, AO, FS, AN, and SHS sought funding for the study. VM is leading trial coordination and data collection while ARM provides supervision and oversight. AN and SHS provide mentorship to ARM and her team throughout the trial. AM drafted the initial version of the manuscript and ARM revised the manuscript. All authors read and provided feedback on the final manuscript.

Authors’ information

ARM is an Associate Professor in the Department of Psychology at Lakehead University and Clinical Psychologist at Dilico Anishinabek Family Care in Thunder Bay, Ontario. AM and VM are both clinical psychology PhD students in the Department of Psychology at Lakehead University. FS is an Adjunct Professor in the Department of Psychology at Lakehead University. ET is an Adjunct Professor in the Department of Psychology at Lakehead University and Clinical Psychologist at Dilico Anishinabek Family Care. JO is an Associate Professor in the Department of Psychology at the University of New Brunswick. SHS is a Professor in the Departments of Psychiatry and Psychology & Neuroscience at Dalhousie University and a Canada Research Chair (Tier 1) in Addictions and Mental Health and Fellow of the Royal Society of Canada. AN is a Professor in the Department of Pediatrics at the University of Alberta. AO is a Professor in the School of Public Health at the University of Alberta.

Funding {4}

This trial is funded by the Brain Canada—2021 Future Leaders in Canadian Brain Research grant (ARM). We also acknowledge the support provided through a Tier 1 Canada Research Chair in Addictions and Mental Health (SHS).

Availability of data and materials {29}

The final trial dataset will be accessed by the principal investigator ARM as well as graduate student researcher VM.

Declarations

Ethics approval and consent to participate {24}

The Research Ethics Board at the Thunder Bay Regional Health Sciences Centre, which acts as the Board of Record for clinical research projects at Lakehead University, has reviewed and approved all study materials (file #100251). Informed consent will be obtained from all participants; a detailed verbal and written description of the study will be provided and participants will complete a consent form.

Consent for publication {32}

N/A. We have not included any details, images, or videos relating to an individual person. ARM will provide a model consent form upon request.

Competing interests {28}

The majority of the authors have no potential or actual competing interests to declare. The principal investigator (ARM) recently acquired intellectual property ownership rights for the JoyPop™ app (in June 2024). Prior to this time, intellectual property rights were owned by another researcher/institution. ARM did not own any intellectual property rights when applying for funding for this study, when creating the protocol, or when initiating the study. To mitigate risk related to ARM’s new intellectual property ownership rights, ARM will not be involved in collecting or analyzing the data and will consult with co-investigators throughout the study.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Chiu M Gatov E Fung K Kurdyak P Guttmann A Deconstructing the rise in mental health–related ED visits among children and youth in Ontario Canada Health Aff 2020 39 10 1728 1736 10.1377/hlthaff.2020.00232
Chiu M, Gatov E, Fung K, Kurdyak P, Guttmann A. Deconstructing the rise in mental health–related ED visits among children and youth in Ontario. Canada Health Aff. 2020;39(10):1728–36.10.1377/hlthaff.2020.00232
2. Wiens K Bhattarai A Pedram P Dores A Williams J Bulloch A Patten S A growing need for youth mental health services in Canada: examining trends in youth mental health from 2011 to 2018 Epidemiol Psychiatr Sci 2020 17 29 e115 10.1017/S2045796020000281
Wiens K, Bhattarai A, Pedram P, Dores A, Williams J, Bulloch A, Patten S. A growing need for youth mental health services in Canada: examining trends in youth mental health from 2011 to 2018. Epidemiol Psychiatr Sci. 2020;17(29):e115.10.1017/S2045796020000281
3. Linden B Boyes R Stuart H Cross-sectional trend analysis of the NCHA II survey data on Canadian post-secondary student mental health and wellbeing from 2013 to 2019 BMC Public Health 2021 21 590 10.1186/s12889-021-10622-1 33765965
Linden B, Boyes R, Stuart H. Cross-sectional trend analysis of the NCHA II survey data on Canadian post-secondary student mental health and wellbeing from 2013 to 2019. BMC Public Health. 2021;21:590.33765965 10.1186/s12889-021-10622-1
4. Gouin JP MacNeil S De la Torre-Luque A Chartrand E Chadi N Rouquette A Boivin M Côté S Geoffroy MC Depression, anxiety, and suicidal ideation in a population-based cohort of young adults before and during the first 12 months of the COVID-19 pandemic in Canada Can J Public Health 2023 114 3 368 377 10.17269/s41997-023-00772-7 37093498
Gouin JP, MacNeil S, De la Torre-Luque A, Chartrand E, Chadi N, Rouquette A, Boivin M, Côté S, Geoffroy MC. Depression, anxiety, and suicidal ideation in a population-based cohort of young adults before and during the first 12 months of the COVID-19 pandemic in Canada. Can J Public Health. 2023;114(3):368–77.37093498 10.17269/s41997-023-00772-7
5. King N Pickett W Rivera D Byun J Li M Cunningham S Duffy A The impact of the COVID-19 pandemic on the mental health of first-year undergraduate students studying at a major Canadian university: a successive cohort study Can J Psychiatry 2023 68 7 499 509 10.1177/07067437221094549 35450455
King N, Pickett W, Rivera D, Byun J, Li M, Cunningham S, Duffy A. The impact of the COVID-19 pandemic on the mental health of first-year undergraduate students studying at a major Canadian university: a successive cohort study. Can J Psychiatry. 2023;68(7):499–509.35450455 10.1177/07067437221094549
6. Sheikhan NY Hawke LD Ma C Courtney D Szatmari P Cleverley K Voineskos A Cheung A Henderson J A longitudinal cohort study of youth mental health and substance use before and during the COVID-19 pandemic in Ontario, Canada: an exploratory analysis Can J Psychiatry 2022 67 11 842 854 10.1177/07067437221097906
Sheikhan NY, Hawke LD, Ma C, Courtney D, Szatmari P, Cleverley K, Voineskos A, Cheung A, Henderson J. A longitudinal cohort study of youth mental health and substance use before and during the COVID-19 pandemic in Ontario, Canada: an exploratory analysis. Can J Psychiatry. 2022;67(11):842–54.10.1177/07067437221097906
7. Smith AC, Filice D, Poole H, Khan A, Whalen K, Smilek D. Indicators of student well-being in Canadian undergraduates before and during the COVID-19 pandemic. Scholarsh Teach Learn Psychol. 2022. https://psycnet.apa.org/doiLanding?doi=10.1037%2Fstl0000338.
8. Gandhi S Chiu M Lam K Cairney JC Guttmann A Kurdyak P Mental health service use among children and youth in Ontario: population-based trends over time Can J Psychiatry 2016 61 2 119 124 10.1177/0706743715621254 27253703
Gandhi S, Chiu M, Lam K, Cairney JC, Guttmann A, Kurdyak P. Mental health service use among children and youth in Ontario: population-based trends over time. Can J Psychiatry. 2016;61(2):119–24.27253703 10.1177/0706743715621254
9. Jaworska N De Somma E Fonseka B Heck E MacQueen GM Mental health services for students at postsecondary institutions: a national survey Can J Psychiatry 2016 61 12 766 775 10.1177/0706743716640752 27310230
Jaworska N, De Somma E, Fonseka B, Heck E, MacQueen GM. Mental health services for students at postsecondary institutions: a national survey. Can J Psychiatry. 2016;61(12):766–75.27310230 10.1177/0706743716640752
10. Heck E Jaworska N DeSomma E Dhoopar AS MacMaster FP Dewey D MacQueen G A survey of mental health services at post-secondary institutions in Alberta Can J Psychiatry 2014 59 5 250 258 10.1177/070674371405900504 25007278
Heck E, Jaworska N, DeSomma E, Dhoopar AS, MacMaster FP, Dewey D, MacQueen G. A survey of mental health services at post-secondary institutions in Alberta. Can J Psychiatry. 2014;59(5):250–8.25007278 10.1177/070674371405900504
11. Read A Lutgens D Malla A A descriptive overview of mental health services offered in post-secondary educational institutions across Canada Can J Psychiatry 2023 68 2 101 108 10.1177/07067437221128168 36200430
Read A, Lutgens D, Malla A. A descriptive overview of mental health services offered in post-secondary educational institutions across Canada. Can J Psychiatry. 2023;68(2):101–8.36200430 10.1177/07067437221128168
12. Moghimi E Stephenson C Gutierrez G Jagayat J Layzell G Patel C McCart A Gibney C Langstaff C Ayonrinde O Khalid-Khan S Mental health challenges, treatment experiences, and care needs of post-secondary students: a cross-sectional mixed-methods study BMC Public Health 2023 23 1 655 10.1186/s12889-023-15452-x 37020282
Moghimi E, Stephenson C, Gutierrez G, Jagayat J, Layzell G, Patel C, McCart A, Gibney C, Langstaff C, Ayonrinde O, Khalid-Khan S. Mental health challenges, treatment experiences, and care needs of post-secondary students: a cross-sectional mixed-methods study. BMC Public Health. 2023;23(1):655.37020282 10.1186/s12889-023-15452-x
13. Murray JK Knudson S Mental health treatment and access for emerging adults in Canada: a systematic review Front Public Health 2023 11 1088999 10.3389/fpubh.2023.1088999 37501945
Murray JK, Knudson S. Mental health treatment and access for emerging adults in Canada: a systematic review. Front Public Health. 2023;11:1088999.37501945 10.3389/fpubh.2023.1088999
14. Robinson AM, Jubenville TM, Renny K, Cairns SL. Academic and mental health needs of students on a Canadian campus. Can J Couns Psychother. 2016;50(2):108–23.
15. Price M Yuen EK Goetter EM Herbert JD Forman EM Acierno R Ruggiero KJ mHealth: a mechanism to deliver more accessible, more effective mental health care Clin Psychol Psychother 2014 21 5 427 436 10.1002/cpp.1855 23918764
Price M, Yuen EK, Goetter EM, Herbert JD, Forman EM, Acierno R, Ruggiero KJ. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin Psychol Psychother. 2014;21(5):427–36.23918764 10.1002/cpp.1855
16. Eisenstadt M Liverpool S Infanti E Ciuvat RM Carlsson C Mobile apps that promote emotion regulation, positive mental health, and well-being in the general population: systematic review and meta-analysis JMIR Ment Health 2021 8 11 e31170 10.2196/31170 34747713
Eisenstadt M, Liverpool S, Infanti E, Ciuvat RM, Carlsson C. Mobile apps that promote emotion regulation, positive mental health, and well-being in the general population: systematic review and meta-analysis. JMIR Ment Health. 2021;8(11):e31170.34747713 10.2196/31170
17. Linardon J Cuijpers P Carlbring P Messer M Fuller-Tyszkiewicz M The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials World Psychiatry 2019 18 3 325 336 10.1002/wps.20673 31496095
Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019;18(3):325–36.31496095 10.1002/wps.20673
18. Oliveira C Pereira A Vagos P Nóbrega C Gonçalves J Afonso B Effectiveness of mobile app-based psychological interventions for college students: a systematic review of the literature Front Psychol 2021 12 647606 10.3389/fpsyg.2021.647606 34045994
Oliveira C, Pereira A, Vagos P, Nóbrega C, Gonçalves J, Afonso B. Effectiveness of mobile app-based psychological interventions for college students: a systematic review of the literature. Front Psychol. 2021;12:647606.34045994 10.3389/fpsyg.2021.647606
19. Johnson KF Kalkbrenner MT The utilization of technological innovations to support college student mental health: mobile health communication J Technol Hum Serv 2017 35 4 314 339 10.1080/15228835.2017.1368428
Johnson KF, Kalkbrenner MT. The utilization of technological innovations to support college student mental health: mobile health communication. J Technol Hum Serv. 2017;35(4):314–39.10.1080/15228835.2017.1368428
20. Marshall JM Dunstan DA Bartik W The digital psychiatrist: in search of evidence-based apps for anxiety and depression Front Psychiatry 2019 10 831 1 7 30723425
Marshall JM, Dunstan DA, Bartik W. The digital psychiatrist: in search of evidence-based apps for anxiety and depression. Front Psychiatry. 2019;10(831):1–7.30723425
21. Wekerle C. JoyPop app. Resilience in youth [Internet]. Available from: https://youthresilience.net/joypop-app. Accessed 25 Jan 2024.
22. MacIsaac A Mushquash AR Mohammed S Grassia E Smith S Wekerle C Adverse childhood experiences and building resilience with the JoyPop app: evaluation study JMIR Mhealth Uhealth 2021 9 1 e25087 10.2196/25087 33393908
MacIsaac A, Mushquash AR, Mohammed S, Grassia E, Smith S, Wekerle C. Adverse childhood experiences and building resilience with the JoyPop app: evaluation study. JMIR Mhealth Uhealth. 2021;9(1):e25087.33393908 10.2196/25087
23. Compas BE Jaser SS Bettis AH Watson KH Gruhn MA Dunbar JP Williams E Thigpen JC Coping, emotion regulation, and psychopathology in childhood and adolescence: a meta-analysis and narrative review Psychol Bull 2017 143 9 939 10.1037/bul0000110 28616996
Compas BE, Jaser SS, Bettis AH, Watson KH, Gruhn MA, Dunbar JP, Williams E, Thigpen JC. Coping, emotion regulation, and psychopathology in childhood and adolescence: a meta-analysis and narrative review. Psychol Bull. 2017;143(9):939.28616996 10.1037/bul0000110
24. Jarrell A Lajoie SP Hall NC Horrocks PTM Antecedents and consequences of emotion regulation in STEM degree programs Innov High Educ 2022 47 493 514 10.1007/s10755-021-09587-1
Jarrell A, Lajoie SP, Hall NC, Horrocks PTM. Antecedents and consequences of emotion regulation in STEM degree programs. Innov High Educ. 2022;47:493–514.10.1007/s10755-021-09587-1
25. MacCann C Jiang Y Brown LER Double KS Bucich M Minbashian A Emotional intelligence predicts academic performance: a meta-analysis Psychol Bull 2020 146 2 150 186 10.1037/bul0000219 31829667
MacCann C, Jiang Y, Brown LER, Double KS, Bucich M, Minbashian A. Emotional intelligence predicts academic performance: a meta-analysis. Psychol Bull. 2020;146(2):150–86.31829667 10.1037/bul0000219
26. Mushquash AR Pearson ES Waddington K MacIsaac A Mohammed S Grassia E Smith S Wekerle C User perspectives on the resilience-building JoyPop app: qualitative study JMIR Mhealth Uhealth 2021 9 7 e28677 10.2196/28677 34255696
Mushquash AR, Pearson ES, Waddington K, MacIsaac A, Mohammed S, Grassia E, Smith S, Wekerle C. User perspectives on the resilience-building JoyPop app: qualitative study. JMIR Mhealth Uhealth. 2021;9(7):e28677.34255696 10.2196/28677
27. Malik I Perez A Toombs E Schmidt F Olthuis JV Charlton J Grassia E Squier C Stasiuk K Bobinski T Mushquash AR Female youth and mental health service providers’ perspectives on the JoyPopTM app: a qualitative study Front Digit Health 2023 5 1 20 10.3389/fdgth.2023.1197362
Malik I, Perez A, Toombs E, Schmidt F, Olthuis JV, Charlton J, Grassia E, Squier C, Stasiuk K, Bobinski T, Mushquash AR. Female youth and mental health service providers’ perspectives on the JoyPopTM app: a qualitative study. Front Digit Health. 2023;5:1–20.10.3389/fdgth.2023.1197362
28. Durand-Bush N, McNeill K, Harding M, Dobransky J. Investigating stress, psychological well-being, mental health functioning, and self-regulation capacity among university undergraduate students: is this population optimally functioning? Can J Couns Psychother. 2015;49(3):253–74.
29. Barnett AG Van Der Pols JC Dobson AJ Regression to the mean: what it is and how to deal with it Int J Epidemiol 2005 1 215 220
Barnett AG, Van Der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol. 2005;1:215–20.
30. Gratz KL Roemer L Multidimensional assessment of emotion regulation and dysregulation: development, factor structure, and initial validation of the difficulties in emotion regulation scale J Psychopathol Behav Assess 2004 26 1 41 54 10.1023/B:JOBA.0000007455.08539.94
Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: development, factor structure, and initial validation of the difficulties in emotion regulation scale. J Psychopathol Behav Assess. 2004;26(1):41–54.10.1023/B:JOBA.0000007455.08539.94
31. Kaufman EA Xia M Fosco G Yaptangco M Skidmore CR Crowell SE The difficulties in emotion regulation scale short form (DERS-SF): validation and replication in adolescent and adult samples J Psychopathol Behav Assess 2016 38 443 455 10.1007/s10862-015-9529-3
Kaufman EA, Xia M, Fosco G, Yaptangco M, Skidmore CR, Crowell SE. The difficulties in emotion regulation scale short form (DERS-SF): validation and replication in adolescent and adult samples. J Psychopathol Behav Assess. 2016;38:443–55.10.1007/s10862-015-9529-3
32. Lovibond PF Lovibond SH The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories Behav Res Ther 1995 33 3 335 343 10.1016/0005-7967(94)00075-U 7726811
Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995;33(3):335–43.7726811 10.1016/0005-7967(94)00075-U
33. Goldberg DP Williams P A user’s guide to the General Health Questionnaire 1988
Goldberg DP, Williams P. A user’s guide to the General Health Questionnaire. 1988.
34. Stewart-Brown S, Janmohamed K. Warwick-Edinburgh Mental Well-being Scale (WEMWBS): user guide version 1 (J. Parkinson, Ed.). Scotland: University of Warwick and the University of Edinburgh; 2008.
35. Tennant R Hiller L Fishwick R Platt S Joseph S Weich S Parkinson J Secker J Stewart-Brown S The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation Health Qual Life Outcomes 2007 5 1 63 10.1186/1477-7525-5-63 18042300
Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, Parkinson J, Secker J, Stewart-Brown S. The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):63.18042300 10.1186/1477-7525-5-63
36. Westerhof GJ Keyes CLM Mental Illness and mental health: the two continua model across the lifespan J Adult Dev 2010 17 2 110 119 10.1007/s10804-009-9082-y 20502508
Westerhof GJ, Keyes CLM. Mental Illness and mental health: the two continua model across the lifespan. J Adult Dev. 2010;17(2):110–9.20502508 10.1007/s10804-009-9082-y
37. Campbell-Sills L Stein MB Psychometric analysis and refinement of the Connor-Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience J Trauma Stress 2007 20 6 1019 1028 10.1002/jts.20271 18157881
Campbell-Sills L, Stein MB. Psychometric analysis and refinement of the Connor-Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience. J Trauma Stress. 2007;20(6):1019–28.18157881 10.1002/jts.20271
38 Masten AS Barnes AJ Resilience in children: developmental perspectives Children. 2018 5 7 98 10.3390/children5070098 30018217
Masten AS, Barnes AJ. Resilience in children: developmental perspectives. Children. 2018;5(7):98.30018217 10.3390/children5070098
39. Stoyanov SR Hides L Kavanagh DJ Wilson H Development and validation of the user version of the Mobile Application Rating Scale (uMARS) JMIR Mhealth Uhealth 2016 4 2 e5849 10.2196/mhealth.5849
Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth. 2016;4(2):e5849.10.2196/mhealth.5849
40. Stoyanov SR Hides L Kavanagh DJ Zelenko O Tjondronegoro D Mani M Mobile app rating scale: a new tool for assessing the quality of health mobile apps JMIR Mhealth Uhealth 2015 3 1 e3422 10.2196/mhealth.3422
Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015;3(1):e3422.10.2196/mhealth.3422
41. Peng W Kanthawala S Yuan S A qualitative study of user perceptions of mobile health apps BMC Public Health 2016 16 1158 10.1186/s12889-016-3808-0 27842533
Peng W, Kanthawala S, Yuan S, et al. A qualitative study of user perceptions of mobile health apps. BMC Public Health. 2016;16:1158.27842533 10.1186/s12889-016-3808-0
42. Thabrew H Boggiss AL Lim D Schache K Morunga E Cao N Cavadino A Serlachius AS Well-being app to support young people during the COVID-19 pandemic: randomised controlled trial BMJ Open 2022 12 5 e058144 10.1136/bmjopen-2021-058144 35589362
Thabrew H, Boggiss AL, Lim D, Schache K, Morunga E, Cao N, Cavadino A, Serlachius AS. Well-being app to support young people during the COVID-19 pandemic: randomised controlled trial. BMJ Open. 2022;12(5):e058144.35589362 10.1136/bmjopen-2021-058144
43. Faul F Erdfelder E Lang AG Buchner A G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences Behav Res Methods 2007 39 175 191 10.3758/BF03193146 17695343
Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.17695343 10.3758/BF03193146
44. Lui J Marcus D Barry C Evidence-based apps? A review of mental health mobile applications in a psychotherapy context Prof Psychol Res Pract 2017 48 199 210 10.1037/pro0000122
Lui J, Marcus D, Barry C. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pract. 2017;48:199–210.10.1037/pro0000122
45. Bauer M Glenn T Geddes J Gitlin M Grof P Kessing LV Whybrow PC Smartphones in mental health: a critical review of background issues, current status and future concerns Int J Bipolar Disord 2020 8 1 20 10.1186/s40345-019-0164-x 31903509
Bauer M, Glenn T, Geddes J, Gitlin M, Grof P, Kessing LV, Whybrow PC. Smartphones in mental health: a critical review of background issues, current status and future concerns. Int J Bipolar Disord. 2020;8:1–20.31903509 10.1186/s40345-019-0164-x
46. Sealed Envelope Ltd. Create a blocked randomisation list. 2022. https://www.sealedenvelope.com/simple-randomiser/v1/list
47. Survey Monkey Inc. Survey Monkey. San Mateo, California, USA. www.surveymonkey.com. Accessed 25 Jan 2024.
48. Osman A Wong JL Bagge CL Freedenthal S Gutierrez PM Lozano G The Depression Anxiety Stress Scales—21 (DASS-21): further examination of dimensions, scale reliability, and correlates J Clin Psychol 2012 68 12 1322 1338 10.1002/jclp.21908 22930477
Osman A, Wong JL, Bagge CL, Freedenthal S, Gutierrez PM, Lozano G. The Depression Anxiety Stress Scales—21 (DASS-21): further examination of dimensions, scale reliability, and correlates. J Clin Psychol. 2012;68(12):1322–38.22930477 10.1002/jclp.21908
49. Romppel M Braehler E Roth M Glaesmer H What is the General Health Questionnaire-12 assessing?: dimensionality and psychometric properties of the General Health Questionnaire-12 in a large scale German population sample Compr Psychiatry 2013 54 4 406 413 10.1016/j.comppsych.2012.10.010 23206494
Romppel M, Braehler E, Roth M, Glaesmer H. What is the General Health Questionnaire-12 assessing?: dimensionality and psychometric properties of the General Health Questionnaire-12 in a large scale German population sample. Compr Psychiatry. 2013;54(4):406–13.23206494 10.1016/j.comppsych.2012.10.010
50. Survey Monkey Inc. Security statement. 2023. https://www.surveymonkey.com/mp/legal/security/.
51. Singer JD, Willett JB. Applied longitudinal data analysis: modeling change and event occurrence. New York: Oxford University Press; 2003.
52. Dong Y Peng C-Y Principled missing data methods for researchers Springerplus 2023 2 222 10.1186/2193-1801-2-222
Dong Y, Peng C-Y. Principled missing data methods for researchers. Springerplus. 2023;2:222.10.1186/2193-1801-2-222
53. Jelicić H Phelps E Lerner RM Use of missing data methods in longitudinal studies: the persistence of bad practices in developmental psychology Dev Psychol 2009 45 4 1195 1199 10.1037/a0015665 19586189
Jelicić H, Phelps E, Lerner RM. Use of missing data methods in longitudinal studies: the persistence of bad practices in developmental psychology. Dev Psychol. 2009;45(4):1195–9.19586189 10.1037/a0015665
54. Otte A Maier-Lenz H Dierckx RA Good Clinical Practice: historical background and key aspects Nucl Med Commun 2005 26 563 574 10.1097/01.mnm.0000168408.03133.e3 15942475
Otte A, Maier-Lenz H, Dierckx RA. Good Clinical Practice: historical background and key aspects. Nucl Med Commun. 2005;26:563–74.15942475 10.1097/01.mnm.0000168408.03133.e3
