
==== Front
JMIR Res Protoc
JMIR Res Protoc
ResProt
JMIR Research Protocols
1929-0748
JMIR Publications Toronto, Canada

v13i1e56960
10.2196/56960
Protocol
Protocol
Evaluating the Efficacy of a Digital Therapeutic (CT-152) as an Adjunct to Antidepressant Treatment in Adults With Major Depressive Disorder: Protocol for the MIRAI Remote Study
Ma Simone
Imbert Bruce
Rothman Brian PhD https://orcid.org/0009-0006-5780-2559
1Otsuka Pharmaceutical Development & Commercialization, Inc 508 Carnegie Center Dr Princeton, NJ, 08540 United States 1 609 524 6788 brian.rothman@otsuka-us.com

Slomkowski Mary PharmD 1https://orcid.org/0000-0001-7309-0914

Speier Austin BA 2https://orcid.org/0000-0002-4260-7172

Rush A John MD 34https://orcid.org/0000-0003-2004-2382

Trivedi Madhukar H MD 56https://orcid.org/0000-0002-2983-1110

Lawson Erica BS 1https://orcid.org/0009-0005-5285-466X

Fahmy Michael MS 1https://orcid.org/0009-0006-3294-8586

Carpenter Daniel PhD 7https://orcid.org/0009-0008-1782-4432

Chen Dalei PhD 1https://orcid.org/0009-0004-4264-7248

Forbes Ainslie PhD 1https://orcid.org/0000-0003-1301-7752

1 Otsuka Pharmaceutical Development & Commercialization, Inc Princeton, NJ United States
2 Click Therapeutics, Inc New York, NY United States
3 School of Medicine Duke University Durham, NC United States
4 Duke-National University of Singapore Medical School Singapore Singapore
5 Department of Psychiatry University of Texas Southwestern Medical Center Dallas, TX United States
6 O’Donnell Brain Institute University of Texas Southwestern Medical Center Dallas, TX United States
7 Otsuka Precision Health, Inc Princeton, NJ United States
Corresponding Author: Brian Rothman brian.rothman@otsuka-us.com
2024
20 8 2024
13 e569601 2 2024
1 3 2024
31 5 2024
17 6 2024
©Brian Rothman, Mary Slomkowski, Austin Speier, A John Rush, Madhukar H Trivedi, Erica Lawson, Michael Fahmy, Daniel Carpenter, Dalei Chen, Ainslie Forbes. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 20.08.2024.
2024
https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.

Background

Major depressive disorder (MDD) is common worldwide and can be highly disabling. People with MDD face many barriers to treatment and may not experience full symptom relief even when treated. Therefore, new treatment modalities are needed for MDD. Digital therapeutics (DTx) may provide people with MDD an additional treatment option.

Objective

This study aimed to describe a phase 3 remote, multicenter, randomized, masked, sham-controlled trial evaluating the efficacy of a smartphone app–based DTx (CT-152) in adult participants diagnosed with MDD, used as an adjunct to antidepressant therapy (ADT).

Methods

Participants aged 22-64 years with a current primary diagnosis of MDD and an inadequate response to ADT were included. Participants were randomized 1:1 to CT-152 or a sham DTx. CT-152 is a smartphone app–based DTx that delivers a cognitive-emotional and behavioral therapeutic intervention. The core components of CT-152 are the Emotional Faces Memory Task exercises, brief lessons to learn and apply key therapeutic skills, and SMS text messaging to reinforce lessons and encourage engagement with the app. The sham DTx is a digital working memory exercise with emotionally neutral stimuli designed to match CT-152 for time and attention. Participants took part in the trial for up to 13 weeks. The trial included a screening period of up to 3 weeks, a treatment period of 6 weeks, and an extension period of 4 weeks to assess the durability of the effect. Sites and participants had the option of an in-person or remote screening visit; the remaining trial visits were remote. Efficacy was evaluated using the Montgomery-Åsberg Depression Rating Scale, the Generalized Anxiety Disorder-7, Clinical Global Impression–Severity scale, the Patient Health Questionnaire-9, and the World Health Organization Disability Assessment Schedule 2.0. The durability of the effect was evaluated with the Montgomery-Åsberg Depression Rating Scale and Generalized Anxiety Disorder-7 scale. Adverse events were also assessed. Satisfaction, measured by the Participant and Healthcare Professional Satisfaction Scales, and health status, measured by the EQ-5D-5L, were summarized using descriptive statistics.

Results

This study was initiated in February 2021 and had a primary completion date in October 2022.

Conclusions

This represents the methodological design for the first evaluation of CT-152 as an adjunct to ADT. This study protocol is methodologically robust and incorporates many aspects of conventional pivotal pharmaceutical phase 3 trial design, such as randomization and safety end points. Novel considerations included the use of a sham comparator, masking considerations for visible app content, and outcome measures relevant to DTx. The rigor of this methodology will provide a more comprehensive understanding of the effectiveness of CT-152.

Trial Registration

ClinicalTrials.gov NCT04770285; https://clinicaltrials.gov/study/NCT04770285

International Registered Report Identifier (IRRID)

RR1-10.2196/56960

adherence
digital placebo
sham control
cognitive-emotional training
Emotional Faces Memory Task
mobile phone
==== Body
pmcIntroduction

Depression is a disabling and common disorder that affected 279.6 million people (5% of adults) worldwide in 2019 [1-3]. Major depressive disorder (MDD) accounted for 32.8 million years lived with disability globally in 2017, a 12.6% increase from 2007, and is predicted to be the leading global cause of disability by 2030 [4,5].

Early effective treatment of MDD is associated with better outcomes, and decreasing the duration of the depressive episode can reduce symptoms, improve clinical and functional outcomes, and increase the chance of full symptom relief [6-8]. However, people with MDD face many barriers to antidepressant therapy (ADT) and psychotherapy treatment, including access to mental health care professionals (HCPs) [9,10], logistical challenges such as long wait times for appointment scheduling and limited schedules due to work and family demands [11], and the stigma associated with mental illness [12]. In 2022, only 61.5% of US adults with a major depressive episode (MDE) in the past year received treatment for depression [13]. Global treatment rates are lower, with only 34.8% of people with depression receiving treatment annually [14].

Even when people with MDD can access treatment, it is not always successful. Many people with MDD do not respond to or achieve remission with ADT alone [15]. In the foundational Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial, only 48.6% and 36.8% of participants showed treatment response and achieved remission after first-line ADT, respectively, with lower percentages responding and remitting at each subsequent treatment step [15]. Psychotherapy may also not be effective for all patients; a meta-analysis of 193 studies of psychotherapy for the treatment of depression found a response rate of 41% at a 2-month follow-up [16]. Although ADT and psychotherapy alone may not work for all patients, data support that combining the 2 approaches is more effective for treating MDD symptoms than either approach alone [17-19].

Current treatment options may be difficult to access and are not effective for all people with MDD; therefore, new approaches for treating MDD are needed. One approach that may address this unmet need is the use of digital therapeutics (DTx), a subset of digital health [20-24]. DTx are software programs that deliver evidence-based interventions for medical conditions [25]. DTx may address a medical condition, manage, prevent, or treat a medical disorder or disease, or optimize medication [25]. When used to treat mental disorders, DTx may help overcome some barriers to treatment [26]. These software programs can be accessed remotely, allowing people with MDD more options than what may be available locally, and to access treatment on their own schedule [27-29]. Remote access to treatment can also ease stigma concerns and allow for more privacy [30]. DTx may offer an additional option for nonpharmacological treatment, expanding patient choice [31]. However, there are different types of digital health products in the United States that require different levels of regulatory oversight. Products that claim a treatment benefit must be cleared by the US Food and Drug Administration (FDA) and may require a prescription [25,32,33].

Otsuka Pharmaceutical Development & Commercialization, Inc, in collaboration with Click Therapeutics, Inc, has developed a smartphone app–based DTx, CT-152, for depression, used as an adjunct to ADT with the goal of reducing symptoms of depression. The core components of CT-152 are evidence-based training exercises (Emotional Faces Memory Task [EFMT]), brief lessons and activities to learn and apply key therapeutic skills, and supportive SMS text messaging to reinforce lessons and encourage engagement with the app. Although there are no comparable FDA-authorized digital therapeutics for MDD in the United States, there are some digital tools designed to support patients with MDD in addition to standard treatment that are available under FDA enforcement discretion (software that meets the regulatory definition of a device but pose minimal risk to patients, which can be obtained without a prescription). These tools have psychotherapy-based components but do not have training exercises [34]. The EFMT component of CT-152 was developed to adjust the connectivity in the affective network and regions of the cognitive control network that provide cognitive control over emotions [35,36].

A growing body of research has linked MDD with impaired cognitive control of emotionally salient information [37,38]. Impaired cognitive control is believed to manifest in perseverative thinking patterns (eg, rumination) [39]; these patterns serve to maintain or exacerbate the sad or depressed mood component in MDD [37-39]. Cognitive-emotional training, such as EFMT, has been shown to enhance cognitive control over emotions in people with MDD [40].

The goal of the EFMT exercises is to modify connectivity between the amygdala and dorsolateral prefrontal cortex by simultaneous emotion recognition and working memory tasks [36,37]. In 2 randomized controlled trials (RCTs), an improvement in depression symptoms was found to be statistically significant for participants with MDD who completed a digital EFMT regimen (4 and 6 weeks, respectively) compared with control participants who were not being treated with ADT or cognitive behavioral therapy (CBT) [36]. A follow-up study with participants who completed the 6-week EFMT regimen indicated that EFMT use altered connectivity between the amygdala with the right dorsolateral prefrontal cortex and bilateral dorsal anterior cingulate cortex; these altered connections correlated with improved symptoms [40].

The MIRAI trial (ClinicalTrials.gov NCT04770285) was designed to measure the effectiveness and safety of CT-152 for reducing depressive symptoms in patients diagnosed with MDD who were on ADT monotherapy for the treatment of depression. Data from MIRAI supported the FDA application for this product to be cleared as a prescription DTx for the treatment of people with MDD. There is also the potential to use these data to support regulatory approvals in other global markets.

MIRAI had design components reflecting conventional phase 3 pharmaceutical trials and novel components designed for DTx evaluation specifically. Conventional components included a control group and the evaluation of both effectiveness and safety. Effectiveness end points were established from gold-standard scales that are acceptable to the FDA [41-43]. Safety was evaluated by the frequency and severity of adverse events (AEs). Novel considerations in MIRAI included a sham comparator. The sham DTx consisted of a digital working memory exercise with emotionally neutral stimuli and was matched to CT-152 for time, attention, and expectation of benefit. The sham DTx also had an SMS text message component. Another consideration was treatment masking, which is important when designing studies for DTx [44]. Both participants and sites were masked to the treatment assignment. In DTx trials, a participant can spontaneously speak to an investigator, clinical raters, or study site staff about the exercises on the app (eg, participants reporting having difficulty with recognizing emotions on faces), leading to unmasking treatment assignment and introducing bias. Thus, additional masking considerations are required for both participants and investigators; steps taken in the MIRAI trial are listed in the Methods section. Finally, MIRAI incorporated outcomes that are relevant to DTx, including participant satisfaction with the treatment and adherence metrics.

There is a growing body of literature calling for DTx to be held to the same evidentiary standards as pharmacotherapy [45-50]. It is critical that the effectiveness and safety of these novel DTx are evaluated in RCTs with adequate sample sizes and clinically meaningful end points [51-53]. This trial will extend the findings from earlier EFMT studies and provide data on the efficacy and safety of this DTx. If shown to be effective, CT-152 may be considered a novel nonpharmacological option for people diagnosed with MDD to use as an adjunct to ADT. Here, we review the rationale and study design for the MIRAI trial, including procedures to enable a methodologically rigorous investigation of the new treatment options offered with DTx.

Methods

Overview

This was a phase 3 multicenter, randomized, masked, sham-controlled trial to evaluate the efficacy of CT-152 in adult participants diagnosed with MDD who were taking ADT. Participants took part in the trial for up to 13 weeks. The trial included a screening period of up to 3 weeks, a treatment period of 6 weeks, and an extension period of 4 weeks to assess the durability of the effect (Figure 1). Given the COVID-19 pandemic and the advantages of remote trials, especially for a DTx, MIRAI was run as a remote study. Except the screening visit, all visits took place remotely; the screening visit had an in-person or remote option per investigator and patient preference. Due to the DTx nature of this trial, participants needed routine access to their smartphones in their work and home life.

Figure 1 MIRAI Trial design.

Study Participants

Key participant inclusion and exclusion criteria are listed in Textbox 1.

Key inclusion and exclusion criteria for participation.

Inclusion criteria

Aged 22 to 64 years at the time of informed consent.

Fluent in written and spoken English.

Taking antidepressant therapy (ADT) monotherapy with a current primary diagnosis of major depressive disorder (MDD) based on the criteria in the DSM-5 (Diagnostic and Statistical Manual of Mental Disorders [Fifth Edition]), single or recurrent episode, without psychotic features, and did not meet the criteria for MDD with mixed features subtype.

Had a score for the Hamilton Rating Scale for Depression, 17-item of ≥18 at screening and the baseline visit (day 1).

Had a reported history of inadequate response to an adequate trial of ADT in the current episode.

Inadequate response was defined as a reduction in depression symptom severity <50% per the Massachusetts General Hospital-Antidepressant Treatment Response Questionnaire (MGH-ATRQ).

An adequate trial was defined as at least 6 weeks at a minimum therapeutic dose (or higher), according to the MGH-ATRQ.

Were willing to maintain their current ADT at the current dose for the duration of their participation in the trial.

Were currently using a smartphone not shared with anyone else, with an Android operating system 9.0 or greater capabilities or with iPhone Operating System 13.0 (Apple, Inc) or greater capabilities and agreed to download and use the smartphone app as required by the protocol.

Had completed the onboarding module in the smartphone app during the screening period, continued consent to participate in the trial, and understood how to use the app at the baseline visit.

Exclusion criteria

Had a reported inadequate response to >1 adequate trial of ADT (as defined in the inclusion criteria) for the current major depressive episode (MDE).

Had been treated with psychopharmacological augmentation (eg, lithium, triiodothyronine, antipsychotics added to ADT, and multiple ADT) for any past episode or the current episode.

Had received psychotherapy within 90 days before screening.

Had significant suicidal ideation within the last year, suicidal behavior within the past 2 years, or presented a serious risk of suicide (in the opinion of the investigator).

Had been treated with electroconvulsive therapy or neuromodulation devices (ie, transcranial magnetic stimulation, vagus nerve stimulation, or transcranial direct current stimulation) for MDD in the past.

Were currently using a computer, web, or smartphone software–based application or equivalent for mental health or MDD; participants could enroll if this app was discontinued before the baseline visit.

Had been in a current MDE lasting longer than 2 years.

Were considered resistant or refractory to treatment by history and per investigator judgment.

Had a lifetime diagnosis of schizophrenia, schizoaffective disorder, other psychotic disorder, bipolar I or II disorder, or current posttraumatic stress disorder, panic disorder, or obsessive-compulsive disorder.

Had been diagnosed with any DSM-5 personality disorder.

Had MDD due to a general medical condition or a neurologic disorder.

Ethical Considerations

This trial was conducted in accordance with local laws, the International Conference on Harmonization Good Clinical Practice guidelines, and the Declaration of Helsinki. The protocol was reviewed and approved by the governing institutional review board (IRB) for each investigational site. Of the 54 study sites, IRB services were provided by Advarra (approval number 00000971) for 53 sites, and WCG IRB (approval number 1-1377994-1) for 1 site. IRB approvals were granted between November 2020 and January 2021. All participants were asked to provide consent to participate in this study before enrollment and agree to its privacy policy and terms of service required for the remote collection of their information. All information collected in the trial was considered confidential and was only used in accordance with the protocol. Participants were identified only by a unique identification number in eSource. However, regulatory officials or sponsor personnel could be allowed access to records, consistent with local requirements. Participants could withdraw from the trial at any time without justification. Participants received a stipend to help cover the costs associated with being in the study. Participants also had the option to have their ADT provided to them by the sponsor at no charge for the length of the study.

Intervention

CT-152 is a smartphone app–based DTx that delivers a proprietary interactive cognitive-emotional and behavioral intervention, intended for use as an adjunct to ADT. The core components of CT-152 are the EFMT exercises; brief CBT-based lessons, each paired with an out-of-app activity or guided audio psychotherapy exercise targeting the most common MDD symptoms; and SMS text messaging to reinforce the CBT-based lessons and encourage engagement with the app. Each treatment session consisted of an EFMT exercise and a brief lesson with an activity. During the EFMT exercises, participants were presented with a series of pictures of faces, tasked with identifying the emotions they observed and remembering the sequence of emotions [35]. Using an N-back working memory training paradigm, for each face observed, participants indicated whether the emotion is the same as the emotion N faces back [35]. To date, CT-152 has only been developed in English.

A sham DTx served as a control and provided a cognitive training exercise designed to retain user interest while minimizing therapeutic effects. Each sham session consisted of a Shapes Memory Task (SMT) exercise. The SMT is a working memory task that provides users with an analogous structure, matched for time and attention, to the cognitive-emotional training exercise (EFMT) in CT-152. The sham DTx did not include EFMT exercises or lessons to retain its intended nontherapeutic nature. Participants in the control group also received supportive SMS text messages related to maintaining engagement with study treatment. The SMT exercise has been previously used in EFMT pilot studies as the control condition [35,36].

In order to be eligible to participate, during the screening period, participants had to complete an N-back working memory task simulating basic features of CT-152 to ensure they had the necessary skills to use it. This also confirmed their interest in participating in the study. After eligibility was confirmed, participants were randomized 1:1 to either CT-152 or sham DTx at the baseline visit (day 1). The participants progressed through a treatment schedule of 18 sessions at a rate of 3 sessions per week over the 6-week treatment period (day 1 to week 6). Participants had a remote study visit at weeks 2, 4, and 6 and were contacted by telephone at weeks 1, 3, and 5.

After week 6, participants continued in the trial during the extension period (weeks 7 to 10) to assess the durability of the effect. In the extension period, the smartphone app remained installed for each group; no new therapeutic content was introduced, and EFMT and SMT exercises were not available. The 2 groups each received brief supportive SMS text messages in the extension period with content related to the previously completed exercises. Participants had a remote study visit at weeks 8 and 10 and were contacted by the trial site by telephone at weeks 7 and 9.

A dedicated call center was available to support the participants and the trial sites on the initial downloading of and access to the smartphone app, as well as any technical issues with the app throughout the trial.

Outcomes and Assessments

The primary objective of the trial was to evaluate the efficacy of CT-152 in improving depressive symptoms versus the sham DTx, which served as a control. The primary efficacy end point was the change from baseline to week 6 in the Montgomery-Åsberg Depression Rating Scale (MADRS) total score.

Other efficacy end points included assessment of the MADRS at weeks 2 and 4; Generalized Anxiety Disorder-7 (GAD-7) scores at weeks 2, 4, and 6; the Clinical Global Impression–Severity scale scores (CGI-S) at weeks 2, 4, and 6; and the Patient Health Questionnaire-9 (PHQ-9) scores at weeks 4 and 6. Functional outcomes were assessed by World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) scores and quality of life by EQ-5D-5L scores at week 6. To evaluate the durability of the effect of the DTx, the change from baseline in MADRS and GAD-7 total score as well as MADRS response and remission rates were evaluated at weeks 8 and 10. The MADRS was conducted by trained and calibrated independent raters. Trial site staff administered the CGI-S. Participants completed the GAD-7, PHQ-9, WHODAS 2.0, and EQ-5D-5L (for scale descriptions, refer to Table 1).

Table 1 Scales used for study assessments.

Scales	Description	
MGH-ATRQa	A 6-item, participant self-reported scale to evaluate treatment-resistant depression [54,55]	
HAM-D17b	A 17-item, observer-reported scale to evaluate depression severity and response to treatment [56]	
MADRSc	A 10-item, HCPd-reported scale to evaluate depression severity [57]	
CGI-Se	A single-item, observer-reported scale to evaluate the severity of mental illness over the past week [58]	
PHQ-9f	A 9-item, participant self-reported scale to screen for and assess depression severity [56]	
GAD-7g	A 7-item, participant self-reported scale to screen for and assess the severity of general anxiety disorder [59]	
WHODAS 2.0h	A 36-item, participant self-reported scale to assess the impact of a health condition on functioning [60]	
Participant satisfaction scale	A 9-item, participant self-reported scale designed for the MIRAI trial to assess participants’ satisfaction with their DTxi assignment for the trial	
HCP satisfaction scale	A 2-item, HCP self-reported scale designed for the MIRAI trial to assess HCP satisfaction with delivering the DTx used by their patients during the trial	
EQ-5D-5L	A 5-item, participant self-reported scale to measure overall health status. It includes a vertical visual analog scale for participants to indicate their perception of their overall health [61]	
C-SSRSj	A 6-item, participant self-reported scale to assess suicide risk, severity of risk, and support needed [62]	
aMGH-ATRQ: Massachusetts General Hospital-Antidepressant Treatment Response Questionnaire.

bHAM-D17: Hamilton Rating Scale for Depression, 17-item.

cMADRS: Montgomery-Åsberg Depression Rating Scale.

dHCP: health care professional.

eCGI-S: Clinical Global Impressions-Severity.

fPHQ-9: Patient Health Questionnaire-9.

gGAD-7: Generalized Anxiety Disorder-7.

hWHODAS: World Health Organization Disability Assessment Schedule.

iDTx: digital therapeutics.

jC-SSRS: Columbia-Suicide Severity Rating Scale.

The safety objective was to assess the safety of CT-152 in adult participants with MDD who were concurrently taking ADT. Safety was evaluated by the frequency and severity of AEs, serious AEs, and discontinuations from the trial due to AEs. AEs related to smartphone use may include dizziness, fatigue, or headache and were anticipated as potential AEs reported by the participants [63,64]. Previous research did not assess AEs related to EFMT use [35,36]. Participants were required to be on ADT monotherapy, and the safety profile of ADTs used for MDD is well understood. Based on the known risks of ADT and smartphone use and a preliminary understanding of the risks of EFMT use, no additional risks were anticipated as related to CT-152 use. All participants were assessed weekly for both anticipated and unanticipated AEs; any participants reporting serious AEs were monitored.

Participant adherence was monitored weekly. Furthermore, the number of CT-152 or sham sessions completed over the entire trial course was recorded.

Masking Considerations

In addition to masking the participants and trial sites for the treatment assignment, masked independent raters conducted the MADRS assessment by telephone. The independent raters had no access to trial procedures, treatment assignment, trial data, or clinical information other than what was solicited for the assessment. In addition, both independent raters and participants were instructed to avoid discussion of the DTx content and other aspects of the trial.

To minimize potential bias, trial sites had designated administrators of the other efficacy assessments. Administrators had limited contact with the participants and did not perform adherence checks. Participants completed self-reported outcomes independently and remotely.

The hypothesis of this trial—CT-152 would lead to greater improvement in MDD symptoms versus the sham DTx—was masked, and participants were informed that either DTx may be effective. This approach was intended to optimize adherence in both arms and limit the risk of participant unmasking and associated expectation biases. At the end of the study, participants were informed of the trial hypothesis, but were not informed of which DTx they had received.

CT-152 meets the definition of a nonsignificant risk device per 21 Code of Federal Regulations § 812.3(m) and a low- to moderate-risk device per Software as a Medical Device [65,66]. The masking approaches used in this trial were only used because (1) CT-152 posed minimal risks, (2) masking did not adversely affect the participants, and (3) the trial could only be conducted by withholding the information on participant treatment assignment.

Sample Size Calculation

The initial sample size was calculated to detect a 3-unit difference between the CT-152 plus ADT and sham DTx plus ADT in the change from baseline in MADRS total score, with 85% power at a 2-sided α=.05 level. The resulting sample size was 324 evaluable participants in total (162 in each arm). The full analysis set was defined as all randomized participants who received at least 1 CT-152 or sham DTx session and had at least 2 MADRS total score assessments, 1 of which was at baseline. To compensate for participants who failed to have evaluable assessments of MADRS total score in the full analysis set sample (estimated at up to 10% of all participants), a total of approximately 360 participants (180 in each arm) were planned to be randomized. The trial randomized eligible participants 1:1 across 54 sites. The sample size at any single trial site was capped at approximately 15% of the total number of randomized participants. Randomization was stratified by the trial site.

Due to the limitations of applying assumptions on the treatment effect size, and to ensure adequate power of the trial, an unmasked interim analysis was conducted by a data monitoring committee, whose members recommended continuing the trial with the planned sample size of 360 participants (actual enrollment included 386 participants).

Statistical Methods

Baseline and demographic characteristics, including age, sex, race, and ethnicity, were summarized using descriptive statistics.

End Points and Assessments

The primary efficacy analysis was based on the mixed-model repeated measurements method and the change from baseline score to week 6 for the MADRS. The MADRS score change from baseline to weeks 2 and 4 was also assessed. Additional efficacy analyses included changes from baseline to predetermined end points for the GAD-7, CGI-S, WHODAS 2.0, and PHQ-9. MADRS partial and full response rates (≥30% to <50% from baseline and ≥50% reduction from baseline, respectively) at weeks 2, 4, and 6 were analyzed; MADRS response rates were also analyzed during weeks 8 and 10 of the extension period. All efficacy end points were based on the full analysis set.

The durability of the treatment effect was assessed based on MADRS and GAD-7 scores during the extension period. The SAS Enterprise Guide version 8.3 (SAS Institute Inc.) was used for statistical analyses.

Plan for Missing Data in Efficacy End Points

All data collected during the trial treatment period and extension period were used for statistical analysis. The mixed-model repeated measurements method assumed data were missing at random, which is a reasonable assumption for longitudinal clinical trials in MDD [67]. However, the possibility of missing not at random data was also considered. As sensitivity analyses, pattern-mixture models were used to explore data missing mechanisms of missing not at random and investigate the response profile of participants by dropout reason.

Exploratory End Points

MADRS remission rates at weeks 2, 4, and 6 were analyzed. Remission was defined as a MADRS total score of ≤10 and a decrease of 50% or more of the total score from baseline. Satisfaction, measured by the Participant and Healthcare Professional Satisfaction Scales, and health status, measured by the EQ-5D-5L, were summarized using descriptive statistics.

Safety End Points

All AEs were coded by system organ class and Medical Dictionary for Regulatory Activities (MedDRA)-preferred term. The incidence of the following events was summarized by treatment group: (1) AEs that are potentially causally related to the treatment or sham DTx, (2) AEs with an outcome of death, serious AEs, and (3) AEs leading to discontinuation from the trial.

Adherence

Adherence was evaluated for all participants as the percentage of sessions completed over the course of the trial.

Results

The MIRAI study was funded in September 2020. It was initiated in February 2021, with a primary completion date of October 2022. As of study completion, 386 participants were enrolled. The primary data analysis is complete and expected to be published in the fall of 2024.

Discussion

Principal Findings

This was the first investigation of CT-152 as an adjunct to ADT for the treatment of depression. CT-152 has both cognitive-emotional training and behavioral intervention components, which is a novel combination compared with other digital tools designed to support patients with MDD currently available under FDA enforcement discretion [34]. This protocol incorporated aspects of conventional phase 3 pharmaceutical trials, including using a randomized controlled design and gold-standard scales for evaluating effectiveness. In addition, the safety analyses were intended to evaluate if CT-152, designed to provide a clinical intervention, had a low safety risk and was well tolerated by participants. MIRAI also considered factors that are not typical for DTx study design. The use of a sham DTx comparator minimized the potential that any changes between groups were because of CT-152 and not due to the demand characteristics of using a DTx. Bias related to knowledge of study treatment assignment was managed through robust masking procedures. Adherence was evaluated for both CT-152 and the sham DTx arms.

Safety

Although DTx for mental disorders may be less likely to lead to AEs than pharmacotherapy, AEs, such as dizziness, fatigue, and headache, that are associated with mobile device use may also be associated with DTx [63,64,68]. Therefore, it is important to assess the safety of DTx. Other current clinical trials for DTx for MDD do not assess safety outcomes [69]. Among 11 protocols for clinical trials assessing DTx for MDD published recently, only 1 mentions assessing AEs along with suicide risk [70]; 5 others assess suicide risk only [71-75]. Unlike the pivotal trials for current FDA-cleared DTx for mental disorders, the MIRAI trial assessed safety outcomes [76-79]. CT-152 was expected to be well tolerated due to the digital delivery of therapy and the mechanisms of action of its core therapeutic modules.

Sham Comparator

In trials for pharmacological treatments, masking is typically achieved through a placebo that resembles the treatment being tested. In RCTs for DTx, having a placebo that allows for masking can be challenging [50]. Of 11 clinical trial protocols published in the past few years that evaluated digital interventions for MDD, only 5 involved masking; 1 of the 5 trials masked both participants and interviewers [80]. When a participant is waitlisted or given treatment as usual during a trial of digital intervention, they may become aware of not receiving the treatment under study, which could impact their involvement and expectations [81]. In conclusion, there are masking advantages to using a sham comparator.

The variability in comparator arms in DTx trials can lead to challenges in understanding the effects of digital treatments as a class and relative to one another. In total, 3 meta-analyses of digital interventions for depression found lower effect sizes in studies of smartphone interventions that included comparators that controlled for time and attention but did not provide therapeutic intervention, compared with those studies with inactive comparators or waitlist controls [81-83]. This suggests that smartphone use alone may confer psychological benefits [46,81-84]. Using a digital comparator may raise the evidentiary standard for evaluating a digital treatment [83,84]. Given these recent findings and the potential confounding factors when using a nondigital comparator for the control group (eg, waitlist or treatment as usual), a digital comparator is important for a more unbiased and accurate understanding of the effectiveness of a DTx [81-83].

The MIRAI trial design is innovative because it included a sham DTx as the comparator, which helps to ensure that the comparison reflected a true change in symptoms (if any), rather than a nonspecific benefit related to digital interventions more broadly [46,81,82]. The sham DTx more easily allowed for masking, with less risk of participants and investigators becoming unmasked.

Outcomes Important for DTx

An important consideration for any treatment is understanding the relationship between improvement in patient symptoms and increased patient adherence to the treatment; this is a novel consideration for digital interventions [85,86]. For MDD specifically, among 11 study protocols involving DTx for MDD, 9 protocols described adherence assessments [80]. While data on adherence were collected, none of the protocols defined adherence or how it would be evaluated.

A systematic literature review on adherence and engagement with digital interventions for MDD defined adherence as the percentage of sessions completed (the number of sessions completed divided by the total number of sessions) and found that only 38 out of 94 studies reviewed had this information available [80]. Their findings highlight the importance of establishing a standardized metric for adherence to digital interventions to better compare results across studies and different types of interventions [80]. The MIRAI study was designed to collect data that can be used to measure the percentage of sessions completed, which will contribute to the body of literature measuring adherence to digital interventions in a way that can be compared across studies.

Another novel consideration for DTx that does not often apply to pharmacological treatments is the consideration of patient satisfaction with and the ability to easily use the intervention. This can include features such as an easy onboarding practice, personalizing app content, and text reminders for patients [87]. CT-152 includes text messages to remind participants of what they learned and to engage them with treatment. In addition, MIRAI had both a participant and an HCP satisfaction survey to gain further insights into how easy CT-152 was to use and how satisfied participants were with their experience.

Limitations

While this study design addressed many common concerns about trials for digital interventions, it had limitations, such as not including adults younger than 22 years or older than 65 years. Therefore, potential benefits for these populations will need to be explored in future research [88]. The inclusion criterion for MDD severity was a Hamilton Rating Scale for Depression, 17-item score of ≥18, potentially excluding participants with milder MDD. A further limitation is that the MIRAI trial assessed CT-152 as an adjunct to ADT, and thus the results cannot be generalized to using the digital intervention without ADT.

There were also potential limitations related to remote trials and digital interventions, such as limited access to technology or low digital literacy. In this study, participants were required to own a smartphone and had to complete an onboarding module to show they had an interest in study participation and the skills necessary to use CT-152 or the sham DTx. These requirements limited the potential for low digital access or literacy to impact participation.

In addition, a common occurrence in trials for digital interventions or that use remote data collection is participant attrition [89,90]. Although most clinical trials have some participants discontinue or be unresponsive to follow-up, this is more likely to happen in studies involving DTx [89,90]. Participant attrition was evaluated after the trial but was expected to be similar to what is seen in the literature. Participant engagement and adherence to digital interventions were also concerns [80]. Both CT-152 and the sham DTx had text messages designed to encourage participant adherence and engagement throughout the study. To encourage overall continued adherence to the study program, participants were informed before enrolling that remote visits would be required, and they reviewed the visit schedule with trial site staff once randomized to a treatment arm. Adherence checks were also performed during the treatment period and adherence was monitored remotely through the DTx. Any signs of nonadherence were addressed by the investigator to remind and encourage participants to continue adhering to the study program.

Conclusions

DTx has the potential to address many barriers to MDD treatment, including increasing access to more treatment options, allowing for remote access, easily fitting into a patient’s schedule, and easing stigma concerns [26-28,30]. However, it is important that the effectiveness and safety of DTx are evaluated in RCTs with rigorous study design [51-53].

Many considerations in conventional phase 3 trial design, such as safety end points, adherence, and the use of a sham comparator, have been incorporated into this protocol to assess the efficacy of CT-152 for MDD, although the way these considerations are assessed is often different for pharmacological treatment. Incorporating safety and adherence considerations, along with participant masking to the treatment, are not yet typical for DTx evaluation. Moreover, rigorous assessments of these outcomes will provide a more comprehensive understanding of the effectiveness of CT-152.

This study was funded and supported by Otsuka Pharmaceutical Development & Commercialization, Inc. Click Therapeutics was a codevelopment collaborator. Medical writing support for this manuscript was provided by Caroline Leitschuh, PhD, and Madeline Keleher, PhD, of Oxford Pharma Genesis Inc, Newtown, Pennsylvania, which was funded by Otsuka Pharmaceutical Development & Commercialization, Inc, Princeton, New Jersey.

Abbreviations

ADT antidepressant therapy

AE adverse event

CBT cognitive behavioral therapy

CGI-S Clinical Global Impression–Severity scale

DTx digital therapeutics

EFMT Emotional Faces Memory Task

FDA US Food and Drug Administration

GAD-7 General Anxiety Disorder-7 scale

HCP health care professional

IRB institutional review board

MADRS Montgomery-Åsberg Depression Rating Scale

MDD major depressive disorder

MDE major depressive episode

MedDRA Medical Dictionary for Regulatory Activities

PHQ-9 Patient Health Questionnaire-9

RCT randomized controlled trial

SMT Shapes Memory Task

STAR*D Sequenced Treatment Alternatives to Relieve Depression

WHODAS 2.0 World Health Organization Disability Assessment Schedule 2.0

Authors' Contributions: BR, MS, AS, EL, MF, DC, D Chen, and AF made a significant contribution to the conception, study design, execution, acquisition of data, analysis, and interpretation of the work reported. AJR and MHT made a significant contribution to the conception, study design, and interpretation of the work reported. BR, MS, AS, AJR, MHT, EL, MF, DC, D Chen, and AF took part in drafting, revising, and critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agreed to be accountable for all aspects of the work.

Conflicts of Interest: BR, MS, EL, MF, D Chen, and AF are employees of Otsuka Pharmaceutical Development & Commercialization, Inc, and DC is an employee of Otsuka Precision Health, Inc. AS is an employee of Click Therapeutics, Inc. MHT is a consultant for Otsuka Pharmaceutical Development & Commercialization, Inc. AJR has received consulting fees from Compass Inc., Curbstone Consultant LLC, Emmes Corp., Evecxia Therapeutics, Inc, Holmusk Technologies, Inc, ICON, PLC, Johnson & Johnson (Janssen), Liva-Nova, MindStreet, Inc, Neurocrine Biosciences Inc, Otsuka-US; speaking fees from Liva-Nova, Johnson & Johnson (Janssen); and royalties from Wolters Kluwer Health, Guilford Press, and the University of Texas Southwestern Medical Center, Dallas, TX.
==== Refs
1 GBD 2019 Mental Disorders Collaborators Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019 Lancet Psychiatry 2022 02 9 2 137 150 10.1016/S2215-0366(21)00395-3 35026139 S2215-0366(21)00395-3 35026139
2 Depression World Health Organization 2021 2021-12-11 https://www.who.int/news-room/fact-sheets/detail/depression
3 Greenberg PE Fournier AA Sisitsky T Simes M Berman R Koenigsberg SH Kessler RC The economic burden of adults with major depressive disorder in the United States (2010 and 2018) Pharmacoeconomics 2021 06 39 6 653 665 10.1007/s40273-021-01019-4 33950419 10.1007/s40273-021-01019-4 33950419
4 GBD 2017 Disease and Injury Incidence and Prevalence Collaborators Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease study 2017 Lancet 2018 392 10159 1789 1858 10.1016/S0140-6736(18)32279-7 30496104 S0140-6736(18)32279-7 30496104
5 Liu Q He H Yang J Feng X Zhao F Lyu J Changes in the global burden of depression from 1990 to 2017: Findings from the Global Burden of Disease study J Psychiatr Res 2020 07 126 134 140 10.1016/j.jpsychires.2019.08.002 31439359 S0022-3956(19)30738-1 31439359
6 Borentain S Nash AI Dayal R DiBernardo A Patient-reported outcomes in major depressive disorder with suicidal ideation: a real-world data analysis using PatientsLikeMe platform BMC Psychiatry 2020 07 23 20 1 384 10.1186/s12888-020-02758-y 32703173 10.1186/s12888-020-02758-y 32703173
7 Dong M Zeng LN Lu L Li XH Ungvari GS Ng CH Chow IHI Zhang L Zhou Y Xiang YT Prevalence of suicide attempt in individuals with major depressive disorder: a meta-analysis of observational surveys Psychol Med 2019 07 49 10 1691 1704 10.1017/S0033291718002301 30178722 S0033291718002301 30178722
8 Habert J Katzman MA Oluboka OJ McIntyre RS McIntosh D MacQueen GM Khullar A Milev RV Kjernisted KD Chokka PR Kennedy SH Functional recovery in major depressive disorder: focus on early optimized treatment Prim Care Companion CNS Disord 2016 09 01 18 5 10.4088/PCC.15r01926 27835721
9 Mongelli F Georgakopoulos P Pato MT Challenges and opportunities to meet the mental health needs of underserved and disenfranchised populations in the United States Focus (Am Psychiatr Publ) 2020 01 18 1 16 24 10.1176/appi.focus.20190028 32047393 FOC_20190028 32047393
10 Andrilla CHA Patterson DG Garberson LA Coulthard C Larson EH Geographic variation in the supply of selected behavioral health providers Am J Prev Med 2018 06 54 6 Suppl 3 S199 S207 10.1016/j.amepre.2018.01.004 29779543 S0749-3797(18)30005-9 29779543
11 Ofonedu ME Belcher HME Budhathoki C Gross DA Understanding Barriers to Initial Treatment Engagement among Underserved Families Seeking Mental Health Services J Child Fam Stud 2017 03 26 3 863 876 10.1007/s10826-016-0603-6 28584498 28584498
12 Arnaez JM Krendl AC McCormick BP Chen Z Chomistek AK The association of depression stigma with barriers to seeking mental health care: a cross-sectional analysis J Ment Health 2020 04 29 2 182 190 10.1080/09638237.2019.1644494 31373519 31373519
13 Substance Abuse and Mental Health Services Administration Key substance use and mental health indicators in the United States: results from the 2022 National Survey on Drug Use and Health (HHS Publication No. PEP23-07-01-006, NSDUH Series H-58) 2024-07-11 https://www.samhsa.gov/data/sites/default/files/reports/rpt42731/2022-nsduh-nnr.pdf
14 Mekonen T Chan GCK Connor JP Hides L Leung J Estimating the global treatment rates for depression: a systematic review and meta-analysis J Affect Disord 2021 295 1234 1242 10.1016/j.jad.2021.09.038 34665135 S0165-0327(21)00997-6 34665135
15 Rush AJ Trivedi MH Wisniewski SR Nierenberg AA Stewart JW Warden D Niederehe G Thase ME Lavori PW Lebowitz BD McGrath PJ Rosenbaum JF Sackeim HA Kupfer DJ Luther J Fava M Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report Am J Psychiatry 2006 163 11 1905 1917 10.1176/ajp.2006.163.11.1905 17074942 163/11/1905 17074942
16 Cuijpers P Karyotaki E Ciharova M Miguel C Noma H Furukawa TA The effects of psychotherapies for depression on response, remission, reliable change, and deterioration: a meta-analysis Acta Psychiatr Scand 2021 144 3 288 299 10.1111/acps.13335 34107050 34107050
17 Cuijpers P Berking M Andersson G Quigley L Kleiboer A Dobson KS A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments Can J Psychiatry 2013 58 7 376 385 10.1177/070674371305800702 23870719 23870719
18 Karyotaki E Smit Y Holdt Henningsen K Huibers MJH Robays J de Beurs D Cuijpers P Combining pharmacotherapy and psychotherapy or monotherapy for major depression? A meta-analysis on the long-term effects J Affect Disord 2016 194 144 152 10.1016/j.jad.2016.01.036 26826534 S0165-0327(15)31038-7 26826534
19 Kamenov K Twomey C Cabello M Prina AM Ayuso-Mateos JL The efficacy of psychotherapy, pharmacotherapy and their combination on functioning and quality of life in depression: a meta-analysis Psychol Med 2017 47 3 414 425 10.1017/S0033291716002774 27780478 S0033291716002774 27780478
20 Williams LM Getting personalized: brain scan biomarkers for guiding depression interventions Am J Psychiatry 2017 174 6 503 505 10.1176/appi.ajp.2017.17030314 28565957 28565957
21 Hallgren KA Bauer AM Atkins DC Digital technology and clinical decision making in depression treatment: current findings and future opportunities Depress Anxiety 2017 34 6 494 501 10.1002/da.22640 28453916 28453916
22 Lake J Turner MS Urgent need for improved mental health care and a more collaborative model of care Perm J 2017 21 17 24 10.7812/TPP/17-024 28898197 17-024
23 Ormel J Cuijpers P Jorm A Schoevers RA What is needed to eradicate the depression epidemic, and why Mental Health & Prevention 2020 17 1 11 10.1016/j.mhp.2019.200177
24 Ceskova E Silhan P Novel treatment options in depression and psychosis Neuropsychiatr Dis Treat 2018 14 741 747 10.2147/NDT.S157475 29559781 ndt-14-741 29559781
25 Digital therapeutics: combining technology and evidence-based medicine to transform personalized patient care Digital Therapeutics Alliance 2018 2022-08-04 https://www.dtxalliance.org/wp-content/uploads/2018/09/DTA-Report_DTx-Industry-Foundations.pdf
26 Lord SE Campbell ANC Brunette MF Cubillos L Bartels SM Torrey WC Olson AL Chapman SH Batsis JA Polsky D Nunes EV Seavey KM Marsch LA Workshop on implementation science and digital therapeutics for behavioral health JMIR Ment Health 2021 8 1 e17662 10.2196/17662 33507151 v8i1e17662 33507151
27 Dang A Arora D Rane P Role of digital therapeutics and the changing future of health care J Family Med Prim Care 2020 9 5 2207 2213 10.4103/jfmpc.jfmpc_105_20 32754475 JFMPC-9-2207 32754475
28 Muñoz RF Pineda BS Barrera AZ Bunge E Leykin Y Digital tools for prevention and treatment of depression: lessons from the Institute for International Internet Interventions for Health Clínica y Salud 2021 32 1 37 40 10.5093/clysa2021a2
29 Renfrew ME Morton DP Morton JK Hinze JS Przybylko G Craig BA The influence of three modes of human support on attrition and adherence to a web- and mobile app-based mental health promotion intervention in a nonclinical cohort: randomized comparative study J Med Internet Res 2020 22 9 e19945 10.2196/19945 32990633 v22i9e19945 32990633
30 Hassanpour S Tomita N DeLise T Crosier B Marsch LA Identifying substance use risk based on deep neural networks and instagram social media data Neuropsychopharmacology 2019 44 3 487 494 10.1038/s41386-018-0247-x 30356094 10.1038/s41386-018-0247-x 30356094
31 Carl JR Jones DJ Lindhiem OJ Doss BD Weingardt KR Timmons AC Comer JS Regulating digital therapeutics for mental health: opportunities, challenges, and the essential role of psychologists Br J Clin Psychol 2022 61 Suppl 1 130 135 10.1111/bjc.12286 33650131 33650131
32 General wellness: policy for low risk devices guidance for industry and Food and Drug Administration staff United States Food and Drug Administration 2019 2022-08-10 https://www.fda.gov/regulatory-information/search-fda-guidance-documents/general-wellness-policy-low-risk-devices
33 Policy for device software functions and mobile medical applications: guidance for industry and Food and Drug Administration staff United States Food and Drug Administration 2019 2022-08-10 https://www.fda.gov/regulatory-information/search-fda-guidance-documents/policy-device-software-functions-and-mobile-medical-applications
34 McIntyre RS Greenleaf W Bulaj G Taylor ST Mitsi G Saliu D Czysz A Silvesti G Garcia M Jain R Digital health technologies and major depressive disorder CNS Spectr 2023 28 6 662 673 10.1017/S1092852923002225 37042341 S1092852923002225 37042341
35 Iacoviello BM Wu G Alvarez E Huryk K Collins KA Murrough JW Iosifescu DV Charney DS Cognitive-emotional training as an intervention for major depressive disorder Depress Anxiety 2014 31 8 699 706 10.1002/da.22266 24753225 24753225
36 Iacoviello BM Murrough JW Hoch MM Huryk KM Collins KA Cutter GR Iosifescu DV Charney DS A randomized, controlled pilot trial of the emotional faces memory task: a digital therapeutic for depression NPJ Digit Med 2018 1 21 10.1038/s41746-018-0025-5 10.1038/s41746-018-0025-5 30854473 21 30854473
37 Iacoviello BM Charney DS Developing cognitive-emotional training exercises as interventions for mood and anxiety disorders Eur Psychiatry 2015 30 1 75 81 10.1016/j.eurpsy.2014.09.415 25451246 S0924-9338(14)00612-9 25451246
38 Murrough JW Iacoviello B Neumeister A Charney DS Iosifescu DV Cognitive dysfunction in depression: neurocircuitry and new therapeutic strategies Neurobiol Learn Mem 2011 96 4 553 563 10.1016/j.nlm.2011.06.006 21704176 S1074-7427(11)00117-1 21704176
39 Watkins ER Constructive and unconstructive repetitive thought Psychol Bull 2008 134 2 163 206 10.1037/0033-2909.134.2.163 18298268 2008-01984-001 18298268
40 Hoch MM Doucet GE Moser DA Hee Lee W Collins KA Huryk KM DeWilde KE Fleysher L Iosifescu DV Murrough JW Charney DS Frangou S Iacoviello BM Initial evidence for brain plasticity following a digital therapeutic intervention for depression Chronic Stress (Thousand Oaks) 2019 3 2470547019877880 10.1177/2470547019877880 32440602 10.1177_2470547019877880 32440602
41 Patient-focused drug development: incorporating clinical outcome assessments into endpoints for regulatory decision-making guidance for industry, Food and Drug Administration staff, and other stakeholders (draft guidance) United States Food and Drug Administration 2023 2023-06-09 https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-incorporating-clinical-outcome-assessments-endpoints-regulatory
42 Löwe B Kroenke K Herzog W Gräfe K Measuring depression outcome with a brief self-report instrument: sensitivity to change of the patient health questionnaire (PHQ-9) J Affect Disord 2004 81 1 61 66 10.1016/S0165-0327(03)00198-8 15183601 S0165032703001988 15183601
43 Hudgens S Floden L Blackowicz M Jamieson C Popova V Fedgchin M Drevets WC Cooper K Lane R Singh J Meaningful change in depression symptoms assessed with the Patient Health Questionnaire (PHQ-9) and Montgomery-Asberg Depression Rating Scale (MADRS) among patients with treatment resistant depression in two, randomized, double-blind, active-controlled trials of esketamine nasal spray combined with a new oral antidepressant J Affect Disord 2021 281 767 775 10.1016/j.jad.2020.11.066 33261932 S0165-0327(20)32996-7 33261932
44 Lutz J Offidani E Taraboanta L Lakhan SE Campellone TR Appropriate controls for digital therapeutic clinical trials: a narrative review of control conditions in clinical trials of digital therapeutics (DTx) deploying psychosocial, cognitive, or behavioral content Front Digit Health 2022 4 823977 10.3389/fdgth.2022.823977 36060538 36060538
45 Espie CA Firth J Torous J Evidence-informed is not enough: digital therapeutics also need to be evidence-based World Psychiatry 2022 21 2 320 321 10.1002/wps.20993 35524617 35524617
46 Ghaemi SN Sverdlov O van Dam J Campellone T Gerwien R A smartphone-based intervention as an adjunct to standard-of-care treatment for schizophrenia: randomized controlled trial JMIR Form Res 2022 6 3 e29154 10.2196/29154 35343910 v6i3e29154 35343910
47 Insel TR The NIMH experimental medicine initiative World Psychiatry 2015 14 2 151 153 10.1002/wps.20227 26043323 26043323
48 Bernstein EE Weingarden H Wolfe EC Hall MD Snorrason I Wilhelm S Human support in app-based cognitive behavioral therapies for emotional disorders: scoping review J Med Internet Res 2022 24 4 e33307 10.2196/33307 35394434 v24i4e33307 35394434
49 Torous J Bucci S Bell IH Kessing LV Faurholt-Jepsen M Whelan P Carvalho AF Keshavan M Linardon J Firth J The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality World Psychiatry 2021 20 3 318 335 10.1002/wps.20883 34505369 34505369
50 Dang A Dang D Rane P The expanding role of digital therapeutics in the post-Covid-19 era The Open COVID Journal 2021 1 1 32 37 10.2174/2666958702101010032
51 Developing novel endpoints generated by digital health technology for use in clinical trials Clinical Trial Transformation Initiative 2022 2023-01-05 https://ctti-clinicaltrials.org/wp-content/uploads/2022/03/CTTI-Digital-Health-Trials-Novel-Endpoint-Acceptance-Recommendations.pdf
52 Murray E Hekler EB Andersson G Collins LM Doherty A Hollis C Rivera DE West R Wyatt JC Evaluating digital health interventions: key questions and approaches Am J Prev Med 2016 51 5 843 851 10.1016/j.amepre.2016.06.008 27745684 S0749-3797(16)30229-X 27745684
53 Weir K The ascent of digital therapies 2018 2022-09-13 https://www.apa.org/monitor/2018/11/cover-digital-therapies
54 Chandler GM Iosifescu DV Pollack MH Targum SD Fava M RESEARCH: Validation of the Massachusetts General Hospital Antidepressant Treatment History Questionnaire (ATRQ) CNS Neurosci Ther 2010 16 5 322 325 10.1111/j.1755-5949.2009.00102.x 19769599 CNS102 19769599
55 Desseilles M Witte J Chang TE Iovieno N Dording CM Ashih H Nyer M Freeman MP Fava M Mischoulon D Assessing the adequacy of past antidepressant trials: a clinician's guide to the antidepressant treatment response questionnaire J Clin Psychiatry 2011 72 8 1152 1154 10.4088/JCP.11ac07225 21899818 21899818
56 Ma S Yang J Yang B Kang L Wang P Zhang N Wang W Zong X Wang Y Bai H Guo Q Yao L Fang L Liu Z The patient health questionnaire-9 vs. the hamilton rating scale for depression in assessing major depressive disorder Front Psychiatry 2021 12 747139 10.3389/fpsyt.2021.747139 34803766
57 Quilty LC Robinson JJ Rolland JP Fruyt FD Rouillon F Bagby RM The structure of the montgomery-asberg depression rating scale over the course of treatment for depression Int J Methods Psychiatr Res 2013 22 3 175 184 10.1002/mpr.1388 24038301 24038301
58 Busner J Targum SD The clinical global impressions scale: applying a research tool in clinical practice Psychiatry (Edgmont) 2007 4 7 28 37 20526405
59 Spitzer RL Kroenke K Williams JBW Löwe Bernd A brief measure for assessing generalized anxiety disorder: the GAD-7 Arch Intern Med 2006 166 10 1092 1097 10.1001/archinte.166.10.1092 16717171 166/10/1092 16717171
60 Measuring health and disability: manual for WHO disability assessment schedule (?WHODAS 2.0) World Health Organization 2010 2023-03-21 https://www.who.int/publications/i/item/measuring-health-and-disability-manual-for-who-disability-assessment-schedule-(-whodas-2.0)
61 EQ-5D-5L user guide (version 3.0) EuroQol Research Foundation 2019 2023-03-21 https://euroqol.org/wp-content/uploads/2023/03/EQ-5D-5LUserguide-08-0421.pdf
62 The columbia protocol (C0SSRS): about the protocol The Columbia Lighthouse Project 2016 2023-03-21 https://cssrs.columbia.edu/the-columbia-scale-c-ssrs/about-the-scale/
63 Chung JY Digital therapeutics and clinical pharmacology Transl Clin Pharmacol 2019 27 1 6 11 10.12793/tcp.2019.27.1.6 32055575 32055575
64 Kollins SH DeLoss DJ Cañadas E Lutz J Findling RL Keefe RSE Epstein JN Cutler AJ Faraone SV A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial Lancet Digit Health 2020 2 4 e168 e178 10.1016/S2589-7500(20)30017-0 33334505 S2589-7500(20)30017-0 33334505
65 Guidance for the content of premarket submissions for software contained in medical devices United States Food and Drug Administration 2005 2023-04-17 https://www.fda.gov/regulatory-information/search-fda-guidance-documents/content-premarket-submissions-device-software-functions
66 21 Code of Federal Regulations part 812 United States Food and Drug Administration 2023 2023-04-17 https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H
67 Siddiqui O Hung HMJ O'Neill R MMRM vs. LOCF: a comprehensive comparison based on simulation study and 25 NDA datasets J Biopharm Stat 2009 19 2 227 246 10.1080/10543400802609797 19212876 908658805 19212876
68 Crisafulli S Santoro E Recchia G Trifirò G Digital therapeutics in perspective: from regulatory challenges to post-marketing surveillance Front Drug Saf Regul 2022 2 10.3389/fdsfr.2022.900946
69 Software treatments for actively reducing severity of cognitive deficits in MDD (STARS-MDD) ClinicalTrials.gov 2018 2022-09-14 https://clinicaltrials.gov/ct2/show/NCT03310281
70 Antle BF Owen JJ Eells TD Wells MJ Harris LM Cappiccie A Wright B Williams SM Wright JH Dissemination of computer-assisted cognitive-behavior therapy for depression in primary care Contemp Clin Trials 2019 78 46 52 10.1016/j.cct.2018.11.001 30572162 S1551-7144(18)30138-1 30572162
71 Fitzsimmons-Craft EE Taylor CB Newman MG Zainal NH Rojas-Ashe EE Lipson SK Firebaugh ML Ceglarek P Topooco N Jacobson NC Graham AK Kim HM Eisenberg D Wilfley DE Harnessing mobile technology to reduce mental health disorders in college populations: a randomized controlled trial study protocol Contemp Clin Trials 2021 103 106320 10.1016/j.cct.2021.106320 33582295 S1551-7144(21)00056-2 33582295
72 Williams C McClay CA Martinez R Morrison J Haig C Jones R Farrand P Online CBT life skills programme for low mood and anxiety: study protocol for a pilot randomized controlled trial Trials 2016 17 1 220 10.1186/s13063-016-1336-y 27121090 10.1186/s13063-016-1336-y 27121090
73 Weisel KK Zarski AC Berger T Schaub MP Krieger T Moser CT Berking M Ebert DD Transdiagnostic tailored internet- and mobile-based guided treatment for major depressive disorder and comorbid anxiety: study protocol of a randomized controlled trial Front Psychiatry 2018 9 274 10.3389/fpsyt.2018.00274 30022954 30022954
74 Krämer R Köhler S Evaluation of the online-based self-help programme "selfapy" in patients with unipolar depression: study protocol for a randomized, blinded parallel group dismantling study Trials 2021 22 1 264 10.1186/s13063-021-05218-4 33836810 10.1186/s13063-021-05218-4 33836810
75 Richards D Duffy D Blackburn B Earley C Enrique A Palacios J Franklin M Clarke G Sollesse S Connell S Timulak L Digital IAPT: the effectiveness & cost-effectiveness of internet-delivered interventions for depression and anxiety disorders in the Improving Access to Psychological Therapies programme: study protocol for a randomised control trial BMC Psychiatry 2018 18 1 59 10.1186/s12888-018-1639-5 29499675 10.1186/s12888-018-1639-5 29499675
76 Campbell AN Nunes EV Matthews AG Stitzer M Miele GM Polsky D Turrigiano E Walters S McClure EA Kyle TL Wahle A Van Veldhuisen P Goldman B Babcock D Stabile PQ Winhusen T Ghitza UE Internet-delivered treatment for substance abuse: a multisite randomized controlled trial Am J Psychiatry 2014 171 6 683 690 10.1176/appi.ajp.2014.13081055 24700332 1859474 24700332
77 Christensen DR Landes RD Jackson L Marsch LA Mancino MJ Chopra MP Bickel WK Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence J Consult Clin Psychol 2014 82 6 964 972 10.1037/a0037496 25090043 2014-31346-001 25090043
78 Investigation of the freespira breathing system in the treatment of post traumatic stress disorder (PTSD) ClinicalTrials.gov 2020 2022-08-16 https://clinicaltrials.gov/ct2/show/NCT03039231
79 Christensen H Batterham PJ Gosling JA Ritterband LM Griffiths KM Thorndike FP Glozier N O'Dea B Hickie IB Mackinnon AJ Effectiveness of an online insomnia program (SHUTi) for prevention of depressive episodes (the GoodNight Study): a randomised controlled trial Lancet Psychiatry 2016 3 4 333 341 10.1016/S2215-0366(15)00536-2 26827250 S2215-0366(15)00536-2 26827250
80 Forbes A Keleher MR Venditto M DiBiasi F Assessing patient adherence to and engagement with digital interventions for depression in clinical trials: systematic literature review J Med Internet Res 2023 25 e43727 10.2196/43727 37566447 v25i1e43727 37566447
81 Firth J Torous J Nicholas J Carney R Rosenbaum S Sarris J Can smartphone mental health interventions reduce symptoms of anxiety? a meta-analysis of randomized controlled trials J Affect Disord 2017 218 15 22 10.1016/j.jad.2017.04.046 28456072 S0165-0327(17)30015-0 28456072
82 Firth J Torous J Nicholas J Carney R Pratap A Rosenbaum S Sarris J The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials World Psychiatry 2017 16 3 287 298 10.1002/wps.20472 28941113 28941113
83 Goldberg SB Lam SU Simonsson O Torous J Sun S Mobile phone-based interventions for mental health: a systematic meta-review of 14 meta-analyses of randomized controlled trials PLOS Digit Health 2022 1 1 e0000002 10.1371/journal.pdig.0000002 35224559 35224559
84 Linardon J Cuijpers P Carlbring P Messer M Fuller-Tyszkiewicz M The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials World Psychiatry 2019 18 3 325 336 10.1002/wps.20673 31496095 31496095
85 Moritz S Schilling L Hauschildt M Schröder J Treszl A A randomized controlled trial of internet-based therapy in depression Behav Res Ther 2012 50 7-8 513 521 10.1016/j.brat.2012.04.006 22677231 S0005-7967(12)00075-7 22677231
86 MacLean S Corsi DJ Litchfield S Kucharski J Genise K Selaman Z Testa V Hatcher S Coach-facilitated web-based therapy compared with information about web-based resources in patients referred to secondary mental health care for depression: randomized controlled trial J Med Internet Res 2020 22 6 e15001 10.2196/15001 v22i6e15001 32515740
87 Dahlhausen F Zinner M Bieske L Ehlers JP Boehme P Fehring L There's an app for that, but nobody's using it: insights on improving patient access and adherence to digital therapeutics in Germany Digit Health 2022 8 20552076221104672 10.1177/20552076221104672 35811758 10.1177_20552076221104672 35811758
88 Walker DM Hefner JL Fareed N Huerta TR McAlearney AS Exploring the digital divide: age and race disparities in use of an inpatient portal Telemed J E Health 2020 26 5 603 613 10.1089/tmj.2019.0065 31313977 31313977
89 Meyerowitz-Katz G Ravi S Arnolda L Feng X Maberly G Astell-Burt T Rates of attrition and dropout in app-based interventions for chronic disease: systematic review and meta-analysis J Med Internet Res 2020 22 9 e20283 10.2196/20283 32990635 v22i9e20283 32990635
90 Pratap A Neto EC Snyder P Stepnowsky C Elhadad N Grant D Mohebbi MH Mooney S Suver C Wilbanks J Mangravite L Heagerty PJ Areán P Omberg L Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants NPJ Digit Med 2020 3 21 10.1038/s41746-020-0224-8 10.1038/s41746-020-0224-8 32128451 224 32128451
