==== Front Internet IntervInternet IntervInternet Interventions2214-7829Elsevier S2214-7829(17)30012-X10.1016/j.invent.2017.10.003Full length articleThe effectiveness of unguided internet cognitive behavioural therapy for mixed anxiety and depression Morgan Carla aMason Elizabeth bNewby Jill M. bcMahoney Alison E.J. alison.mahoney@svha.org.aub⁎Hobbs Megan J. bMcAloon John aAndrews Gavin ba University of Technology, Sydney, Australiab Clinical Research Unit for Anxiety and Depression, School of Psychiatry, University of New South Wales at St. Vincent's Hospital, Sydney, Australiac School of Psychology, University of New South Wales, Sydney, Australia⁎ Corresponding author at: Clinical Research Unit for Anxiety and Depression, University of New South Wales at St Vincent's Hospital, Level 4 O'Brien Centre, 394-404 Victoria Street, Darlinghurst, NSW 2010, Australia. alison.mahoney@svha.org.au24 10 2017 12 2017 24 10 2017 10 47 53 13 2 2017 13 10 2017 18 10 2017 © 2017 The Authors2017This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).Clinician-guided internet-delivered cognitive behavioral therapy (iCBT) is an effective treatment for depression and anxiety disorders. However, few studies have examined the effectiveness of completely unguided iCBT. The current research investigated adherence to, and the effects of two brief unguided iCBT programs on depression and anxiety symptom severity, and psychological distress. Study 1 evaluated a four-lesson transdiagnostic iCBT program for anxiety and depression (N = 927). Study 2 then evaluated a three-lesson version of the same program (N = 5107) in order to determine whether reducing the duration of treatment would influence adherence and treatment effects. Cross-tabulations and independent t-tests were used to examine the extent to which users adhered and remitted with treatment. Linear mixed models were used to evaluate the effects of treatment in the entire sample, and stratified by gender and completer-type (e.g., users who completed some but not all lessons vs. those who completed all lessons of treatment). Among those who began treatment, 13.83% completed all four lessons in Study 1. Shortening the course to three lessons did not improve adherence (e.g., 13.11% in Study 2). In both studies, users, on average, experienced moderate to large effect size reductions in anxiety and depressive symptom severity, as well as psychological distress. This pattern of results was robust across gender and for those who did and did not complete treatment. Approximately two-thirds of those who completed treatment experienced remission. These data show that unguided iCBT programs, which have the capacity to attract large numbers of individuals with clinically significant symptoms of depression and anxiety, and psychological distress, can produce significant improvements in wellbeing. Highlights • This study investigated two brief unguided iCBT programs for mixed anxiety and depression • Participants who completed experienced significant reductions in anxiety and depression symptoms, as well as distress • The pre-post effect sizes were large and similar to those found in guided iCBT interventions • Unguided iCBT programs are effective for people who complete the program Keywords Internet cognitive behavioral therapy (iCBT)Self-helpUnguidedEffectivenessAdherenceDepressionAnxietyDistress ==== Body 1 Introduction Depressive and anxiety disorders are highly prevalent, annually affecting between 10 and 20% of adults (Andrews et al., 2001, Kessler et al., 2005, McEvoy et al., 2011). Remarkably, less than half of these individuals seek help from a mental health professional (Andrews et al., 2001, Harris et al., 2015). Although cognitive behavior therapy (CBT) has been shown to be an effective treatment for depression and anxiety (Butler et al., 2006), many people have difficulty accessing face-to-face CBT. This is because of the direct and indirect costs of treatment, limited access to qualified therapists, and concerns about stigma (Anderson et al., 2016, Andrade et al., 2014). These barriers have led researchers to develop Internet-based CBT (iCBT). This treatment modality can be used to provide evidence-based treatments at a low cost to end-users who can then access care in a way that is convenient and private. iCBT programs can be guided or unguided. Guided programs also referred to as “clinician-guided” or “guided self-help” involve support from a clinician or technician (e.g., the user receives emails or phone calls during the program), whereas unguided programs are delivered entirely by computer and are also referred to as “unguided self-help” or “pure self-help” (Furmark et al., 2009). Fully automated, unguided iCBT programs represent an attractive avenue for optimizing the benefits of iCBT because they are eminently scalable and can be rapidly disseminated (Leykin et al., 2014). There is considerable evidence that supports the use of guided iCBT for treating depressive and anxiety disorders (Andrews et al., 2010, Othuis et al., 2016, Păsărelu et al., 2017). Indeed, iCBT results in similar outcomes as traditional face-to-face CBT (e.g., Andersson et al., 2014, Cuijpers et al., 2010, Reger and Gahm, 2009), and has been shown to be effective under randomized controlled trial conditions as well as in “real world” clinical practice (Hobbs et al., 2018, Hobbs et al., 2017, Mewton et al., 2012, Newby et al., 2013, Newby et al., 2017, Newby et al., 2014, Williams and Andrews, 2013). Early meta-analytic studies suggest that guided iCBT programs attain better results than unguided iCBT programs (e.g., Andersson and Cuijpers, 2009, Richards and Richardson, 2012, Spek et al., 2007). However, recent efficacy studies have found that unguided iCBT programs can be just as beneficial as guided iCBT in treating depression and anxiety (e.g., Dear et al., 2014, Furmark et al., 2009, Titov et al., 2013). That said, there are few studies that have investigated the effectiveness of completely unguided iCBT. Many studies that have investigated unguided iCBT have either done so in the context of a randomized clinical trial, offered payment for participation, or included some form of human encouragement such as telephone calls, diagnostic interviews, moderated discussion forums, or personalised automated email reminders (e.g., Berger et al., 2011, Berger et al., 2017, Boettcher et al., 2012, Christensen et al., 2004a, Christensen et al., 2004b, Dear et al., 2014, Donker et al., 2013a, Donker et al., 2013b, Karyotaki et al., 2017, Lintvedt et al., 2013, Twomey et al., 2014). It is largely unclear whether the outcomes of these studies generalize to completely unguided self-help programs that are widely disseminated to the general population in uncontrolled settings. While hundreds of mental health programs are available on the Internet and via mobile phone applications, the evidence base demonstrating the effectiveness of these programs in naturalistic settings is very limited, as is the literature regarding the effect of treatment length on adherence and effects (Donker et al., 2013a, Donker et al., 2013b). The limited number of studies that have evaluated fully-automated, publically available iCBT programs in naturalistic settings do however, suggest that these programs are feasible and can lead to significant reductions in depression and anxiety symptom severity but adherence is modest (~ 16%) (e.g., Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014). Nevertheless, fully-automated iCBT programs may represent the most scalable and cost-effective way to disseminate such interventions. Given the scant data regarding the effectiveness of publically available iCBT self-help programs, the current study examined adherence to, and the effects of two freely available self-guided versions of a validated transdiagnostic program for symptoms of depression and anxiety (the Worry and Sadness program [now called the Mixed Depression and Anxiety program]) (Newby et al., 2013, Newby et al., 2014). In order to maximize the ecological validity of the study, no control group or exclusion criteria were utilized and no personal contact with users was provided. Specifically, Study 1 evaluated the effectiveness of a four lesson Worry and Sadness program and Study 2 evaluated a shorter three lesson version of the same program to investigate whether reducing the number of lessons would improve adherence. The two self-help programs were offered sequentially by the ThisWayUp.org.au service, and as such users did not nominate the length of their program. On the basis of the unguided self-help literature and previous evaluations of the guided Worry and Sadness course, we hypothesized that: (1) completion rates would be modest; (2) that the 3 three lesson program would yield greater adherence than the four lesson program; and (3) that users would experience significant effect size reductions in depression and anxiety symptom severity, and psychological distress. 2 Methods 2.1 Participants 927 users who enrolled themselves in the four-lesson Worry and Sadness iCBT program between 1st July 2012 and 8th April 2013 were included in Study 1. 5107 users who enrolled in the three-lesson Worry and Sadness iCBT program between 9th April 2013 and 21st July 2015 were included in Study 2. 2.2 Assessments The assessment measures, procedures and analytical strategies applied in Study 1 were replicated in Study 2 and are therefore presented below in one Methods section. 2.2.1 Demographic information Consistent with the anonymity of pure self-help iCBT programs, users were only required to report their gender. 2.2.2 Outcome measures Users completed the Generalized Anxiety Disorder-7 (GAD-7) and the Patient Health Questionnaire (PHQ-9) at the first and last lesson of treatment, and the Kessler Psychological Distress Scale (K-10) at each lesson. 2.2.3 Depression and anxiety symptom severity The PHQ-9 is a 9-item self-report screener for Major Depressive Disorder (MDD) (Kroenke et al., 2001) and the GAD-7 is a 7-item self-report screener for Generalized Anxiety Disorder (GAD) (Spitzer et al., 2006). On both measures, users reported how often they had experienced symptoms in the past two weeks as either “not at all”, “on several days”, “on more than half the days” or “on nearly every day”. Items were scored from 0 to 3, with a total score of 10 (or more) indicative of a probable MDD and/or GAD diagnosis respectively. The PHQ-9 has excellent reliability and validity (Kroenke et al., 2001). The internal consistency of the PHQ-9 prior to treatment was α = 0.86 in both Study 1 and Study 2. Evidence of the reliability and validity of the GAD-7 has been demonstrated (Löwe et al., 2008, Spitzer et al., 2006). The internal consistency of the GAD-7 prior to treatment was α = 0.87 in Study 1 and 0.86 in Study 2. 2.2.4 Distress The K-10 is a 10-item self-report measure that indexes non-specific psychological distress experienced in the past two weeks (Kessler et al., 2002). Users reported how frequently they experienced each item as either “none”, “a little”, “some” or “all” of the time. Each item was scored from 1 to 5 with total scores ranging from 10 to 50. The K-10 can be used as a valid predictor of common mental disorders, with scores of 20 or more indicative of clinical levels of distress (Andrews and Slade, 2001, Kessler et al., 2002). The K-10 has strong psychometric properties, including excellent internal reliability and validity (Furukawa et al., 2003, Sunderland et al., 2012). The internal consistency of the K-10 prior to treatment in Study 1 was α = 0.89 and 0.88 in Study 2. 2.3 Procedure and intervention In general, users registered with ThisWayUp (https://thiswayup.org.au) by providing an email address, specifying their gender, creating their own username and password, and confirming that they were an Australian resident. All users provided their electronic informed consent that their pooled de-identified data could be collected, collated, analyzed and published for quality assurance purposes. Users then selected the Worry and Sadness program from a list of available treatments for depressive and anxiety disorders. The two programs evaluated in this manuscript are shortened versions of the six-lesson Worry and Sadness guided iCBT program described by Newby et al. (2013). During the latter program, users follow an illustrated character that is gaining mastery over her depression and anxiety. The program includes: psychoeducation about depression and anxiety, cognitive restructuring, education about avoidance and safety behaviors, graded exposure and relapse prevention activities. This 6 lesson program has been shown to be effective using intention-to-treat analyses under randomized controlled trial conditions (N = 99; g range = 0.87–1.05; adherence = 89%)(Newby et al., 2013) and in routine clinical care (e.g., N = 175, g range = 0.98–1.09, adherence = 33%: Newby et al., 2013; and N = 707, g range = 0.51–1.20; adherence = 47.3%: Newby et al. (2014)). The content of the two unguided programs is detailed in Table 1. The three lesson program was highly similar to the four-lesson program, but included additional information about rumination and worry, and omitted information about troubleshooting difficulties with safety behaviors and graded exposure.Table 1 Lesson content of the unguided Worry and Sadness iCBT programs evaluated in Study 1 and Study 2. Table 1Lesson Number Content Homework tasks Study 1. 4 lesson program 1 Psychoeducation about anxiety and depression, controlled breathing, and physical exercise Controlled breathing, physical exercise 2 Cognitive model, common thinking distortions, thought monitoring, activity planning, and pleasant event scheduling Thought monitoring, activity planning, pleasant event scheduling 3 Thought challenging, shifting attention, structured problem solving, and hunt for positives Thought challenging, hunt for positives, structured problem solving 4 Avoidance and safety behaviors, graded exposure, common pitfalls to exposure, relapse prevention and how to get more help Facing fears with exposure hierarchies Study 2. 3 lesson program 1 Psychoeducation about anxiety and depression, controlled breathing, and physical exercise Controlled breathing, physical exercise 2 Cognitive model, common thinking distortions, rumination and worry, thought monitoring, activity planning, and pleasant event scheduling Thought monitoring, activity planning, pleasant event scheduling 3 Thought challenging, shifting attention, structured problem solving, hunt for positives, graded exposure, relapse prevention and how to get more help Thought challenging, hunt for positives, structured problem solving, facing fears with exposure hierarchies. After registering for their respective program, users were required to complete each lesson before moving onto the subsequent lesson. A minimum of five days was required to elapse before users could access subsequent lessons. At the end of each lesson, a lesson summary that described practical homework exercises was available for download and users were encouraged to complete this homework before their next lesson. A lesson was considered to be complete once users had viewed all lesson slides and downloaded the lesson summary. Automatic emails were sent to users congratulating them once a lesson was completed. Users also received an automated email if their K-10 total score was greater or equal to 30. This email advised the user that their scores were severe and recommended that they seek help from a general practitioner, psychologist, or crisis support telephone service. Users were given 60 days to complete their respective program. 2.4 Analyses 2.4.1 Adherence Users were categorized as either non-starters (e.g., those who enrolled but did not start their respective program), completers (e.g., users in Study 1 who completed 4 lessons and users in Study 2 who completed 3 lessons) or non-completers (e.g., users who completed 0–3 lessons in Study 1 or 0–2 lessons in Study 2). Regression analyses and t-tests were used to examine whether gender, pre-treatment symptom severity (GAD-7 and PHQ-9 total scores) and/or psychological distress (K-10 total scores) were associated with the extent to which users adhered to treatment. 2.4.2 Treatment effects All analyses were conducted using SPSS version 23. Linear mixed models (MIXED) with a random intercept for users were conducted to examine the effects of treatment on users' depression and anxiety symptom severity, and their psychological distress. Measurement occasion was treated as a categorical variable. These intention-to-treat models account for the unbalanced nature of these data and yield more accurate estimates of effect compared to completer analyses (Salim et al., 2008, West et al., 2014). First, models were estimated separately for each outcome measure and the covariance structure that provided the closest model fit for the residual correlation matrix was identified. Based on model fit indices, the compound symmetry covariance structure was chosen for the GAD-7 and PHQ-9, and a toeplitz covariance structure for the K-10 in Study 1. In contrast, a first order autoregressive structure provided the closest model fit for the K-10 and an unstructured covariance matrix provided the closest fit for the GAD-7 and PHQ-9 total scores in Study 2. Three types of models were then estimated. First, treatment effects were calculated for the entire sample. Second, effects were stratified by gender. Thirdly, the relationship between treatment completion and distress was examined. The K-10 was selected for the latter analysis rather than the GAD-7 or the PHQ-9 because the K-10 was measured at each lesson and the GAD-7 and PHQ-9 measured at the first and last lesson of the respective program, which precluded such analyses. Within–group Hedges' g effect sizes were calculated using the estimated marginal means at pre- and post-treatment assessments correcting for the correlation between repeated measurements. Between-group effect sizes (e.g., male vs. female; completers vs. non-completers) were estimated by calculating the difference between the estimated marginal means of each group divided by the pooled standard deviation of the two groups at their post-treatment assessments. Effect sizes of ≤ 0.49, ≥ 0.50 and ≥ 0.80 were considered to be small, moderate and large respectively (Cohen, 1988). 2.4.3 Clinically meaningful change among completers Clinically meaningful change was operationalized with respect to remission and reliable change. First, users who began treatment with a probable diagnosis of GAD and/or MDD and completed treatment with total GAD-7 and/or PHQ-9 total scores less than the standard threshold for a probable diagnosis of GAD and/or MDD (< 10) were classified as in remission. Second, reliable change indices (RCI) were calculated for the GAD-7 and PHQ-9 to determine the proportion of patients who evidenced reliable improvements with treatment (Jacobson and Truax, 1991). Test-retest reliability estimates of 0.83 for the GAD-7 (Spitzer et al., 2006) and 0.84 for the PHQ-9 (Kroenke et al., 2001), and the baseline standard deviation of each measure in Studies 1 and 2 were used in these computations (detailed further below in the respective Results section). Clinically meaningful change was limited to completers because the GAD-7 and PHQ-9 were administered before and after treatment. 3 Results 3.1 Study 1: four lesson Worry and Sadness program 3.1.1 Sample characteristics Of the 927 users who enrolled in the four-lesson Worry and Sadness program, 14.99% did not start the program (i.e., n = 139 were categorized as non-starters). Gender was the only known characteristic of this group, 63.31% of which were females (n = 88). Gender was not associated with enrolling and commencing treatment compared to enrolling and not starting treatment (χ2 (1) = 0.72, p = 0.40; OR(95% CI) = 1.18(0.81–1.71)). Non-starters were excluded from all subsequent analyses. The pre-treatment characteristics of users in Study 1 are shown in Table 2. Of the 788 who started the Program, females comprised 67.01% of the sample. In general, users enrolled in the four-lesson Worry and Sadness course with clinical levels of anxiety and depressive symptom severity, and psychological distress. Indeed, 81.85% met criteria for a probable diagnosis of GAD and/or for MDD, and 87.56% with clinically significant levels of psychological distress (i.e., K-10 total score ≥ 20).Table 2 Pre-treatment characteristics and adherence rates of users who started their 4- or 3-lesson unguided Worry and Sadness iCBT program. Table 2 Study 1: 4 lesson program (N = 788) Study 2: 3 lesson program (4233) n % n % Pre-treatment characteristics  Sex  Female 528 67.01 3087 72.93  Male 260 32.99 1146 27.07  Probable diagnoses  Subthreshold GAD and MDD  Yes 143 18.15 711 16.80  No 645 81.85 3522 83.20  GAD only  Yes 137 17.39 415 9.80  No 651 82.61 3818 90.20  MDD only  Yes 78 9.90 624 14.74  No 710 90.10 3609 85.26  Comorbid GAD/MDD  Yes 430 54.57 2483 58.66  No 358 45.43 1750 41.34  Clinically significant distress  Yes 690 87.56 3793 89.61  No 98 12.44 440 10.39 Adherence  Completed 0 lessons 114 14.47 516 12.19  Completed 1 lesson 389 49.37 2517 59.46  Completed 2 lessons 105 13.32 645 15.24  Completed 3 lessons 71 9.01 555 13.11  Completed 4 lessons 109 13.83 – – 3.1.2 Predictors of adherence Of the 788 who started the program, 14.47% did not complete any of the lessons, 49.37% completed only one lesson, 13.32% completed a total of two lessons, 9.01% completed a total of three lessons and 13.83% completed all four lessons (see Table 2). Compared to those users who did not complete the program, completers did not differ with respect to their gender (χ2 (1) = 1.20, p = 0.27; OR(95% CI) = 1.26(0.83–1.93)), anxiety symptom severity (t(df) = 1.61(786), p = 0.12) or psychological distress (t(df) = 1.95 (786), p = 0.05). Completers were however, less depressed prior to the program than those who completed 0–3 lessons (t(df) = 2.15 (786), p < 0.05). 3.1.3 Treatment effects The effects of the four-lesson Worry and Sadness program are displayed in Table 3 for the entire sample and stratified by gender. Treatment produced, on average, large effect size reductions in depressive and anxiety symptom severity, and psychological distress in the entire sample. The effect of treatment on anxiety symptom severity (GAD-7: F(1, 196) = 2.22, p = 0.14) and psychological distress did not vary by gender (K10: F(3, 464) = 1.49, p = 0.22); however, females showed a small but superior effect size reduction in depression symptom severity compared to males (PHQ-9: F(1172) = 6.27, p < 0.05). Completers and non-completers did not vary with respect to reductions in their psychological distress across treatment (F(2, 378) = 0.56, p = 0.57).Table 3 Treatment effects of the four lesson unguided Worry and Sadness iCBT program (N = 788). Table 3 Pre-treatment Post-treatment Pre-post treatment within-group ES Post between-group ES EMM SD EMM SD t p-Value r g (95% CI) g (95% CI) GAD-7  Entire sample 11.55 5.28 6.55 4.42 11.90 p < 0.001 0.54 1.10 (0.89–1.30)  Male 11.30 4.56 6.93 4.27 6.61 p < 0.001 0.55 0.98 (0.63–1.32)  Female 11.79 4.63 6.17 4.30 10.86 p < 0.001 0.54 1.27 (1.00–1.53) 0.17 (− 0.21–0.56) PHQ-9  Entire sample 13.56 6.46 7.98 5.18 11.48 p < 0.001 0.62 0.96 (0.76–1.17)  Male 13.34 5.58 8.97 5.01 5.70 p < 0.001 0.67 0.73 (0.39–1.07)  Female 13.79 5.66 6.99 5.03 11.34 p < 0.001 0.58 1.27 (1.00–1.53) 0.39 (0.00–0.78) K10  Entire sample 28.70 8.25 21.85 5.85 12.64 p < 0.001 0.67 0.99 (0.78–1.19)  Male 28.44 7.12 22.61 5.64 6.86 p < 0.001 0.82 0.65 (0.31–0.99)  Female 28.97 7.25 21.09 5.71 11.70 p < 0.001 0.59 1.29 (1.03–1.56) 0.26 (− 0.12–0.65) Note. EMM = estimated marginal means were calculated using intention-to-treat mixed models. 3.1.4 Clinically meaningful change among completers Among the 109 users who completed the program, 67.19% of those who began treatment with a probable diagnosis of GAD (e.g., n = 43/64) and 62.31% who began treatment with a probable MDD diagnosis (e.g., n = 43/69) remitted with treatment. Based on the standard deviations of the estimated marginal means (e.g., 5.28 for the GAD-7 and 6.46 for the PHQ-9), users whose pre-treatment GAD-7 total score varied by at least 6.03 points and/or pre-treatment PHQ-9 total score varied by at least 6.25 with treatment were classified as either experiencing reliable improvement or deterioration with 95% confidence (Jacobson and Truax, 1991). Based on these estimates, 28.44% experienced reliable improvements in their anxiety symptom severity (n = 31/109), 34.86% experienced reliable change in their depression symptom severity (n = 38/109), and 19.27% evidenced reliable change in both their anxiety and depression symptom severity (n = 21/109). One user experienced reliable deterioration in symptom severity (i.e., GAD-7: n = 0; PHQ-9: n = 1). No user experienced reliable deterioration with respect to both their anxiety and depression symptom severity. 3.2 Study 2: three-lesson Worry and Sadness program 3.2.1 Sample characteristics Of the 5107 users who enrolled in the three-lesson Worry and Sadness program, 17.11% did not start the Program (i.e., n = 874 were classified as non-starters). Gender was the only known characteristic of this group, 70.14% of which were females (n = 613). Gender was not associated with enrolling and commencing treatment compared to enrolling and not starting the Program (χ2 (1) = 2.79, p = 0.10; OR(95%CI) = 1.15(0.98–1.35)). Non-starters were excluded from all subsequent analyses. The pre-treatment characteristics of users in Study 2 are shown in Table 2. Of the 4233 users who started the Program, females comprised 72.93% of the sample. In general, users enrolled in the three-lesson Worry and Sadness course with clinical levels of anxiety and depressive symptom severity, and psychological distress. Indeed, 83.20% met criteria for a probable diagnosis of GAD and/or MDD, and 89.61% with clinically significant levels of psychological distress (i.e., K-10 total score ≥ 20). 3.2.2 Predictors of adherence Of those 4233 who started the program, 12.19% did not complete any of the lessons, 59.46% completed only one lesson, 15.24% completed a total of two lessons, and 13.11% completed all three lessons (see Table 2). Compared to those who completed all 3 lessons of the program, non-completers (e.g., completed 0–2 lessons) were more anxious (t(df) = 3.82(4231), p < 0.001), depressed (t(df) = 6.24(4231), p < 0.001), and distressed (t(df) = 7.00(4231), p < 0.001) prior to treatment. Completers were also more likely to be male compared to non-completers (χ2(1) = 4.85, p < 0.05;OR(95% CI) = 1.24(1.03–1.51)). 3.2.3 Treatment effects The effects of the three-lesson Worry and Sadness course are displayed in Table 4 for the entire sample and stratified by gender. Treatment was associated with, on average, large effect size reductions for all outcome measures in the entire sample. Gender differences in treatment effects were however identified, with females experiencing significantly larger reductions in anxiety symptom severity (F(1, 609) = 6.28, p < 0.05) and psychological distress (F(1, 808) = 6.44, p < 0.01) compared to males. There was also a trend towards females experiencing greater effect size reductions in depression symptom severity (F(1, 595) = 3.77, p = 0.05). Although gender was associated with some statistical differences, these effects are unlikely to be clinically meaningful, given the small magnitude of the between group differences (Table 4). Completers did not differ from non-completers with respect to their reductions in psychological distress (F(1, 3850) = 0.15, p = 0.70).Table 4 Effects of the three lesson unguided Worry and Sadness iCBT Program (N = 4233). Table 4 Pre-treatment Post-treatment Pre-post treatment within-group ES Post-treatment between-group ES EMM SD EMM SD t p-Value r g (95% CI) g (95% CI) GAD-7  Entire sample 11.96 5.47 7.55 4.45 23.19 p < 0.001 0.49 0.99 (0.90–1.08)  Male 11.55 4.84 7.62 4.12 12.44 p < 0.001 0.49 0.96 (0.80–1.13)  Female 12.37 4.83 7.49 4.11 23.14 p < 0.001 0.50 1.19 (1.08–1.30) 0.03 (− 0.14–0.21) PHQ-9  Entire sample 13.59 6.70 8.63 4.88 24.69 p < 0.001 0.62 0.91 (0.82–1.00)  Male 13.10 5.96 8.52 4.51 13.64 p < 0.001 0.67 0.85 (0.69–1.02)  Female 14.08 5.95 8.73 4.46 24.00 p < 0.001 0.60 1.10 (0.99–1.20) − 0.05 (− 0.23–0.13) K10  Entire sample 28.94 8.52 23.10 6.08 23.46 p < 0.001 0.62 0.87 (0.78–0.96)  Male 28.29 7.58 23.29 5.64 12.07 p < 0.001 0.62 0.80 (0.64–0.96)  Female 29.60 16.89 22.91 5.60 24.15 p < 0.001 0.63 1.07 (0.96–1.18) 0.07 (− 0.11–0.24) Note. EMM = estimated marginal means were calculated using intention-to-treat mixed models. 3.2.4 Clinically meaningful change among completers Among users who completed the program, 63.97% of GAD cases (n = 222/347) and 59.94% of MDD cases (n = 217/362) remitted with treatment. Based on the standard deviations of the estimated marginal means (e.g., 5.47 for the GAD-7 and 6.70 for the PHQ-9), users whose pre-treatment GAD-7 total score varied by at least 7.16 points and/or pre-treatment PHQ-9 total score varied by at least 7.43 with treatment were classified as either experiencing reliable improvement or deterioration with 95% confidence (Jacobson and Truax, 1991). As a result, 25.41% and 23.42% experienced reliable change in their anxiety (n = 141/555) and depression (n = 130/555) respectively, and 13.33% experienced reliable reductions in both their anxiety and depression symptom severity. A total of 6 users (1.08%) evidenced reliable deterioration with respect to their anxiety only (n = 1/555), depression only (n = 4/555) or both their anxiety and depression symptom severity (n = 1/555). 4 Discussion This study evaluated two fully-automated, self-guided iCBT programs for mixed anxiety and depression in a naturalistic setting. Users experienced significantly milder symptoms of depression, anxiety and psychological distress following their program. We observed moderate to large within group effect size reductions (g range = 0.65–1.29) on all outcome measures in both the four- and three-lesson programs, and when the samples were stratified by gender and completer type. Among those who completed their program, approximately two thirds of probable cases of GAD and MDD had remitted. As predicted, completion rates were modest in both studies (13.83% in Study 1 and 13.11% in Study 2). Reducing the length of the program from four to three lessons thus does not appear to have changed treatment adherence or effects. 4.1 Predictors of adherence iCBT programs tend to have lower completion rates once disseminated in naturalistic vs. controlled settings. This trend is generally exacerbated with unguided programs (Leykin et al., 2014). Consistent with this trend, the adherence rates reported herein were lower than those reported in controlled studies (e.g., Donker et al., 2013a, Donker et al., 2013b, Lintvedt et al., 2013). Yet these rates are commensurate with those found in previous unguided iCBT studies that have been conducted in naturalistic conditions (e.g., Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014). We note that there were minor differences in the predictors of adherence between Study 1 and 2. First, baseline anxiety symptom severity and distress were not associated with adherence in Study 1. In Study 2 these differences were statistically significant but may not be clinically meaningful, with completers reporting only slightly less distress, anxiety and depression than non-completers (e.g., 1 or 2 points difference on each measure). Second, although gender did not predict adherence in Study 1, males were slightly more likely to complete treatment in Study 2. The magnitude of these differences (2–3 percentage points) is unlikely to be clinically meaningful, instead these discrepancies are more likely to be associated with differential power to detect subtle variations. A consistent trend across studies was however, the relationship between distress and adherence. The rates of change in psychological distress across treatment did not vary between non-completers and completers, whom, on average, received similar benefits from each lesson. It is therefore possible that failure to complete all lessons was not due to completers receiving additional benefits from each lesson compared to non-completers. 4.2 Treatment effects The effects of the three and four-lesson self-guided Worry and Sadness courses are consistent with previous findings. We observed similar reductions in depression and anxiety symptoms severity and psychological distress in the six-lesson clinician-guided Worry and Sadness program (Newby et al., 2013, Newby et al., 2017, Newby et al., 2014). Compared to other fully unguided iCBT studies, our study found somewhat larger effect size reductions in anxiety and depression symptom severity (e.g., Boettcher et al., 2012, Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014). Importantly, baseline levels of psychopathology observed in our samples were not trivial, with approximately 90% of users reporting clinically significant symptoms of psychological distress, and 80% reporting symptoms consistent with a probable diagnosis of MDD and/or GAD. Thus, our findings, coupled with results from previous research, indicate that brief self-guided iCBT programs that are disseminated in “real world” settings have the capacity to attract large numbers of users who are experiencing clinically significant levels of anxiety and depression symptoms, and that these programs can improve users' wellbeing. With anxiety and depressive disorders affecting between 10 and 20% of adults each year (Andrews et al., 2001, Kessler et al., 2005, McEvoy et al., 2011), effective and scalable treatments are needed. Although face-to-face CBT and clinician-guided iCBT have been shown to be effective, fully automated unguided iCBT programs are eminently scalable and can be rapidly disseminated. Within a stepped-model of care, unguided iCBT may represent an initial phase of treatment for some who subsequently progress to face-to-face CBT or guided iCBT. Others may benefit from using unguided iCBT as an adjunct to other treatments. Unguided iCBT programs also present a feasible option for individuals who do not want clinical assistance and prefer an anonymous and autonomous treatment. Nevertheless, as we have shown herein for some users – two thirds had remitted and a third of whom achieved reliable improvements in their anxiety and/or depression symptom severity – unguided iCBT appears to be a sufficient stand-alone intervention. 4.3 Strengths and limitations The two studies reported herein have several strengths. Both programs were fully-structured and therefore have more fidelity than face-to-face services. The predictors of adherence and treatment effects were examined among a large number of users seeking help for their mental health problems. Moreover, both programs were completely unguided and thus true represent the effectiveness of unguided interventions once disseminated into “real world” settings. These studies however, are not without limitation. First, our use of “real world” data means that we did not employ a control group. It is therefore not possible to determine the programs' relative effectiveness against spontaneous remission and regression to the mean. Although a number of existing RCTs suggest that unguided iCBT is effective, future research may compare self-guided iCBT programs to other self-guided online treatments (such as mindfulness-based interventions) and control conditions. Second, no follow-up data were collected and as a result, we were unable to assess the long-term effects of the unguided iCBT programs. Finally, our results can only be generalized to individuals who are computer-literate and have enough insight into their symptoms to enroll in an iCBT program. Despite the limitations of these two studies, the automated nature of theses interventions renders them scalable and cost-effective, and as such, these type of programs are likely to continue to play a role in the delivery of mental health care. 5 Conclusions This evaluation of two fully automated self-guided transdiagnostic iCBT programs for depression and anxiety found that a substantial number of individuals with clinically significant symptoms are attracted to such treatments. Although adherence was relatively low, the extent to which users adhered to treatment was comparable to other unguided iCBT programs that are available. Importantly, these programs significantly reduced users' depression and anxiety symptom severity and psychological distress, and effects were robust across gender and for those users who did and did not complete treatment. Funding acknowledgment JN is supported by a NHMRC Early Career Research Fellowship (1037787). AM is supported by an Australian Government Research Training Program Scholarship. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. ==== Refs References Anderson R. Wong N. Newby J.M. Andrews G. The non-medical out of pocket costs to attend a free anxiety disorders treatment clinic in Australia Australiasian Psychiatry 24 2016 261 263 Andersson G. Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis Cogn. Behav. Ther. 38 2009 196 205 20183695 Andersson G. Cuipjers P. Carlbring P. Riper H. Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis World Psychiatry 13 2014 288 295 25273302 Andrade L. Alonso J. Mneimneh Z.N. Wells J.E. Al-Hamzawi A. Borges G. … Kessler R.C. Barriers to mental health treatment: results from the WHO mental health surveys Psychol. Med. 44 2014 1303 1317 23931656 Andrews G. Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10) Aust. N. Z. J. Psychiatry 25 2001 494 497 Andrews G. Henderson S. Hall W. Prevalence, comorbidity, disability and service utilization: overview of the Australian National Mental Health Survey Br. J. Psychiatry 178 2001 145 153 11157427 Andrews G. Cuijpers P. Craske M.G. McEvoy P.M. Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis PLoS One 5 2010 e13196 Berger T. Caspar F. Richardson R. Kneubuhler B. Sutter D. Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help Behav. Res. Ther. 49 2011 158 169 21255767 Berger T. Urech A. Krieger T. Stolz T. Schulz A. Vincent A. … Meyer B. Effects of a transdiagnostic unguided internet intervention ('velibra') for anxiety disorders in primary care: results of a randomized controlled trial Psychol. Med. 47 2017 67 80 27655039 Boettcher J. Berger T. Renneberg B. Does a pre-treatment diagnostic interview affect the outcome of internet-based self-help for social anxiety disorder? A randomized controlled trial Behav. Cogn. Psychother. 40 2012 513 528 22800984 Butler J. Chapman J.E. Forman E.M. Beck A.T. The empirical status of cognitive behavioral therapy: a review of meta-analyses Clin. Psychol. Rev. 26 2006 17 31 16199119 Christensen H. Griffiths K.M. Jorm A.F. Delivering interventions for depression by using the internet: randomised controlled trial BMJ 328 2004 265 14742346 Christensen H. Griffiths K.M. Korten A.E. Brittliffe K. Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website J. Med. Internet Res. 6 2004 e46 Christensen H. Griffiths K. Groves C. Korten A.E. Free range users and one hit wonders: community users of an internet-based cognitive behaviour therapy program Aust. N. Z. J. Psychiatry 40 2006 59 62 16403040 Cohen J. Statistical Power Analysis for the Behavioral Sciences 2nd ed. 1988 Earlbaum Hillsdale, NY Cuijpers P. Donker T. van Straten A. Li J. Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? Psychol. Med. 40 2010 1943 1957 20406528 Dear B.F. Zou J. Ali S. Lorian C.N. Johnston L. Terides M.D. Examining self-guided internet delivered cognitive behavior therapy for older adults with symptoms of anxiety and depression: two feasibility studies Internet Interventions 2 2014 17 23 Donker T. Bennett K. Bennett A. Mackinnon A. van Straten A. Cuijpers P. … Griffiths K.M. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial J. Med. Internet Res. 15 2013 e82 Donker T. Petrie K. Proudfoot J. Clarke J. Birch M.-R. Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review J. Med. Internet Res. 15 2013 e247 Farvolden P. Denisoff E. Selby P. Bagby R.M. Rudy L. Usage and longitudinal effectiveness of a web-based self-help cognitive behavioral therapy program for panic disorder J. Med. Internet Res. 7 2005 e7 Furmark T. Carlbring P. Hedman E. Sonnenstein A. Clevberger P. Bohman B. … Ekselius L. Guided and unguided self-help for social anxiety disorder: randomised controlled trial Br. J. Psychiatry 195 2009 440 447 19880935 Furukawa T.A. Kessler R.C. Andrews G. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of mental health and well-being Psychol. Med. 33 2003 357 362 12622315 Harris M.G. Hobbs M.J. Burgess P.M. Pirkis J.E. Diminic S. Siskind D.J. … Whiteford H. Frequency and quality of mental health treatment for affective and anxiety disorders among Australian adults Med. J. Aust. 202 2015 185 189 25716600 Hobbs M.J. Mahoney A.E.J. Andrews G. Integrating iCBT for generalized anxiety disorder into routine clinical practice: treatment effects across the adult lifespan J. Anxiety Disord. 51 2017 47 54 28926805 Hobbs M.J. Joubert A.J. Mahoney A.E.J. Andrews G. Treating late-life depression: comparing the effects of internet-delivered cognitive behavior therapy across the adult lifespan J. Affect. Disord. 226C 2018 58 65 Jacobson N.S. Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research J. Consult. Clin. Psychol. 59 1 1991 12 19 2002127 Karyotaki E. Riper H. Twisk J. Hoogendoorn A. Kleiboer A. Mira A. … Cuijpers P. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data JAMA Psychiatry 74 2017 351 359 28241179 Kessler R.C. Andrews G. Colpe L.J. Hiripi E. Mroczek D.K. Normand S.-L. … Zaslavsky A.M. Short screening scales to monitor population prevalences and trends in non-specific psychological distress Psychol. Med. 32 2002 959 976 12214795 Kessler R.C. Wai T.C. Demler O.V. Merikangas K.R. Walters E.E. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Survey Replication Arch. Gen. Psychiatry 62 2005 617 627 15939839 Kroenke K. Spitzer R. Williams J. The PHQ-9: validity of a brief depression severity measure J. Gen. Intern. Med. 16 2001 606 613 11556941 Leykin Y. Munoz R.F. Conteras O. Latham M.D. Results from a trial of an unsupported internet intervention for depressive symptoms Internet Interventions 1 2014 175 181 25485233 Lintvedt O.K. Griffiths K.M. Sorensen K. Ostvik A.R. Wang C.E.A. Eisemann M. Waterloo K. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial Clinical Psychology and Psychotherapy 20 2013 10 27 21887811 Löwe B. Decker O. Muller S. Brahler E. Schellberg D. Herzog W. Herzberg P.Y. Validation and standardization of the Generalized Anxiety Disorder screener (GAD-7) in the general population Med. Care 46 2008 266 274 18388841 McEvoy P.M. Grove R. Slade T. Epidemiology of anxiety disorders in the Australian general population: findings of the 2007 Australian National Survey of Mental Health and Wellbeing Aust. N. Z. J. Psychiatry 45 2011 957 967 22044173 Mewton L. Wong N. Andrews G. The effectiveness of internet cognitive behavioral therapy for generalized anxiety disorder in clinical practice Depression and Anxiety 29 2012 843 849 22949296 Newby J.M. Mackenzie A. Williams A.D. Watts S. McIntyre K. Wong N. Andrews G. Internet cognitive behavioral therapy for mixed anxiety and depression: a randomized controlled trial and evidence of effectiveness in primary care Psychol. Med. 43 2013 2635 2648 23419552 Newby J.M. Mewton L. Williams A.D. Andrews G. The effectiveness of transdiagnostic internet cognitive behavioural (iCBT) treatment for anxiety and depression in primary care J. Affect. Disord. 165 2014 45 52 24882176 Newby J.M. Mewton L. Andrews G. Transdiagnostic versus disorder-specific internet-delivered cognitive behaviour therapy for anxiety and depression in primary care J. Anxiety Disord. 46 2017 25 34 27396841 Othuis J.V. Watt M.C. Bailey K. Hayden J.A. Stewart S.H. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults Cochrane Database Syst. Rev. 3 2016 CD011565 Păsărelu C.R. Andersson G. Bergman Nordgren L. Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials Cogn. Behav. Ther. 46 2017 1 28 27712544 Reger M.A. Gahm G.A. A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety J. Clin. Psychol. 65 2009 53 75 19051274 Richards D. Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis Clin. Psychol. Rev. 32 2012 329 342 22466510 Salim A. MacKinnon A. Christensen H. Griffiths K. Comparison of data analysis strategies for intent-to-treat analysis in pre-test-post-test designs with substantial dropout rates Psychiatry Res. 160 2008 335 345 18718673 Spek V. Nyklicek I. Smits N. Cuipjers P. Riper H. Keyzer J. Pop V. Internet-based cognitive behavioral therapy for subthreshold depression in people over 50 years old: a randomized controlled trial Psychol. Med. 37 2007 1797 1806 17466110 Spitzer R.L. Kroenke K. Williams J.B.W. Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7 Arch. Intern. Med. 166 2006 1092 1097 16717171 Sunderland M. Hobbs M.J. Anderson T.M. Andrews G. Psychological distress across the lifespan: examining age-related item bias in the Kessler 6 psychological distress scale Int. Psychogeriatr. 24 2012 231 242 21933463 Titov N. Dear B.F. Johnston L. McEvoy P.M. Wootton B. Terides M.D. … Rapee R.M. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial PLoS One 8 2013 e62873 Twomey C. O'Reilly G. Byrne M. Bury M. White A. Kissane S. … Clancy N. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions Br. J. Clin. Psychol. 53 2014 433 450 24831119 West B.T. Welch K.B. Galecki A.T. Linear Mixed Models: A Practical Guide Using Statistical Software 2nd ed. 2014 Taylor and Francis Boca Raton, FL Williams A.D. Andrews G. Effectiveness of internet cognitive behavioral therapy (iCBT) for depression in primary care: a quality assurance study PLoS One 2013 art 8 10.1371/journal.pone.0057447