
==== Front
NPJ Digit Med
NPJ Digit Med
NPJ Digital Medicine
2398-6352
Nature Publishing Group UK London

1241
10.1038/s41746-024-01241-2
Article
A randomized trial testing digital medicine support models for mild-to-moderate alcohol use disorder
Quanbeck Andrew andrew.quanbeck@fammed.wisc.edu

1
Chih Ming-Yuan 2
Park Linda 1
http://orcid.org/0000-0002-7221-3174
Li Xiang 1
http://orcid.org/0000-0003-3968-9629
Xie Qiang 1
Pulvermacher Alice 1
Voelker Samantha 1
Lundwall Rachel 1
Eby Katherine 3
Barrett Bruce 1
Brown Randall 1
1 grid.14003.36 0000 0001 2167 3675 University of Wisconsin, Department of Family Medicine and Community Health, Madison, WI USA
2 https://ror.org/02k3smh20 grid.266539.d 0000 0004 1936 8438 University of Kentucky, Department of Health & Clinical Sciences, Lexington, KY USA
3 https://ror.org/02mqqhj42 grid.412647.2 0000 0000 9209 0955 University of Wisconsin Hospital & Clinics, Madison, WI USA
14 9 2024
14 9 2024
2024
7 2481 3 2024
29 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
This paper reports the results of a hybrid effectiveness-implementation randomized trial that systematically varied levels of human oversight required to support the implementation of a digital medicine intervention for persons with mild-to-moderate alcohol use disorder (AUD). Participants were randomly assigned to three groups representing possible digital health support models within a health system: self-monitored use (SM; n = 185), peer-supported use (PS; n = 186), or a clinically integrated model CI; (n = 187). Across all three groups, the percentage of self-reported heavy drinking days dropped from 38.4% at baseline (95% CI [35.8%, 41%]) to 22.5% (19.5%, 25.5%) at 12 months. The clinically integrated group showed significant improvements in mental health and quality of life compared to the self-monitoring group (p = 0.011). However, higher attrition rates in the clinically integrated group warrant consideration in interpreting this result. Results suggest that making a self-guided digital intervention available to patients may be a viable option for health systems looking to promote alcohol risk reduction. This study was prospectively registered at clinicaltrials.gov on 7/03/2019 (NCT04011644).

Subject terms

Health care
Biomedical engineering
https://doi.org/10.13039/100000027 U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA) R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 R01-AA024150 Quanbeck Andrew Chih Ming-Yuan Park Linda Li Xiang Xie Qiang Pulvermacher Alice Voelker Samantha Lundwall Rachel Eby Katherine Barrett Bruce U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)U.S. Department of Health & Human Services | NIH | National Institute on Alcohol Abuse and Alcoholism (NIAAA)issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Excessive alcohol use is associated with an increased risk of injury, liver disease, heart disease, and cancer, resulting in roughly 140,000 U.S. deaths and 3.6 million years of potential life lost annually1,2. Approximately 17% of U.S. adults binge drink (drinking 5 or more drinks on an occasion for men or 4 or more drinks for women)1,2. There has been an acceleration of alcohol-induced deaths across the U.S. population in the first two decades of the 21st century3, with notable increases in mortality associated with alcohol-associated liver disease in women4.

Alcohol use disorder (AUD) can be conceptualized along a spectrum of risk, with appropriate interventions dependent on an individual’s risk of progressing to severe AUD over the lifetime5. Improving access to effective interventions across a broad base of the population is warranted for a disorder as pervasive and damaging as AUD1,4. Patient self-management can be fundamental to addressing AUD because it allows individuals with heavy use to initiate their own positive behavior change, and technology can play an important role. Riper et al.6 completed a systematic review highlighting the potential of internet-based interventions in offering scalable, accessible, and effective support for individuals with AUD in both community and healthcare settings6. Digital medicine apps hold promise for the treatment and continuing care of patients with AUD because such systems can provide a widely accessible vehicle for both self-management and clinical monitoring7. However, there has been little research on the implementation of digital medicine technologies for AUD to reach populations that could benefit from early intervention5. A key, unanswered research question relates to the role of human support (e.g., medical providers, health coaches) in enhancing the effectiveness of digital medicine apps8–10.

This study tested several pragmatic digital medicine support models that systematically varied the level of human support used to facilitate the implementation of an evidence-based digital medicine intervention for AUD. The primary aim was to detect the effectiveness of each support model on self-reported heavy drinking days (HDD: 5 or more standard drinks on any day for men under 65, 4 or more standard drinks for women and men over 65) and quality of life (QOL). We conducted a 12-month randomized controlled trial using a hybrid type 1 effectiveness-implementation study design. Type I hybrids are intended to test the effects of an intervention while gathering information on implementation11. Our rationale for this design is that a low-touch model is the least costly and easiest to implement. However, if the active involvement of either peer support specialists or health coaches increases the intervention’s effectiveness, the additional cost may represent a worthwhile investment. Our hypotheses were that, for patients with mild-to-moderate AUD, (1) group-oriented peer mentorship would be more effective than self-monitoring; (2) 1:1 health coaching would be more effective than self-monitoring; and (3) 1:1 health coaching would be more effective than group peer mentorship

Results

Study flow and survey completion

Among 2317 individuals who completed the eligibility screening, 1254 (55%) met the screening criteria and were invited to join the onboarding process; 558 (44%) completed the onboarding process and were randomized (Fig. 1). Two main reasons for those excluded during onboarding (n = 696) were not creating a Tula account (n = 414 or 59%) and not scheduling a randomization call (n = 156 or 22%). The completion rate of the follow-up surveys was overall 78% (1756/2232), ranging from 93% at the end of month 3 to 67% at month 12. The mean number of surveys completed per person across groups was 3.14 out of 4 follow-up surveys, with SD of 1.31. The clinically integrated (CI) group participants (M = 2.75, SE = 0.11) completed fewer follow-up surveys than the self-monitored (SM) or peer-supported (PS) group participants (Wald Chi-square (2) = 14.179, p < 0.001).Fig. 1 Study flow diagram.

The study flow diagram shows the number of participants from the initial screening to randomization through completion of follow-up surveys. Participants who contacted the research team to dropout of the study: SM = 0, PS = 1, CI = 9.

Characteristics of the study population

Table 1 shows the baseline characteristics of the study population. The majority (91%) of the study population was white, slightly higher than the proportion of white patients in the overall patient population in the health system (82%). The proportion of black participants enrolled in the study was roughly the same as their representation in the health system population (4.8%) and the Madison, WI metropolitan area. Females were overrepresented, comprising almost 2/3 of the study population. The population was highly educated, reflecting that Madison, WI, is the 5th most educated city in the US. Except for education level, all other demographics and outcomes at baseline were similar between groups. The CI group had a slightly higher percentage with a high school degree/GED as their highest education level but a slightly lower percentage of those who completed a vocational or associate degree. The PS group had a slightly lower percentage with master’s degrees.Table 1 Baseline characteristics of the study population

Characteristic	Self-monitored (n = 185)	Peer-supported (n = 186)	Clinically integrated (n = 187)	Total (n = 558)	
Age (M, SD)	(42.8, 12.9)	(42.8, 12.9)	(42.6, 12.9)	(42.8, 12.9)	
Female (N, %)	(121, 65.4%)	(122, 65.6%)	(123, 65.8%)	(366, 65.6%)	
Race (N, %)*					
 White	(170, 91.9%)	(169, 90.9%)	(171, 91.4%)	(510, 91.4%)	
 Black	(7, 3.8%)	(10, 5.4%)	(10, 5.3%)	(27, 4.8%)	
 AI/NA	(2, 1.1%)	(2, 1.1%)	(1, 0.5%)	(5, 0.9%)	
 Asian	(5, 2.7%)	(3, 1.6%)	(4, 2.1%)	(12, 2.2%)	
 NH/PI	(0, 0%)	(1, 0.5%)	(1, 0.5%)	(2, 0.4%)	
 Other	(1, 0.5%)	(4, 2.2%)	(1, 0.5%)	(6, 1.1%)	
 Hispanic or Latino (N, %)	(4, 2.2%)	(6, 3.2%)	(4, 2.1%)	(14, 2.5%)	
Education (N, %)					
 <High school	(0, 0%)	(2, 1.1%)	(1, 0,5%)	(3, 0.5%)	
 HS or GED	(16, 8.6%)	(19, 10.2%)	(32, 17.1%)	(67, 12%)	
 Vocation or associate	(35, 18.9%)	(34, 18.3%)	(16, 8.6%)	(85, 15.2%)	
 Bachelors	(76, 41.1%)	(83, 44.6%)	(76, 40.6%)	(235, 42.1%)	
 Masters	(44, 23.8%)	(29, 15.6%)	(48, 25.7%)	(121, 21.7%)	
 Doctorate	(14, 7.6%)	(19, 10.2%)	(14, 7.5%)	(47, 8.4%)	
Marital status (N, %)					
 Married	(104, 56.2%)	(95, 51.1%)	(102, 54.5%)	(301, 53.9%)	
 Widowed	(3, 1.6%)	(3, 1.6%)	(4, 2.1%)	(10, 1.8%)	
 Divorced	(16, 8.6%)	(22, 11.8%)	(17, 9.1%)	(55, 9.9%)	
 Separated	(2, 1.1%)	(3, 1.6%)	(1, 0.5%)	(6, 1.1%)	
 Never married	(35, 18.9%)	(42, 22.6%)	(32, 17.1%)	(109, 19.5%)	
 Living with a partner	(24, 13%)	(19, 10.2%)	(30, 16%)	(73, 13.1%)	
 Refused	(0, 0%)	(0, 0%)	(1, 0.5%)	(1, 0.2%)	
 Don’t know	(1, 0.5%)	(2, 1.1%)	(0, 0%)	(3, 0.5%)	
 Severity-mild (N, %)	(118, 63.8%)	(120, 64.5%)	(118, 63.1%)	(356, 63.8%)	
 AUDIT (M, SD)	(12.2, 5.4)	(12, 6.1)	(12.5, 5.7)	(12.2, 5.7)	
AUDIT category (N, %)					
 Low risk	(39, 21.1%)	(53, 28.5%)	(41, 21.9%)	(133, 23.8%)	
 Harmful drinking	(85, 45.9%)	(72, 38.7%)	(84, 44.9%)	(241, 43.2%)	
 Alcohol dependence	(61, 33%)	(61, 32.8%)	(62, 33.2%)	(184, 33%)	
AI/NA American Indian/Native American, NH/PI Native Hawaiian/Pacific Islander, HS high school, GED general educational development, M mean, SD standard deviation, N number, % Percentage, QOL quality of life.

*The numbers in race added within groups were over 100% because some people reported multiple races.

Intervention utilization

Figure 2 depicts the percentage of participants who logged into Tula at least once during each study week. Use declined over time, most notably among the CI participants. Participants clicked an average of 806 links/pages in the Tula app during the 12-month study period with a wide range from 6 to 4844 pages. An average of 11.2 out of 12 weekly check-in surveys were submitted during the 3-month intervention period (with incentives), and 28.3 out of 40 weekly surveys were submitted during the rest of the 9-month follow-up period (without incentives). The PS and CI group participants could read and post messages on Tula discussion forums. Most PS and CI participants (96%, 178/186 and 97%, 182/187, respectively) read Tula discussion forum messages, and 96 (52%) PS and 52 (28%) CI participants posted one or more messages. Health coaching visits were utilized by 125 (67%) CI participants, with a mean number of 1.7 visits and a SD of 1.3.Fig. 2 Percentage of weekly Tula users over time.

Illustrates the percentage of Tula users each week during the 52-week (or 12-month) study period. The numerators are the number of participants who used the Tula app in a particular study week while the denominators are the number of participants actively enrolled in the study during that week.

Descriptive analysis–primary outcomes

Mean values of primary outcomes from baseline to month 12 for all three groups are listed in Table 2 and visualized in Fig. 3.Table 2 Outcomes of the study population

Characteristic	Self-monitored	Peer-supported	Clinically integrated	Total	
	n	(M, SD)	n	(M, SD)	n	(M, SD)	n	(M, SD)	
Percentage of heavy drinking days, PHDD	
 Baseline	185	(37.1, 30.6)	186	(36.2, 31.2)	186	(41.9, 31.4)	557	(38.4, 31.1)	
 3 Months	178	(25.5, 28.2)	178	(23.8, 30.1)	162	(29.4, 28.2)	518	(26.1, 28.9)	
 6 Months	161	(23.6, 27.3)	162	(24.3, 30.1)	139	(25.3, 31.2)	462	(24.4, 29.4)	
 9 Months	145	(20.3, 24.6)	145	(21.2, 27.7)	109	(22.4, 27.4)	399	(21.2, 26.5)	
 12 Months	139	(24.4, 28.2)	130	(21.4, 28.9)	100	(21.3, 30.4)	369	(22.5, 29)	
 ARR or RRR*	−12.7 or −34.2%	−14.8 or −40.9%	−20.6 or −49.2%	−15.9 or 41.4%	
Quality of life-physical health, QOL-PH*	
 Baseline	185	(48.4, 6.3)	186	(48.1, 6.7)	186	(48.3, 6.5)	557	(48.3, 6.5)	
 3 months	178	(48.6, 6.6)	177	(48.4, 6.6)	162	(47.8, 6.7)	517	(48.3, 6.6)	
 6 months	161	(48.6, 6.4)	162	(48.7, 7.2)	139	(48.1, 7.2)	462	(48.5, 6.9)	
 9 months	145	(48.5, 6.3)	145	(49.6, 6.1)	109	(48.1, 7.4)	399	(48.8, 6.6)	
 12 months	139	(48.6, 6.8)	130	(49.3, 6.4)	100	(48.1, 7.5)	369	(48.6, 6.9)	
 ARR or RRR*	−0.2 or −0.4%	1.2 or 2.5%	−0.2 or −0.4%	0.3 or 0.6%	
Quality of life-mental health, QOL-MH*	
 Baseline	185	(45.5, 6.9)	186	(45,5, 7.5)	186	(45.4, 6.9)	557	(45.4, 7.1)	
 3 Months	178	(46.3, 7)	177	(45,9, 7.3)	162	(47.3, 7.1)	517	(46.5, 7.1)	
 6 Months	161	(46.4, 7.1)	162	(47.4, 7.5)	139	(48, 7.5)	462	(47.2, 7.4)	
 9 Months	145	(46.4, 7.2)	145	(47.3, 7.2)	109	(48.1, 7.2)	399	(47.2, 7.2)	
 12 Months	139	(46.7, 8.3)	130	(47.7, 8.3)	100	(48.1, 7)	369	(47.4, 8)	
 ARR or RRR**	1.2 or 2.6%	2.2 or 4.8%	2.7 or 5.9%	1.8 or 4%	
M mean, SD standard deviation, n sample size, ARR absolute risk reduction, RRR relative risk reduction.

*QOL measures are T-scores.

**ARR and RRR were calculated between values in baseline and 12 months.

Fig. 3 Primary outcomes over time.

Presents the group means of the primary outcomes over time (percentage of heavy drinking days, quality of life-physical health, and quality of life-mental health). The primary outcomes were collected in quarterly surveys at 0, 3, 6, 9, and 12 months after randomization.

Absolute and relative risk reductions between outcomes at baseline to 12 months are listed in Table 2. An overall decline in percentage of heavy drinking days (PHDD) over time—from an average 38.4 PHDD (95% CI [35.8, 41]) at baseline to 22.5 PHDD (19.5, 25.5) at month 12, about a reduction of 15.9 PHDD or a 41% reduction in PHDD—was observed across all the study groups. However, rates of PHDD in the SM group bounced back slightly at month 12 while the PS group remained the same and the CI group continued to decrease. The physical health scores showed little change except for a slight improvement for the PS group from month 6 (48.7) to month 9 (49.6). In the mental health domain, all three groups showed slight improvements in mental health, with more noticeable improvement for the CI (5.9%) and PS groups (4.8%) from baseline to month 12 compared to the SM group (2.6%).

Primary statistical analysis

Percentage of heavy drinking days (PHDD)

The results of the General Linear Mixed Model (GLMM) analysis are listed in Table 3. There was no group effect on PHDD (F(2,1729) = 0.375, p = 0.688). However, a significant overall time effect (F(2,1729) = 6.142, p < 0.001) and a trending group by time effect (F(6, 1729) = 1.936, p = 0.072) were found. The overall PHDD declined significantly from month 3 to months 9 and 12 (p < 0.001 and p = 0.005, respectively). The pair-wise comparisons for the group by time effect showed that PHDD had a significant reduction from month 3 to months 9 and 12 among the CI group participants (F(3,1729) = 6.773, p < 0.001) but not in the other two groups (SM: F(3,1729) = 2.124, p = 0.095; PS: F(3,1729) = 0.474, P = 0.7).Table 3 Treatment and time contrast on primary outcomes

Outcome	Contrast*	Estimate (95% CI)	SE	df	P	
PHDD**	PS-SM	−0.205 (−0.784, 0.374)	0.242	1729	0.783	
	CI-SM	−0.145 (−0.75, 0.46)	0.253	1729	0.918	
	CI-PS	0.059 (−0.546, 0.665)	0.253	1729	0.994	
	12M-3M	−0.484 (−0.867, −0.101)	0.145	1729	0.005	
	9M-3M	−0.557 (−0.928 −0.185)	0.141	1729	<0.001	
	CI: 12M-3M	−1.149 (−1.868, −0.43)	0.273	1729	<0.001	
	CI: 9M-3M	−0.884 (−1.578, −0.189)	0.264	1729	0.005	
QoL-PH	PS-SM	0.339 (−0.592, 1.269)	0.389	1728	0.767	
	CI-SM	−0.326 (−1.297, 0.644)	0.406	1728	0.806	
	CI-PS	−0.665 (−1.636, 0.306)	0.406	1728	0.275	
QoL-MH	PS-SM	0.519 (−0.701, 1.739)	0.510	1728	0.670	
	CI-SM	1.540 (0.268, 2.811)	0.532	1728	0.011	
	CI-PS	1.020 (−0.252, 2.293)	0.532	1728	0.157	
	12M–3M	0.894 (0.078, 1.710)	0.310	1728	0.023	
	6M–3M	0.814 (0.165, 1.463)	0.246	1728	0.006	
	6 M: CI-SM	1.587 (0.002, 3.172)	0.663	1728	0.050	
	9 M: CI-SM	1.972 (0.277, 3.667)	0.709	1728	0.016	
	PS: 12M–3M	1.569 (0.203, 2.934)	0.518	1728	0.015	
	PS: 6M–3M	1.309 (0.214, 2.405)	0.416	1728	0.010	
M month, PHDD percentages of heavy drinking days, QoL-PH quality of life physical health, QoL-MH quality of life-mental health, SM self-monitoring, PS peer-supported, CI clinically integrated.

*Treatment contrasts controlled for alcohol drinking severity at screening, biological sex, and outcomes at baseline. Contrasts for the main group comparisons were included. For the time and the group-by-time effects, only significant pair-wise comparisons lower than 0.05 were listed.

**PHDD is a square-root transformed variable.

Quality of life-physical health (QOL-PH)

No group effect (F(2,1728) = 1.345, p = 0.261) was found in the mixed model analysis for QOL-PH. The overall time effect was significant (F(3, 1728) = 2.856, p = 0.036), but the group by time effect was not (F(6,1728) = 0.695, p = 0.654). The pair-wise comparisons between the time points and the group-by-time interactions were not statistically significant.

Quality of life-mental health (QoL-MH)

An overall group effect (F(2,1728) = 4.277, p = 0.014) and time effect (F(3,1728) = 4.597, p = 0.003) were found for QoL-MH. The group-by-time effect was not significant (F(6,1728) = 0.984, p = 0.434). The pair-wise contrasts showed that the CI participants overall reported a better level of mental health (p = 0.011) than the SM group. The significant group difference was also found at 9 months (F(2,1728) = 3.989, p = 0.019) with a trending significance at 6 months (F(2,1728) = 2.862, p = 0.057) and 12 months (F(2,1728) = 2.644, p = 0.071). The overall QoL-MH across the three groups was found to improve significantly from 3 to 6 months and 3 to 12 months (p = 0.006 and 0.023, respectively). The PS participants reported significant improvement in QOL-MH (F(3,1728) = 4.681, p = 0.003), specifically from 3 months to 6 and 12 months (p = 0.01 and 0.015, respectively).

Discussion

This hybrid implementation-effectiveness trial systematically varied the degree of human touch offered to support the implementation of an evidence-based digital health app for alcohol use reduction in participants with mild-to-moderate AUD. All three randomized groups had reductions in PHDD—from 38.4% (95% CI [35.8%, 41%]) at baseline to 22.5% (19.5%, 25.5%) at 12 months, representing a statistically significant time effect. The CI group showed a significant reduction in alcohol use from 3 to 9 and 12 months (p < 0.001 and p = 0.005, respectively) while the other two groups did not. There were no statistically significant differences in PHDD reduction among the SM, PS, and CI groups. The clinical and societal significance of a 16-percentage point reduction (41% relative reduction) in PHDD is difficult to gauge. For context, the CDC estimates that excessive alcohol use is responsible for more than 140,000 annual deaths and an economic cost of $249 billion1. AUD is a chronic, relapsing, and remitting condition that can worsen over the lifespan, and arresting problematic use across a broad base of the population before alcohol-related complications manifest could have significant population health benefits.

Concerning the primary outcome of QOL, the null hypothesis was not rejected, with the SM group showing similar effectiveness as the PS and CI groups. The CI group showed significant improvements in mental health QOL scores compared to the SM group (p = 0.011). However, higher rates of attrition in the CI group warrant caution in interpreting the results (see Supplementary Information). It appears that younger persons dropped out at a higher rate than older participants, and a pattern mixture model conducted via sensitivity analysis (as opposed to the base case assumption of data missing at random) negated the positive effect on mental health QOL in the CI group (i.e., improved mental health observed among CI participants may be due to different drop-out patterns vs. the intervention itself).

There has been limited published research on digital medicine interventions targeted at this population. Crane et al.12 examined the Drink Less app in the U.K., which enrolled 672 participants12. In that study, only 179 (27%) participants completed the primary outcome measure, and no main effects on alcohol use were found. The literature suggests that systematic tracking of alcohol use and mindful awareness of drinking patterns can serve as effective interventions per se6,13,14, and this type of functionality was supported by all three groups, including the self-monitoring group.

With 558 participants, this study is the largest controlled trial featuring a version of A-CHESS. An implementation study of an A-CHESS variant in 3 federally qualified health centers showed positive effects on alcohol and substance use outcomes but was not sustained after the trial ended15. A 2 × 2 factorial trial featuring a head-to-head comparison of A-CHESS (a digital health intervention) vs. Telephone Monitoring and Counseling (a telehealth intervention) found that both interventions had positive and statistically equivalent effects on HDD, although there was no additional benefit found by combining the 2 interventions16. Kiluk et al.17 compared usual care to clinician-delivered CBT and to an internet-delivered program called CBT4CBT. After 12 weeks, all three arms showed reductions in substance use, with higher reductions in the clinician-delivered CBT arm and the CBT4CBT arm when compared to the usual care. However, the clinician-delivered CBT arm had a higher drop-out rate than the other two arms17. Tetrault et al.18 compared CBT4CBT to in-person treatment and found positive effects for both groups but no significant difference between them18.

It is essential to highlight differences in the current study population compared to prior studies of A-CHESS16,19. In contrast to prior studies that enrolled participants in recovery for severe AUD, our study participants had mild-to-moderate AUD. Connection with peers has been integral to prior versions of A-CHESS, and this dynamic was modeled in the PS group via peer mentorship and the discussion forum. However, peer mentoring had minimal additional effect compared to self-monitored use in this study, perhaps because isolation is not as prevalent as it often becomes for persons with more severe AUD.

With 558 randomized participants and an overall response rate of 78%, this study was among the most expansive and rigorously conducted digital health trials for alcohol use reduction. Follow-up rates were high compared to norms in digital health research, where a 2022 systematic review found median completion rates of only 48% across 37 studies (a 78% response rate would correspond to the first quartile)20. Follow-up rates did vary by group, however. The research team took note of higher drop-out rates in the CI group early in the trial and made repeated efforts to contact participants to increase follow-up, with limited success. We don’t know why attrition was higher in the CI group. While representing only a subset of participants lost to follow-up (Fig. 1), 10/558 participants asked the research team to be withdrawn from the study, and 9/10 were in the CI group. The study team is also conducting qualitative interviews to examine participants’ experiences with the study interventions. Early indications are that attrition may have resulted from mismatches between the intervention and participants’ preferences surrounding issues such as privacy and intervention intensity. This preliminary finding is consistent with evidence that some individuals with behavioral health issues are more comfortable sharing personal information with a computerized intervention vs. a human interventionist21. Our findings are also consistent with findings by Kiluk et al.17, showing unexpectedly high attrition in human-delivered vs. automated cognitive behavioral therapy17.

This study focused on implementation of digital health in a primary healthcare system and was intentionally pragmatic vs. explanatory, as outlined by PRECIS-2 criteria22. Pragmatic study design decisions were made in partnership with health system leadership, including self-reporting of alcohol use without biomarker verification, unblinded assignment for implementers and participants, and implementation by healthcare and community-based practitioners vs. research staff. Importantly, the study did not include a pure control group. Doing so would require a “sham device,” which was deemed impractical given the aims of the study: no plausible placebo digital medicine support model for AUD could be envisioned that would be acceptable to the health system. Participation in research can have effects, regardless of the specific interventions tested, and other factors may explain the alcohol use reductions observed (e.g., temporal trends, responding to monetary incentives).

Prior research suggests that people who use complementary and alternative medicine interventions tend to be female, of middle age, and have more education23. Our sample reflects this. Our use of convenience sampling may limit generalizability to other populations and highlights the need to actively engage communities to keep health equity concerns at the forefront of digital health implementation research24.

Paying mindful attention to alcohol use may be the first step in contemplating and addressing problematic habits and patterns of use that may lead to future problems. While adding peer support or 1:1 health coaching did not clearly yield more effectiveness in alcohol use reduction, incorporating patient preferences for using digital health apps may be worth considering when choosing digital medicine support models, particularly in light of the mental health improvements observed in the CI group. Additionally, the more intensive nature of the CI intervention may have represented a lack of fit for persons with mild/moderate AUD. Choice in selecting an intervention has been shown to improve outcomes in other areas of research, such as heart disease25. Thus, one possible implication for health systems would be to consider offering a self-guided version of the app with an option for supplemental health coaching for individuals who desire a personal connection or additional support.

The trial results suggest that self-guided use may be as effective as more intensive digital health support models guided by peer mentors and health coaches in a population with mild-to-moderate AUD. A self-guided support model is certainly less expensive and simpler for health systems to adopt than human-guided support models, and the differential attrition in the CI group suggests that persons with mild-to-moderate AUDs may value autonomy and anonymity over personalized and more intensive health coaching. In conclusion, making an evidence-based digital health intervention focused on wellness, mindful awareness, and prevention of AUD progression may be a viable option for health systems looking to promote evidence-based interventions for AUD.

Methods

Theoretical and empirical foundations

The protocol was approved by the University of Wisconsin Health Sciences Minimal Risk Institutional Review Board (2019-0337). This study was prospectively registered at clinicaltrials.gov on 7/03/2019 (NCT04011644). The study protocol was previously published and summarized here26. This trial used a digital medicine app called Tula (Sanskrit for “balance”) adapted from the Addiction-Comprehensive Health Enhancement Support System (A-CHESS)—one of the first digital medicine apps whose efficacy was supported in a trial of patients recovering from severe AUD after leaving 90-day residential treatment19. These results were replicated when A-CHESS was shown to be effective in reducing heavy drinking in a 2022 study comparing telephone- and smartphone-based continuing care for AUD16. A-CHESS has also been tested across a range of other populations15,27–29.

The theoretical framework supporting the A-CHESS app is self-determination theory, which asserts that meeting three fundamental psychological needs (competency, relatedness, and autonomous motivation) produces intrinsic motivation for behavior change30. Unlike A-CHESS, which focuses on abstinence as part of recovery from severe AUD, Tula focuses on reducing alcohol consumption for individuals with mild-to-moderate AUD. Tula is built upon the Whole Health Model, which focuses on “what matters to me” as a whole person and not just the disease (i.e., “what’s the matter with me?”)26. Tula places alcohol use within the context of integrative medicine, an adaptation made based on input from partnering healthcare stakeholders26. Tools and services included in Tula are listed in Table 4Table 4 Tula content and tools

Feature*	Description	
Thought of the Day	Daily inspirational quotes intended to motivate and engage participants.	
Whole Health	Information and tools to improve one’s whole health: what is whole health, circle of health, self-care, mindful awareness, whole health resources, and personal health inventory	
Motivation	Users can record in words and photos their reasons for wanting to work on their drinking and wellness	
Tracker **	The Tracker allows users to set and review goals, track their progress, and record their health and wellness patterns related to their quality of life (e.g., mood, sleep, social support).	
Communication***	Users can send and receive private messages with other Tula members and can access in-app discussion forums with other members of their group.	
Information	A content and resource library organized around the Circle of Health’s eight domains of self-care: working your body, sleep & recharge, food & drink, personal development, relationships, mood & mindset, surroundings, and the power of the mind.	
Relaxation	Relaxation techniques, audio recordings for guided meditation, and binaural beats.	
Strategies	Tips for reducing drinking, cognitive behavioral therapy, and goal setting.	
Gratitude	A daily prompt to reflect on gratitude.	
*All intervention content could be accessed immediately by the users.

*Tracker feature was accessed by all participants but is also monitored by the health coaches in the clinically integrated group as part of goal setting.

***Communication features are limited in the self-monitored group.

Eligibility criteria

Eligibility criteria are listed in Table 5. To maintain anonymity, all participants were identified by usernames they created while setting up their accounts.Table 5 Patient inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria	
21 years old or older	Reports symptoms consistent with severe AUD during screen (6 or more of 11 DSM-5 criteria)**	
Wants to reduce drinking	Active psychotic disorder diagnosis	
Owns a smartphone and is willing to download and use the Tula app	Acute medical problem requiring immediate hospitalization	
Lives within the healthcare system service area	Terminal illness	
Meets at least one of the following criteria:

• AUDIT38 screening score 8+, OR

• Responds “yes” to at least 2 questions on the AUD DSM-5+33, OR

• Reports moderate- to high-risk drinking patterns*

 ○ 4+ drinks on any single day and >7 drinks per week (women plus men over 65)

 ○ 5+ drinks on any single day and more than >14 drinks per week (men)

		

Setting

Study participants were recruited within the geographic catchment area of an integrated health system serving more than 700,000 patients each year in Wisconsin and the Upper Midwest. Individuals interested in the study used a QR code or a link from the study website to complete a secure, web-based screening survey. IP addresses and location verification were used to confirm that research participants lived within the health system’s catchment area.

Digital medicine support models: study groups

The study was designed to examine the clinical integration of digital medicine, as described by Hermes et al.31, where we compared a fully automated support model with two human-guided support models, differentiated by graduated levels of external support. While all participants had access to the basic content in Tula, the three study arms varied by the level of human touch (self-monitored “low touch,” peer-supported “medium touch,” and clinically integrated “high touch”). See Fig. 4.Fig. 4 Study groups.

The three study groups are listed in the columns, and their unique properties are compared in the rows.

Self-monitored (SM) group (low touch)

Participants in this group used the app on their own, like most commercially available health apps. They had no access to a discussion forum or private messaging nor any connection to an external support system (e.g., peer support specialist, health coach). Participants could communicate with the study team via email, phone, or a messaging function built into Tula. However, communication was limited to questions about the study and receiving tech support.

Peer-supported (PS) group (medium touch)

Participants in this group had access to social support from others in the same study arm via Tula’s discussion forum and to certified peer specialists who worked for a community-based non-profit organization that has historically partnered with the health system to provide peer mentoring services for patients with substance use disorders. Peer specialists are certified recovery coaches who have lived experience with substance use and/or mental health issues. They were not assigned to individual participants but were available to any participant who reached out via Tula’s private messaging feature. They also facilitated and moderated discussion forums and encouraged participants to utilize Tula’s resources.

Clinically integrated (CI) group (high touch)

Participants in this group worked one-on-one with health coaches employed within the healthcare system and had access to a discussion forum with other participants in their study arm. Health coaches are trained and certified in behavior change theories, motivational strategies, and health education. They worked with participants 1:1 to set goals for a healthier lifestyle through reduced alcohol consumption. Health coaches engaged with participants via Tula’s private messaging feature. Participants were offered three 1:1 health coaching sessions (via phone). Tula’s Tracker feature was also monitored by health coaches as part of goal setting.

Participant recruitment, retention, and timeline

We used a three-pronged recruitment strategy encompassing clinical settings, community-based organizations, and public media. We enlisted clinical study champions (primary care providers, behavioral health specialists, etc.) to provide information to potentially eligible patients. We also engaged local leaders from underrepresented communities to promote the study in ways that invite the inclusion of diverse perspectives. Lastly, we used targeted digital, television, and print media to promote the study broadly.

All participants who completed the eligibility screening survey received a $10 digital gift code. Study incentives were built into the first 12 weekly check-in surveys (as part of the self-monitoring feature of Tula) and four quarterly follow-up surveys at months 3, 6, 9, and 12. The incentives were distributed monthly in the form of digital gift codes sent via the message feature in Tula. By the end of the 12-month study period, participants could earn up to US $250 in digital gift codes for data submission.

Eligible participants were given a 72-hour window to download the app, complete a baseline survey, and schedule a phone call with research staff for enrollment. Recruitment lasted from March 2020–September 2023, with the last participant completing their 12-month intervention in September 2024.

Patient randomization

A sequence of randomized assignments for each group was generated by the project statistician using the block randomization procedure in the Power Analysis and Statistical Software package32. Randomization was stratified on participant-reported biological sex (male/female) and alcohol use severity (mild/moderate) based on DSM-5 score (mild ≤ 3 and moderate = 4–5)33, using participants’ responses to the screening survey. The randomization list was masked and placed in a protected spreadsheet. The randomization process involved a phone call between a study team member and the participant. When the study arm assignment was unmasked, a study ID was entered into the next available placement in the randomization sequence. While on the call, the study team member configured the participant’s app permissions based on their group assignment and reviewed a second informed consent specific to that study arm.

Blinding

Due to pragmatic and ethical considerations, the randomization assignment was not blind to either implementers or participants.

Outcomes

Demographics and severity

Participants reported biological sex and their AUD severity (mild or moderate; those with severe AUD were excluded) in the screening survey, which was used as strata for randomization. Participants reported additional demographics, including age, race/ethnicity, education, and income in the baseline survey.

Primary outcomes

The primary outcomes were HDD and QOL. HDD was measured using the 7-day timeline follow-back survey34,35. At baseline and during each quarterly survey, participants were asked how many days they had 5 or more standard drinks (for men under 65) or 4 or more standard drinks (for women and men over 65) on any day in the last 7 days. The definition of a standard drink was “One drink = one shot of hard liquor (1.5 oz.), 12-ounce can or bottle of beer, or 5-ounce glass of wine.” To allow for comparison with other studies using varying recall periods, we converted the number of HDD to a percentage (PHDD) by dividing by 7 (days/week).

Patient-reported QOL was measured using the four global physical health and four global mental health items in the Patient-Reported Outcome Measurement Information System (PROMIS) global health short form (SF10 ver.1.2)36. The raw scores for physical and mental health were converted to T-scores using the PROMIS scoring manual (p.16)36. T-scores are standardized scores that can be compared to the US general population and have a mean QOL score of 50 with a standard deviation of 10.

Participants received automated reminders to complete surveys in the app. Survey data were stored on a secure server at the University of Wisconsin.

Sample size

The study was designed to detect differences in two primary outcomes (HDD and the QOL) among three study arms across a 12-month period. Sufficient power (1–β = 0.80, multiple comparison-adjusted α = 0.00833, two-tailed) to detect a conservative effect size of Cohen’s d = 0.25 with four repeated measurements and a first-order autoregressive covariance structure (correlation ρ = 0.2) required ~182 participants per study arm (546 total), assuming 28% attrition16. An effect size of 0.25 equated to a difference of ~0.24 HDD per week, 2.18 points for QOL-Physical Health, and 2.03 points for QOL-Mental Health36.

Statistical methods

Descriptive analysis

We conducted descriptive analysis for demographic, utilization, and outcome variables across all three study arms. Group comparisons were conducted using chi-squared tests for categorical variables and t-tests for continuous variables. All statistical tests were two-tailed with α = 0.05.

Primary analysis

We constructed a longitudinal model of the outcome measures at months 3, 6, 9, and 12 after randomization using the GLMM with a random intercept and an auto-regression covariance structure for repeated measures. The stratification variables and the baseline values of the outcomes were included as covariates, with a separate model for each primary outcome. The primary analysis is the group fixed effects and pair-wise comparisons between study arms. Other significant fixed effects and pair-wise comparisons were reported. The Sidak method was used to adjust p-values in pair-wise comparisons37. Because the value of PHDD has a positive skewness, we used the square-root transformed PHDD, consistent with prior analyses of A-CHESS16. Participants who could not be reached for follow-up surveys were analyzed following the intention to treat principle. Missing outcomes data were considered missing at random. Other secondary analyses (mediation, economic, and qualitative)26 will be reported separately.

Ancillary analyses

Sensitivity analyses are included to examine model assumptions regarding equivalence of groups at baseline, skewness of the primary outcome, and to account for differential attrition (see Supplementary Information).

Important changes after trial commencement

Originally, in-person health coaching was available, but due to COVID-19, coaching was only offered via telephone. We also added QOL as a co-primary outcome26 to provide a more holistic evaluation for individuals with mild-to-moderate AUD and performed a square-root transformation on PHDD to be consistent with prior analyses16.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Supplementary information

Supplementary Information

Reporting Summary

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-024-01241-2.

Acknowledgements

This research was funded by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) grant number R01-AA024150. The funder had no role in any aspect of the development, conduct, analysis, or reporting of the study. The authors wish to thank our community partners: Safe Community of Madison and Dane County and the UW Health Wellness Center. In particular, we want to recognize Tanya Kraege, Charlestine “Charlie” Daniel, Cheryl Wittke, and the peer support specialists who were integral to moderating and facilitating discussion forums with participants in this study. We also want to recognize Diane Scherschel, the Integrative Health and Wellness Manager at the UW Health Wellness Center, as well as Katie Eby and Ellen Houston, who provided health coaching to our participants on their journey towards a healthier lifestyle. This project could not have succeeded without all their dedication and support. We thank Dave Gustafson Sr., Dave Gustafson Jr., and Adam Maus of the UW Center for Health Enhancement Systems Studies for their creation of the Tula app and technical support throughout the project. Lastly, we recognize the talents, efforts, and commitment of additional research team members towards this project: A.P., N.J., and S.V.

Author contributions

A.Q., R.B., M.C., B.B., and K.E. contributed to the study conception and design. A.P. and L.P. managed the project with supervisory help from A.Q. and B.B. and technical assistance from R.B. and X.L. L.P., K.E., and S.V. assisted project implementation and data collection. R.B., X.L., M.C., and S.V. performed analysis and interpreted results with input from B.B., L.P., and K.E. A.Q., M.C., and L.P. drafted the manuscript while B.B., R.B., A.P., Q.X., and R.L. reviewed and revised it for intellectual content. All authors read, reviewed the results, and approved the final manuscript.

Data availability

All data are available in the main text or the Supplementary Information.

Code availability

Analyses were conducted using IBM SPSS Statistics 29.0. The SPSS dataset and codes, containing de-identified data, and the SPSS syntax file used to generate the results in this paper are both available upon request and sent to corresponding author Dr. Andrew Quanbeck.

Competing interests

A.Q. has a shareholder interest in CHESS Health, a company that disseminates software technology to the addiction treatment field (CHESS Health did not develop the app tested in the manuscript). A.Q. also consults on the implementation of evidence-based healthcare interventions for the non-profit NIATx Foundation. These relationships are managed by the University of Wisconsin–Madison’s Conflict of Interest Committee. The remaining authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Centers for Disease Control and Prevention. Data on excessive drinking. https://www.cdc.gov/alcohol/data-stats.htm#:~:text=According%20to%20the%20Behavioral%20Risk,drink%20heavily%20also%20binge%20drink (2022).
2. Centers for Disease Control and Prevention. Behavioral risk factor surveillance system survey data. https://www.cdc.gov/brfss/index.html (2022).
3. Spillane S Trends in alcohol-induced deaths in the United States, 2000–2016 JAMA Netw. Open 2020 3 e1921451 10.1001/jamanetworkopen.2019.21451 32083687
Spillane, S. et al. Trends in alcohol-induced deaths in the United States, 2000–2016. JAMA Netw. Open 3, e1921451 (2020).32083687 10.1001/jamanetworkopen.2019.21451
4. Bertha M Shedden K Mellinger J Trends in the inpatient burden of alcohol‐related liver disease among women hospitalized in the United States Liver Int. 2022 42 1557 1561 10.1111/liv.15277 35451173
Bertha, M., Shedden, K. & Mellinger, J. Trends in the inpatient burden of alcohol‐related liver disease among women hospitalized in the United States. Liver Int. 42, 1557–1561 (2022).35451173 10.1111/liv.15277
5. Miller AP Diagnostic criteria for identifying individuals at high risk of progression from mild or moderate to severe alcohol use disorder JAMA Netw. Open 2023 6 e2337192 10.1001/jamanetworkopen.2023.37192 37815828
Miller, A. P. et al. Diagnostic criteria for identifying individuals at high risk of progression from mild or moderate to severe alcohol use disorder. JAMA Netw. Open 6, e2337192 (2023).37815828 10.1001/jamanetworkopen.2023.37192
6. Riper H Effectiveness and treatment moderators of internet interventions for adult problem drinking: an individual patient data meta-analysis of 19 randomised controlled trials PLOS Med, 2018 15 e1002714 10.1371/journal.pmed.1002714 30562347
Riper, H. et al. Effectiveness and treatment moderators of internet interventions for adult problem drinking: an individual patient data meta-analysis of 19 randomised controlled trials. PLOS Med, 15, e1002714 (2018).30562347 10.1371/journal.pmed.1002714
7. Quanbeck A Chih M-Y Isham A Gustafson D Mobile delivery of treatment for alcohol use disorders: a review of the literature Alcohol Res. Curr. Rev. 2014 36 111 122
Quanbeck, A., Chih, M.-Y., Isham, A. & Gustafson, D. Mobile delivery of treatment for alcohol use disorders: a review of the literature. Alcohol Res. Curr. Rev. 36, 111–122 (2014).
8. Domhardt M Geßlein H Von Rezori RE Baumeister H Internet- and mobile-based interventions for anxiety disorders: a meta-analytic review of intervention components Depress Anxiety 2019 36 213 224 10.1002/da.22860 30450811
Domhardt, M., Geßlein, H., Von Rezori, R. E. & Baumeister, H. Internet- and mobile-based interventions for anxiety disorders: a meta-analytic review of intervention components. Depress Anxiety 36, 213–224 (2019).30450811 10.1002/da.22860
9. Lepore SJ Collins BN Killam HW Barry B Supportive accountability and mobile app use in a tobacco control intervention targeting low-income minority mothers who smoke: observational study JMIR MHealth UHealth 2021 9 e28175 10.2196/28175 34255698
Lepore, S. J., Collins, B. N., Killam, H. W. & Barry, B. Supportive accountability and mobile app use in a tobacco control intervention targeting low-income minority mothers who smoke: observational study. JMIR MHealth UHealth 9, e28175 (2021).34255698 10.2196/28175
10. Versluis A Verkuil B Spinhoven P Van Der Ploeg MM Brosschot JF Changing mental health and positive psychological well-being using ecological momentary interventions: a systematic review and meta-analysis J. Med. Internet Res. 2016 18 e152 10.2196/jmir.5642 27349305
Versluis, A., Verkuil, B., Spinhoven, P., Van Der Ploeg, M. M. & Brosschot, J. F. Changing mental health and positive psychological well-being using ecological momentary interventions: a systematic review and meta-analysis. J. Med. Internet Res. 18, e152 (2016).27349305 10.2196/jmir.5642
11. Curran GM Bauer M Mittman B Pyne JM Stetler C Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact Med. Care 2012 50 217 226 10.1097/MLR.0b013e3182408812 22310560
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M. & Stetler, C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50, 217–226 (2012).22310560 10.1097/MLR.0b013e3182408812
12. Crane D Garnett C Michie S West R Brown J A smartphone app to reduce excessive alcohol consumption: identifying the effectiveness of intervention components in a factorial randomised control trial Sci. Rep. 2018 8 4384 10.1038/s41598-018-22420-8 29531280
Crane, D., Garnett, C., Michie, S., West, R. & Brown, J. A smartphone app to reduce excessive alcohol consumption: identifying the effectiveness of intervention components in a factorial randomised control trial. Sci. Rep. 8, 4384 (2018).29531280 10.1038/s41598-018-22420-8
13. Reynolds A Keough MT O’Connor RM Is being mindful associated with reduced risk for internally-motivated drinking and alcohol use among undergraduates? Addict. Behav. 2015 42 222 226 10.1016/j.addbeh.2014.11.027 25489665
Reynolds, A., Keough, M. T. & O’Connor, R. M. Is being mindful associated with reduced risk for internally-motivated drinking and alcohol use among undergraduates? Addict. Behav. 42, 222–226 (2015).25489665 10.1016/j.addbeh.2014.11.027
14. Attwood S Parke H Larsen J Morton KL Using a mobile health application to reduce alcohol consumption: a mixed-methods evaluation of the drinkaware track & calculate units application BMC Public Health 2017 17 394 10.1186/s12889-017-4358-9 28511698
Attwood, S., Parke, H., Larsen, J. & Morton, K. L. Using a mobile health application to reduce alcohol consumption: a mixed-methods evaluation of the drinkaware track & calculate units application. BMC Public Health 17, 394 (2017).28511698 10.1186/s12889-017-4358-9
15. Quanbeck A Implementing a mobile health system to integrate the treatment of addiction into primary care: a hybrid implementation-effectiveness study J. Med. Internet Res. 2018 20 e37 10.2196/jmir.8928 29382624
Quanbeck, A. et al. Implementing a mobile health system to integrate the treatment of addiction into primary care: a hybrid implementation-effectiveness study. J. Med. Internet Res. 20, e37 (2018).29382624 10.2196/jmir.8928
16. McKay JR Efficacy and comparative effectiveness of telephone and smartphone remote continuing care interventions for alcohol use disorder: a randomized controlled trial Addiction 2022 117 1326 1337 10.1111/add.15771 34859519
McKay, J. R. et al. Efficacy and comparative effectiveness of telephone and smartphone remote continuing care interventions for alcohol use disorder: a randomized controlled trial. Addiction 117, 1326–1337 (2022).34859519 10.1111/add.15771
17. Kiluk BD Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes Am. J. Psychiatry 2018 175 853 863 10.1176/appi.ajp.2018.17090978 29792052
Kiluk, B. D. et al. Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes. Am. J. Psychiatry 175, 853–863 (2018).29792052 10.1176/appi.ajp.2018.17090978
18. Tetrault JM Computerized cognitive behavioral therapy for substance use disorders in a specialized primary care practice: a randomized feasibility trial to address the RT component of SBIRT J. Addict. Med. 2020 14 e303 e309 10.1097/ADM.0000000000000663 32371660
Tetrault, J. M. et al. Computerized cognitive behavioral therapy for substance use disorders in a specialized primary care practice: a randomized feasibility trial to address the RT component of SBIRT. J. Addict. Med. 14, e303–e309 (2020).32371660 10.1097/ADM.0000000000000663
19. Gustafson DH A smartphone application to support recovery from alcoholism: a randomized clinical trial JAMA Psychiatry 2014 71 566 572 10.1001/jamapsychiatry.2013.4642 24671165
Gustafson, D. H. et al. A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry 71, 566–572 (2014).24671165 10.1001/jamapsychiatry.2013.4642
20. Daniore P Nittas V Von Wyl V Enrollment and retention of participants in remote digital health studies: scoping review and framework proposal J. Med. Internet Res. 2022 24 e39910 10.2196/39910 36083626
Daniore, P., Nittas, V. & Von Wyl, V. Enrollment and retention of participants in remote digital health studies: scoping review and framework proposal. J. Med. Internet Res. 24, e39910 (2022).36083626 10.2196/39910
21. Lucas GM Gratch J King A Morency L-P It’s only a computer: virtual humans increase willingness to disclose Comput. Hum. Behav. 2014 37 94 100 10.1016/j.chb.2014.04.043
Lucas, G. M., Gratch, J., King, A. & Morency, L.-P. It’s only a computer: virtual humans increase willingness to disclose. Comput. Hum. Behav. 37, 94–100 (2014).10.1016/j.chb.2014.04.043
22. Loudon, K. et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 350, h2147 (2015).
23. Bishop FL Lewith GT Who uses CAM? A narrative review of demographic characteristics and health factors associated with CAM use Evid. Based Complement. Altern. Med. 2010 7 11 28 10.1093/ecam/nen023
Bishop, F. L. & Lewith, G. T. Who uses CAM? A narrative review of demographic characteristics and health factors associated with CAM use. Evid. Based Complement. Altern. Med. 7, 11–28 (2010).10.1093/ecam/nen023
24. Rodriguez JA Lyles CR Strengthening digital health equity by balancing techno-optimism and techno-skepticism through implementation science Npj Digit. Med. 2023 6 203 10.1038/s41746-023-00954-0 37919418
Rodriguez, J. A. & Lyles, C. R. Strengthening digital health equity by balancing techno-optimism and techno-skepticism through implementation science. Npj Digit. Med. 6, 203 (2023).37919418 10.1038/s41746-023-00954-0
25. Clark NM The effect of patient choice of intervention on health outcomes Contemp. Clin. Trials 2008 29 679 686 10.1016/j.cct.2008.04.002 18515187
Clark, N. M. et al. The effect of patient choice of intervention on health outcomes. Contemp. Clin. Trials 29, 679–686 (2008).18515187 10.1016/j.cct.2008.04.002
26. Park LS Testing an mHealth system for individuals with mild to moderate alcohol use disorders: Protocol for a type 1 hybrid effectiveness-implementation trial JMIR Res. Protoc. 2022 11 e31109 10.2196/31109 35179502
Park, L. S. et al. Testing an mHealth system for individuals with mild to moderate alcohol use disorders: Protocol for a type 1 hybrid effectiveness-implementation trial. JMIR Res. Protoc. 11, e31109 (2022).35179502 10.2196/31109
27. Health, C. The mPOWER Project: results of a pharmacotherapy model integrated with mobile health technology for veterans working through substance use disorders. CHESS Health https://www.chess.health/case-studies/the-mpower-project-results-of-a-pharmacotherapy-model-integrated-with-mobile-health-technology-for-veterans-working-through-substance-use-disorders/ (2020).
28. Johnson K A pilot test of a mobile app for drug court participants Subst. Abus. Res. Treat. 2016 10 SART.S33390 10.4137/SART.S33390
Johnson, K. et al. A pilot test of a mobile app for drug court participants. Subst. Abus. Res. Treat. 10, SART.S33390 (2016).10.4137/SART.S33390
29. Johnston DC Mathews WD Maus A Gustafson DH Using smartphones to improve treatment retention among impoverished substance-using Appalachian women: a naturalistic study Subst. Abus. Res. Treat. 2019 13 117822181986137 10.1177/1178221819861377
Johnston, D. C., Mathews, W. D., Maus, A. & Gustafson, D. H. Using smartphones to improve treatment retention among impoverished substance-using Appalachian women: a naturalistic study. Subst. Abus. Res. Treat. 13, 117822181986137 (2019).10.1177/1178221819861377
30. Ryan RM Deci EL Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being Am. Psychol. 2000 55 68 78 10.1037/0003-066X.55.1.68 11392867
Ryan, R. M. & Deci, E. L. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78 (2000).11392867 10.1037/0003-066X.55.1.68
31. Hermes ED Lyon AR Schueller SM Glass JE Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes J. Med. Internet Res. 2019 21 e11752 10.2196/11752 30681966
Hermes, E. D., Lyon, A. R., Schueller, S. M. & Glass, J. E. Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes. J. Med. Internet Res. 21, e11752 (2019).30681966 10.2196/11752
32. PASS 2020. Power Analysis and Sample Size Software. NCSS, LLC (2020).
33. National Institute on Alcohol Abuse and Alcoholism. Alcohol use disorder: a comparison between DSM-IVDSM-5. https://www.niaaa.nih.gov/publications/brochures-and-fact-sheets/alcohol-use-disorder-comparison-between-dsm (2021).
34. Sklar SM Turner NE A brief measure for the assessment of coping self-efficacy among alcohol and other drug users Addiction 1999 94 723 729 10.1046/j.1360-0443.1999.94572310.x 10563037
Sklar, S. M. & Turner, N. E. A brief measure for the assessment of coping self-efficacy among alcohol and other drug users. Addiction 94, 723–729 (1999).10563037 10.1046/j.1360-0443.1999.94572310.x
35. Williams GC Cox EM Kouides R Deci EL Presenting the facts about smoking to adolescents: effects of an autonomy-supportive style Arch. Pediatr. Adolesc. Med. 1999 153 959 964 10.1001/archpedi.153.9.959 10482213
Williams, G. C., Cox, E. M., Kouides, R. & Deci, E. L. Presenting the facts about smoking to adolescents: effects of an autonomy-supportive style. Arch. Pediatr. Adolesc. Med. 153, 959–964 (1999).10482213 10.1001/archpedi.153.9.959
36. Hays RD Bjorner JB Revicki DA Spritzer KL Cella D Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items Qual. Life Res. 2009 18 873 880 10.1007/s11136-009-9496-9 19543809
Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L. & Cella, D. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual. Life Res. 18, 873–880 (2009).19543809 10.1007/s11136-009-9496-9
37. Davis, C. SPSS for Applied Sciences: Basic Statistical Testing. (CSIRO Publ, Collingwood, 2013).
38. Volk RJ Steinbauer JR Cantor SB Holzer CE The Alcohol Use Disorders Identification Test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic backgrounds Addiction 1997 92 197 206 9158231
Volk, R. J., Steinbauer, J. R., Cantor, S. B. & Holzer, C. E. The Alcohol Use Disorders Identification Test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic backgrounds. Addiction 92, 197–206 (1997).9158231
