
==== Front
Npj Ment Health Res
Npj Ment Health Res
NPJ Mental Health Research
2731-4251
Nature Publishing Group UK London

39313519
89
10.1038/s44184-024-00089-4
Article
A systematic review on passive sensing for the prediction of suicidal thoughts and behaviors
Büscher Rebekka rebekka.buescher@mps.uni-freiburg.de

12
Winkler Tanita 1
Mocellin Jacopo 34
Homan Stephanie 34
Josifovski Natasha 5
Ciharova Marketa 256
van Breda Ward 2
Kwon Sam 7
Larsen Mark E. 58
Torous John 7
Firth Joseph 9
Sander Lasse B. 1
1 https://ror.org/0245cg223 grid.5963.9 0000 0004 0491 7203 Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Freiburg, Germany
2 https://ror.org/008xxew50 grid.12380.38 0000 0004 1754 9227 Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
3 https://ror.org/02crff812 grid.7400.3 0000 0004 1937 0650 Experimental Psychopathology and Psychotherapy, Department of Psychology, University of Zurich, Zurich, Switzerland
4 grid.7400.3 0000 0004 1937 0650 Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric University Hospital Zurich, University of Zurich, Zurich, Switzerland
5 grid.1005.4 0000 0004 4902 0432 Black Dog Institute, University of New South Wales, Sydney, NSW Australia
6 https://ror.org/008xxew50 grid.12380.38 0000 0004 1754 9227 Amsterdam Public Health Research Institute, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
7 grid.38142.3c 000000041936754X Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA USA
8 https://ror.org/03r8z3t63 grid.1005.4 0000 0004 4902 0432 Centre for Big Data Research in Health, University of New South Wales, Sydney, NSW Australia
9 grid.5379.8 0000000121662407 Division of Psychology and Mental Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
23 9 2024
23 9 2024
2024
3 4226 10 2023
12 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Passive sensing data from smartphones and wearables may help improve the prediction of suicidal thoughts and behaviors (STB). In this systematic review, we explored the feasibility and predictive validity of passive sensing for STB. On June 24, 2024, we systematically searched Medline, Embase, Web of Science, PubMed, and PsycINFO. Studies were eligible if they investigated the association between STB and passive sensing, or the feasibility of passive sensing in this context. From 2107 unique records, we identified eleven prediction studies, ten feasibility studies, and seven protocols. Studies indicated generally lower model performance for passive compared to active data, with three out of four studies finding no incremental value. PROBAST ratings revealed major shortcomings in methodology and reporting. Studies suggested that passive sensing is feasible in high-risk populations. In conclusion, there is limited evidence on the predictive value of passive sensing for STB. We highlight important quality characteristics for future research.

Subject terms

Risk factors
Psychiatric disorders
issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Over 700,000 people worldwide die by suicide every year1. In the US, the number of suicide attempts is approximately 25 times higher than the number of deaths by suicide2. Suicide is the fourth leading cause of death among young people1 and worryingly, rates are rising in this age group, with deaths by suicide increasing by over 45% among adolescents over the last 10 years in the US3. The majority of transitions from suicidal ideation to behaviors occur within 12 months from onset4.

Research efforts have been extensive in identifying risk factors for suicidal thoughts and behaviors (STB)5,6. However, the prediction of STB remains a major challenge5. There are several reasons for this. First, the base rate of STB, particularly suicide attempts and deaths, is low, making large sample sizes necessary for developing risk prediction models5. Second, research has repeatedly focused on distal risk factors that have little predictive value for the timing of a suicide attempt5. Third, the remarkable fluctuations in suicidal ideation, as observed in several studies based on ecological momentary assessment (EMA), exacerbate the problems around the prediction of STB7,8. While initial suicidal thoughts often occur years before an attempt, the final steps towards attempts, including the final decision, are taken within days to minutes9. Fourth, STB result from a complex interaction of various risk factors that is insufficiently understood10–12. These challenges are mirrored by the finding that clinicians’ risk assessments cannot categorize their patients’ risk states in clinically meaningful ways13. In order to navigate these complexities, it has been suggested that research should focus on short-term risk prediction for STB instead of distal factors5.

New and promising approaches have emerged for studying and potentially identifying shifts in risk states for STB in real-time, such as EMA using smartphone-based surveys and passive sensing14. EMA requires active participation15, which can potentially be burdensome for participants, especially during longer assessment periods8. In contrast, passive sensing passively collects digital markers (e.g., geolocation, sleep parameter) via smartphones or wearable devices14,16,17. A range of sensors can be used to capture a variety of data points, depending on the characteristics of a device. Predominantly, physiological (e.g., electrodermal activity), social (e.g., call logs, apps used), and behavioral (e.g., sleep duration) signals can be captured via sensors18. Compared to questionnaires only capturing a certain moment in time, passive sensing allows an objective and continuous flow of data14,17.

Several viewpoint articles have delineated promises and challenges associated with the application of passive sensing within the field of suicide prevention14,19–21. Nonetheless, a systematic review of the current state of research remains absent. Against this backdrop, we aimed to systematically synthesize the current evidence on passive sensing for the prediction of STB. This review encompassed the exploration of three core facets: (1) the potential of passive sensing data to enhance STB prediction, (2) the discriminative efficacy of various sensor types, and (3) an examination of the statistical analysis methodologies documented in the literature. Additionally, we synthesized findings pertaining to the feasibility of employing passive sensing for the assessment of STB.

Methods

This report follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines22 (see Supplementary Table 1). All procedures were pre-registered (https://osf.io/hzxua) and reported in a study protocol23. Due to the infancy of the field, we decided to broaden the scope of the review by a) additionally including feasibility papers and study protocols, and by b) additionally including prediction papers with non-clinical samples, regardless of the presence of STB.

Eligibility criteria

We included studies with participants with and without STB at baseline, without restrictions on age, gender, or current treatment status. Studies had to address the use of passive data through smartphones or wearables in the context of STB. We included all passive sensor modalities. We included studies regardless of whether participants received treatment. Studies were eligible if they reported a quantitative measure of STB as an outcome; we excluded studies restricted to non-suicidal self-injurious behavior. Studies were eligible if they reported results either on the association between passive data and STB or on the feasibility of passive sensing. Study protocols were additionally included to provide a perspective on the research currently being processed. We included peer-reviewed articles without any restrictions on language or publication date.

Search strategy

The following databases were searched from inception to June 24, 2024: MEDLINE, PubMed, Embase, PsycINFO, and Web of Science. The search string is displayed in the study protocol23. We validated the search string against seven hand-searched relevant articles; the search string reached a coverage rate of 100%. The relevant articles were screened using the online tool Covidence. In the first step, two independent researchers screened all titles and abstracts against the eligibility criteria. After obtaining all relevant full texts, a second screening of the articles was conducted. Conflicts were resolved in discussion with a third reviewer (LS) where necessary.

Data extraction

All relevant information was extracted from the final set of studies and double-checked by an independent researcher, including the following variables: Study identification items, study design, sociodemographic and clinical sample characteristics, data collection device, type and collection frequency of passive data, sensors used, assessment of STB, assessment length, and analysis methods. Any findings about an association between the passive data and STB or the predictive value of the passive data for STB were extracted. Descriptive results on feasibility of passive sensing were extracted as secondary outcomes.

Risk of bias assessment

Two independent researchers assessed risk of bias using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) tool24. The PROBAST tool is specifically designed to assess potential sources of bias in studies for the development or evaluation of prediction models. It covers the following domains: participants, predictors, outcome, and analysis. Conflicts were resolved in discussion with a third researcher.

Data analysis

The results were described narratively. In the field of prediction modeling, there are a variety of performance measures, which complicates the reporting and interpretation of the findings. Therefore, we selected and reported performance measures of prediction models in line with a decision tree developed by WVB and MC25 that provides a hierarchy of the most meaningful performance measures, along with a guideline for the interpretation of the predictive value (i.e., very good, good, acceptable, or poor). These were reported in the section “Main Findings of Prediction Studies”. Where deemed helpful for the understanding of the findings, additional measures were reported. A meta-analytic synthesis of predictive validity was not possible due to the heterogeneity in the identified studies.

Results

Search results

The search yielded 2107 unique records (see Fig. 1 for the study selection process). In a first step, two independent reviewers screened the titles and abstracts against eligibility criteria. In the next step, they screened the 103 potentially relevant full texts and resolved any conflicts with an independent researcher, revealing 27 eligible articles. This included eleven studies investigating the predictive value of passive sensing for the prediction of STB26–36, ten trials focusing on the feasibility of passive sensing28,37–45, and seven study protocols46–52. One article reported on two studies: a feasibility investigation and a prediction study28. All papers were published between 2019 and 2024.Fig. 1 Study selection flowchart.

STB suicidal thoughts and behaviors.

Characteristics of prediction studies

The study characteristics are displayed in Table 1. Sample sizes ranged from 2532 to 2881 participants30. More than half of the studies were based on clinical populations, i.e., participants were in inpatient or outpatient treatment26,29,32–36. All non-clinical studies had larger sample sizes than clinical studies, at least 28127. One study recruited adolescents33, whereas the other studies were in adult populations.Table 1 Characteristics of prediction studies

Author (year)	Predictors (device name)	Sensors/device analytics (study length)	Assessment of STB (frequency)	Main statistical analysis	Sample characteristics	
Barrigon et al.34	Distance traveled, time spent at home, steps, app usage (Smartphone app: eB2)	GPS, device usage, activity (measured by Google Fit), accelerometer (6 months)	Electronic health records: suicide attempts or psychiatric emergency department visits (NA)	Unsupervised machine learning model (Heterogeneous Mixture Model)	225 adult outpatients with a history of suicide attempt or ideation	
Bertrand et al.26	Sleep characteristics (Wrist device: GENEActiv)	Accelerometer (14 days)	C-SSRS quick screen (2), QIDS-SR-16 item 12 (2), MADRS item 10 (2)	Spearman’s rank order correlations, multiple linear regression	76 adult outpatients with bipolar disorder	
Coyne et al.27	Smartphone app use, including social media (Smartphone apps: Moment or Cronicle)	Screentime (14 days)	SBQ-R (1)	Multilevel models	281 adults from the general population	
Czyz et al.35	Sleep, heart rate, steps (Wrist device: Fitbit)	NA (56 days)	Two EMA items: duration and intensity of ideation (4 per day)	Mixed-effects classification and regression trees	102 young adult emergency department patients with acute suicidality	
Dogrucu et al.28	Retrospective: GPS, browser history, call logs, text messages, contacts, social media (Smartphone app: Moodable)	GPS, microphone, calls, quantitative features of contacts, text messages, social media (14 days)	PHQ-9 item 9 (1)	Machine learning algorithms (KNN, SVM, RF)	335 adults (general population), recruited via online platform (Mechanical Turk)	
Haines-Delmont et al.29	Sleep, steps, smartphone usage including social media (Smartphone apps: SwiM, Apple Health kit; wrist device: Fitbit)	Accelerometer, Wi-Fi, screentime (7 days)	C-SSRS (3)	Machine learning algorithms (KNN, RF, SVM), logistic regression	66 adult inpatients (acute mental health)	
Horwitz, Czyz et al.30	Sleep, physical activity, heart rate (Smartphone; wrist device: Fitbit)	Accelerometer, gyroscope, light sensor (92 days)	PHQ-9 item 9 (2)	Three-step hierarchical logistic regression with suicidal subsample (n = 217)	2881 first-year medical residents	
Horwitz, Kentopp et al.31	Sleep, physical activity, heart rate (Smartphone; wrist device: Fitbit)	Accelerometer, gyroscope, light sensor (92 days)	PHQ-9 item 9 (2)	Machine learning algorithms (RF, ENR)	2459 first-year medical residents	
Kleiman et al.32	Physiological distress (electrodermal activity)b (Smartphone app: LifeData; wrist device: Empatica Embrace 2)	Accelerometer, gyroscope, thermostat, electrodermal activity sensor (Inpatient stay + 28 days after discharge)	Three EMA items (6 per day)	Multilevel models	25 adult inpatients with acute suicidality	
Salvatore et al.36	Circadian activity (Wrist device: Motionlogger)	Accelerometer (3 days)	EMA item: wish to die (3; one per day)	Linear regression	83 adult outpatients with a current major depressive episode and either a bipolar or major depressive disorder	
Sheridan et al.33	Heart rate variability (wrist device)	Light sensor (7 days)	C-SSRS (3)	Multilevel models	51 adolescent inpatients with acute suicidality	
C-SSRS Columbia-Suicide Severity Rating Scale, EMA ecological momentary assessment, GPS Global Positioning System, MADRS Montgomery Åsberg Depression Rating Scale, PHQ-9 Patient Health Questionnaire, QIDS-SR Quick Inventory of Depressive Symptomatology, SBQ-R Revised Suicidal Behavior Questionnaire, STB suicidal thoughts and behaviors, Wi-Fi wireless fidelity.

Ten studies investigated behavioral signals26–32,34–36, including sleep, physical activity, smartphone usage, and activity patterns such as time spent at home. Five examined physiological signals using wrist devices30–33,35, including the heart rate (variability) and the electrodermal activity. Three explored social signals27–29, including social media use, text messages, call logs, and phone contacts. Overall, smartphone usage (7/11)27–32,34, sleep characteristics (6/11 studies)26,29–31,35,36, and physical activity (5/11 studies)29–31,34,35 were studied most frequently. Studies that asssessed physiological or social signals typically combined them with behavioral signals; only one study investigated a physiological signal (heart rate variability) alone33. To collect passive data, three studies used smartphones only27,28,34, four used wrist devices only26,33,35,36, and four studies used both30–32. Four studies used research grade wearables26,32,33,36, and four used commercial wearables29–31,35. Only one study reported the sampling frequency of the device; it was a frequency of 4hz in electrodermal activity assessment32. Several studies reported pre-processing regarding the time resolution of the data; one study reported a summary for the past 14 days28, seven studies reported a daily-level, and two an hourly-level aggregation32,33. Two studies provided a detailed listing of the device sensors and analytics32,33. The passive data collection interval varied between 7 days29,33 and 6 months34.

Studies differed in how they assessed STB. Several studies assessed suicidal ideation using single items of validated scales, (e.g., item 9 from the PHQ-9)26,28,30,31. Furthermore, the Columbia-Suicide Severity Rating Scale (C-SSRS) was frequently used26,29,33, and one study used the Suicide Behaviors Questionnaire (SBQ)27. The studies using the C-SSRS and the SBQ used a composite score, which did not differentiate between suicidal thoughts and behaviors. One study assessed suicide attempts and emergency department visits via electronic health records34. Studies divided the outcome into the presence or absence of STB (8/11 studies)28–35 and/or evaluated STB on a severity level (5/11 studies)26,27,32,33,36. Eight studies assessed STB between one and three times in total, and two studies used EMA items to collect momentary data on suicidal ideation several times a day32,35.

Regarding statistical analysis strategies, two studies calculated correlations and/or regression analyses26,36, four studies applied multilevel models27,30,32,33, and five studies used machine learning algorithms28,29,31,34,35.

Main findings of prediction studies

Seven studies developed predictive models based on multiple predictors28–32,34,35; details on the study designs are displayed in Table 1. In an EMA study with suicidal inpatients (n = 25), Kleiman and colleagues32 investigated the electrodermal activity (i.e., autonomic arousal) until 28 days after discharge. While physiological data did predict suicidal ideation both concurrently (according to a performance score on the percentage of investigated performance indices that speak for best fit: 0.00–1.14%; conditional R2 = 0.76–0.92; moderate to strong relationship) and prospectively (performance score: 0.00–40.80%; conditional R2 = 0.78–0.94; moderate to strong relationship), multilevel prediction models performed poorest when only physiological data was included (i.e., poorer than models with only self-reported affective states, with performance score: 29.99–99.95%; conditional R2 = 0.80–0.96; strong relationship). When physiological data was added to self-reported affect, the fit improved for the outcome severity of suicidal ideation in seven out of eight models (performance score: 59.40–100.0%; conditional R2 = 0.80–0.83; strong relationship vs. only self-report performance score: 54.33–86.05%; conditional R2 = 0.80–0.81; strong relationship) and worsened for the presence of suicidal ideation in seven out of eight models (performance score: 27.90–83.02%; conditional R2 = 0.92–0.96; strong relationship vs. only self-report performance score: 29.99–99.95%; conditional R2 = 0.92–0.96; strong relationship). In an EMA study with suicidal emergency department patients (n = 102), Czyz and collegues35 assessed sleep duration, steps, and resting heart rate. While the passive sensing data predicted next-day suicidal ideation, the model performance was poor (mean AUC = 0.56). When combined with EMA data, the model performance was good (mean AUC = 0.84); however, passive sensing did not improve the prediction compared to EMA only (mean AUC = 0.84; good performance).

No other studies assessed momentary STB. Haines-Delmont and colleagues assessed actively collected (i.e., sociodemographic variables, mood assessments, journaling) and passively collected data (sleep, steps, smartphone usage) in a sample of adults admitted to acute mental health wards29. The optimal prediction model for STB was the k-nearest-neighbors algorithm; the performance was poor (AUC = 0.65). In a prospective study, Barrigon and collegues34 passively monitored adult outpatients (n = 225) with a history of suicidal ideation or attempt for six months via their smartphones. Individual daily activity profiles were created based on distance traveled, steps, time at home, and app usage, to predict risk events (suicide attempts or emergency department visits requiring psychiatric assessment) within a 1-week window. The machine learning model based on these profiles predicted short-term risk with a good model performance (AUC = 0.78). The study by Horwitz and colleagues30,31 assessed sleep, physical activity, heart rate, and mood using a Fitbit and a mobile app in first-year medical residents during the first three months of internship. In participants from an earlier wave (academic years 2018–2020, n = 2881), the three-step hierarchical logistic regression indicated that the passive data did not have incremental predictive validity for the prediction of suicidal ideation over baseline demographic and clinical variables (step one AUC 0.735, acceptable performance; vs. step two 0.739, acceptable performance; BIC step one 1125.0 vs. step two 1155.2)30. In a later cohort (academic year 2020–2021, n = 2459), a machine learning model31 that only used daily self-reported mood performed better (i.e., a sensitivity of 0.70 and a specificity of 0.73, AUC = 0.74; acceptable performance) than the model that additionally incorporated passive data (i.e., sensitivity of 0.67 and a specificity of 0.69, AUC = 0.70; acceptable performance). Dogrucu and colleagues28 assessed active data (i.e., a brief voice sample) combined with passive social media account data and smartphone usage data (i.e., GPS, browser history, call logs) that was collected retrospectively from smartphones in a non-clinical adult sample (n = 335). The machine learning algorithm predicted presence vs. absence of suicidal ideation with a good performance of F1 = 0.85 using active and passive data; no alternative models were reported.

Four studies26,27,33,36 investigated associations of single passive data modalities with STB. In a sample of adult outpatients diagnosed with bipolar disorder (n = 76), Bertrand and colleagues26 assessed sleep characteristics via wrist actigraphy. Using three different measurements of STB and no correction for multiple testing, they found associations of several sleep variables with one of the STB measures each, inconsistent across instruments and timepoints. Sheridan and colleagues33 investigated the heart rate variability using a wrist device in acutely suicidal adolescents (n = 51). They found an increase in the high-frequency component (i.e., higher parasympathetic activity) in patients that had at least a 25% decrease in STB within the 7-day study period. Salvatore and collegues36 assessed the circadian activity using a wrist device in adult outpatients with a current major depressive episode and either a major depression or bipolar disorder. They found no association between the self-rated wish to die and circadian activity. This included the amplitude (β = −0.19, p > 0.05), daytime activity (β = −0.19, p > 0.05), mesor (β = −0.19, p > 0.05), and nighttime activity (β = −0.11, p > 0.05). However, they found a positive correlation between these activity indices and the self-rated wish to live and the wish to die. Coyne and colleagues27 assessed smartphone app usage in a non-clinical cohort of adolescents (n = 281). They found that in girls, a higher use of entertainment apps was associated with an increased risk for STB (composite score; β = 0.23–0.25, p < 0.001); in boys, the use of reading apps (e.g., Reddit) was associated with an increased higher risk for STB (β = 0.28–0.29, p < 0.001).

Risk of bias in prediction studies

Risk of bias in prediction model studies was assessed using the PROBAST tool. The ratings per domain are displayed in Table 2 (see Supplementary Table 2 for detailed ratings). Most studies (7/11) had a high risk of bias in the domain of participants, mostly because the eligibility criteria and reasons for exclusion were unclear. Across all studies, the bias potential was low in the domain of predictors. Furthermore, the bias potential regarding the outcome was high in most trials (9/11). STB were often assessed using single items, drastically limiting the reliability and validity of the assessment53. An additional potential for bias was introduced by choosing arbitrary cut-offs to determine high or low suicide risk, or insufficient information on how this was calculated. One study assessed change by 25% in C-SSRS scores33, which does not account for measurement error like in the Reliable Change Index54. Risk of bias was mostly high in the domain of analysis (8/11 studies). Most studies were either not pre-registered or had vague pre-registration, failing to provide clear information on how predictors were assessed in a time-resolved manner, selected, and analyzed. Studies rarely reported on how they dealt with missing data. There was often imbalance in the sample regarding the proportion of participants with the outcome; frequently, it was unclear how imbalance was managed. Another critical issue in the analysis domain was the insufficient validation of prediction models. Only three studies used a train-test split approach or cross validation of the model29,31,35. This failure to test on out-of-sample observations exposes to the risk of over-fitting and does not allow conclusions to be drawn on the generalizability of the prediction results.Table 2 Risk of bias of prediction studies

Study	Participants	Predictors	Outcome	Analysis	
Barrigon et al.34	+	+	+	-	
Bertrand et al.26	+	+	-	+	
Coyne et al.27	-	+	-	+	
Czyz et al.35	+	+	-	+	
Dogrucu et al.28	-	+	-	-	
Haines-Delmont et al.29	-	+	-	-	
Horwitz, Czyz et al.30	-	+	-	-	
Horwitz, Kentopp et al.31	-	+	-	-	
Kleiman et al.32	+	+	+	-	
Salvatore et al.36	-	+	-	-	
Sheridan et al.33	-	+	-	-	
PROBAST Prediction model Risk OF Bias ASsessment Tool, ROB risk of bias. + indicates low ROB; - indicates high ROB.

Characteristics of feasibility studies

In total, ten studies investigated the feasibility of passive sensing in the context of STB. A summary of all relevant study characteristics is displayed in Table 3. Nine studies examined behavioral signals28,37–42,44,45, six studied social signals28,37,38,41,44,45, and three investigated physiological signals39,40,43. Data collection lasted between one retrospective session37,38 and a mean of 229.4 days (SD = 168), varying across participants39. Sample size varied from 50 subjects43 to 1709 subjects45, with five studies investigating fewer than 100 subjects37,39,41–43. Samples consisted of adults in eight studies28,37–41,44,45 and adolescents in two studies42,43. Six studies examined clinical samples39–44, three investigated general samples28,37,38, and one study included students and a sample of patients with and without STB45. The definition of feasibility varied across trials; studies focused on user experience, adherence to passive sensing technology (i.e., wearing the device or keeping a sensing app on a smartphone), and the willingness to share data.Table 3 Characteristics of feasibility studies

Author(s) (year)	Feasibility definition (measure)	Device (name)	Sensors/device analytics (study length)	Sample characteristics	Findings	
Bruen et al.41	User experience (qualitative interview), rate of Facebook sharing	Smartphone app (SwiM), wrist device (Fitbit)	Accelerometer, Wi-Fi, screentime (7 days)	80 adult inpatients	The majority provided positive feedback on Fitbit. A total of 19% of participants provided access to Facebook data.	
Dogrucu et al.28	Willingness to disclose passive smartphone data, social media (survey)	Not applicable: no passive data collection	Not applicable, no passive data collection	202 adults, recruited via online platform (Mechanical Turk)	Participants reported more willingness to share active (e.g., voice recordings) compared to passive data (e.g., browser history).	
Glenn et al.42	Participation rate, device adherence, user experience and acceptability (interview), burdensomeness and impact on care (clinician survey)	Wrist device (Actiwatch), smartphone for EMA	Accelerometer (28 days)	53 adolescents after discharge from acute psychiatric care for suicide risk	Participation rate of 25%. Participants wore the wrist device on 76% of days. 71% of participants stated that wearing the wrist device was comfortable. Clinicians found their study participation was minimally burdensome.	
Jiang et al.40	Device adherence (time worn) and barriers to adherence (questionnaire), acceptability (questionnaire)	Wrist device (Fitbit), smartphone for EMA	No information (56 days)	106 young adults with last-month suicide attempt or last-week ideation	102/106 enrolled participants wore the Fitbit at least 1 minute, with a mean of 53.6% of minutes and on 65.9% of days within the monitoring period. Satisfaction mean 3.89 (SD 1.19), 92.6% found it comfortable, 18.1% reported interference with daily activities, 85.1% would wear it outside a study. Most frequent barriers: water-related activity (45.7%), forgot to wear it (39.4%).	
Kleiman et al.43	Device adherence (hours worn, correct use of event marker), user experience (Wearable Computer Comfort Rating Scale; qualitative interview)	Wrist device (Empatica E4), smartphone for EMA	Accelerometer, thermostat, electrodermal activity sensor, light sensor (duration of hospitalization, mean days = 10.7, SD = 13.86)	50 adolescent inpatients with acute STB	Wrist device worn for M = 20.3 hours per day, and at 95% of all days; 0.46% of event marker presses were accidental. Most items on discomfort rated below 5/10. Qualitative findings: complaints about the monitor (e.g., too clunky), neutral, and positive views. 7 participants stopped wearing the monitor before discharge.	
Ortiz et al.39	Device adherence (days worn), predictors of adherence	Smart ring (Oura Ring)	Accelerometer, gyroscope, infrared optical pulse measurement (varying study length; mean 229.4 days, SD = 168)	87 adults with bipolar disorder	Participants wore the ring on 79.5% of days. Being in the “perfect adherence” group was associated with female gender and a history of suicide attempt (attempt: 11/21 (52.4%) vs. no attempt: 16/66 (24.2%,), p < 0.05)	
Porras-Segovia et al.45	Recruitment rates, retention rates	Smartphone app (eB²), EMA	Accelerometer, gyroscope, GPS, Bluetooth, Wi-Fi, calls, app usage, screentime (60 days)	139 patients with history of STB; 1224 outpatients without STB; 346 psychology students	Recruitment rates were 87% in patients with history of STB and 85% in patients without STB. Rates of installed eB² app were 74% and 72% in patients with and without history of STB. Retention rates of eB² were >65% in all groups.	
Porras-Segovia et al.44	Participation and retention rates, dropout from app, satisfaction survey	Smartphone app (eB²), EMA	Accelerometer, gyroscope, GPS, Bluetooth, Wi-Fi, calls, app usage, screentime (180 days)	393 adult outpatients	Participation rate of 80%; the retention rate for eB2 was 87.8% after 1 month and 46.6% after six months. Mean satisfaction with eB2: 7/10. Most valued aspect was the control of physical activity (14%); 23% reported technical issues.	
Tlachac et al.37	Willingness to share optional passive data vs. active data	Smartphone app: (EMU); for sensing and active data	Text messages, calendar, call log, GPS, usernames for Twitter, Instagram, Google Maps (one session, retrospective data)	70 participants recruited via Mechanical Turk	Participants were unlikely to share passive (compared with active data): 31 (44.3%) text messages, 28 (40.0%) calendar, 25 (35.7%) call logs, 14 (20.0%) GPS, 14 (20.0%) Twitter name, 0 Instagram.	
Tlachac et al.38	Willingness to share optional passive data for depression and suicidal ideation screening	Smartphone app: (EMU); for sensing and active data + similar web app	Text logs, call logs, contacts, calendar entries, GPS history (one session, retrospective data)	302 tertiary students (33 opted for the smartphone app and only they were asked for passive data)	Of 33 students opting for the smartphone app, 21 (63.6%) shared GPS, 11 (33.3%) calendar/contacts, and 10 (30.3%) call/text logs. Of 171 students who reached the Twitter page, 47 (27.5%) had an account, which 16/47 (34.0%) were willing to share.	
EDA electrodermal activity, GPS Global Positioning System, SI suicidal ideation.

Main findings of feasibility studies

Several studies investigated the user experience or satisfaction with passive sensing technology. Adolescent inpatients with acute suicidality rated discomfort in wearing a wrist device (Empatica E4), scoring below 5/10 in most items of the Wearable Computer Comfort Rating Scale43. Qualitative findings revealed complaints about the device (e.g., too clunky), neutral responses, and positive views (e.g., liking to help research)43. In a sample of adolescents who wore a wrist device (Actiwatch) after discharge from acute psychiatric care for suicide risk, 71% found wearing the device comfortable; clinicians also reported that the study participation (that additionally involved EMA) had a neutral to positive impact on the patients42. In a study by Jiang and collegues40 with young adult suicidal patients, 92.6% found the Fitbit comfortable, 18.1% reported that it interfered with daily activities, and 85.1% stated that they would wear it outside the research context. The satisfaction with wearing it was rated 3.89 (SD = 1.19) on a 0–5 scale. The most frequent barriers were water-related activities (45.7%), forgetting to wear it (39.4%), and other activities (20.2%). Another study found that the majority of adult inpatients provided positive qualitative feedback on the use of a Fitbit41. A study that investigated a passive sensing smartphone application in adult outpatients found a mean satisfaction with the app of 7/1044. In addition, the most frequently reported positive aspect of the app was that it helped control physical activity; the most frequent negative aspect was technical problems44.

Regarding adherence to passive sensing technology, two studies on adolescents with STB found that they wore a wrist device on more than 75% of days42,43. A total of 7 out of 50 participants stopped wearing the device before discharge43. The study by Jiang and collegues with young adult suicidal patients recruited via emergency departments40 found that in an 8-week monitoring phase, a wrist device was worn on 65.9% of days, and in 53.6% of minutes. Adherence decreased during the monitoring period, with 37.3% of participants stopping to wear the device before the end of the study. In a study with adult bipolar patients39, a smart ring was worn on 79.5% of days; the likelihood for perfect adherence was higher in patients with a history of suicide attempt. A study with an adult outpatient sample44 and a study with adult patients and university students45 investigated how many participants kept a passive sensing smartphone app installed and did not withdraw from the study. They found retention rates of >65% across all groups after two months45, 74% after three months, and 47% after 6 months44.

In a sample with tertiary students (n = 302)38, only 33 (10.9%) opted to share data for depression and suicidal ideation screening via a smartphone app as opposed to a web app and were asked to share optional passive data via the app. Of these, 21 (63.6%) shared GPS, 11 (33.3%) calendar/contacts, and 10 (30.3%) call/text logs. Of 171 students who reached the Twitter page, 47 (27.5%) had an account, which 16/47 (34.0%) were willing to share. In a sample recruited online via Mechanical Turk (n = 70)37, participants were more willing to share active compared to passive smartphone data. Active data included questionnaires for depression and anxiety screening, demographic data, scripted and unscripted audio, and sharing rates ranged from 78.6% to 100% for these modalities. In contrast, a total of 31 (44.3%) shared text messages, 28 (40.0%) calendar, 25 (35.7%) call logs, 14 (20.0%) GPS, 14 (20.0%) Twitter name, 0 (0%) Instagram. In an online survey study by Dogrucu and colleagues28, where the sample was intended to be representative of the general population, participants reported a higher willingness to share active data (e.g., a brief voice sample) compared with passive data (e.g., browser history). A study with adult inpatients found that beyond compulsory passive data collection using Fitbit, 80% of participants did not provide optional access to their Facebook data41; the majority of reported reasons for opting out were not having an account (56%) or currently not using it (21%).

Upcoming studies

In total, seven study protocols planning to investigate the predictive validity of passive sensing for STB were identified (see Supplementary Table 3 for a detailed overview)46–52. All planned studies have a longitudinal prospective design. They will all investigate behavioral signals, while two will also include physiological signals48,49 and three will examine social signals47,48,51. Most studies47–52 plan to investigate sleep characteristics; four studies will additionally assess other variables such as physical activity47–49,51. Sels and colleagues provided detailed information about the sensors used and the collection frequency of data for each sensor in their protocol51. Brown and colleagues49 reported that data will be available at a one-minute resolution (i.e., physical activity, sleep, heart rate) and Victor and colleagues52 reported a sampling frequency of approximately 30hz (i.e., accelerometers to derive sleep data). The other protocols did not provide detailed information about the frequency and method of aggregating the data. The majority of study protocols designs reported an EMA assessment of STB47–52. Data collection is planned between 14 days50 and 60 months47. One study reported non-clinical recruitment at schools46; all other studies plan to recruit participants from clinical settings. Four studies will only include patients with suicidal ideation or attempts47,49,51,52.

Discussion

This is the first systematic review to address the predictive validity and feasibility of passive sensing for the prediction of STB. Our results indicate that the current evidence base does not permit conclusions about the value of passive sensing modalities for predicting STB. Prediction studies in this nascent field of research are highly heterogeneous in the selection of predictors, samples, modeling strategies, and outcomes. Furthermore, due to the explorative nature of the research so far, there are multiple sources of potential bias and shortcomings in the reporting of methods and results. Thus, the predictive value of passive sensing data remains inconclusive. Nevertheless, preliminary results on models incorporating self-reported outcomes with passive sensing data are broadly in line with machine learning approaches using other data modalities55. Early feasibility studies suggest that passive sensing might be feasible in clinical populations, in terms of user experience and adherence in terms of wearing devices or keeping mobile apps. Moving forward, we propose several avenues for future research to further explore the potential of passive sensing in the prediction of STB.

The prediction studies identified in this review indicated that passive sensing may have no incremental value over active data in predicting STB. Out of four studies investigating the incremental predictive value of passive data over self-reported data, only one study—a study that performed short-term prediction of suicidal ideation based on EMA – found an incremental value of passive sensing (i.e., electrodermal activity) in the prediction of suicidal ideation severity in suicidal inpatients32. However, physiological data worsened the model fit for the occurrence of suicidal ideation32. Another study predicting the short-term presence or absence of suicidal ideation in suicidal patients did not find an incremental value of passive data over EMA35; two other studies did not identify an incremental value of physiological data over daily mood assessments in the prediction of STB at three-month follow-up in medical interns30,31. Algorithms based only on passive modalities did predict STB32,34,35, usually with a lower model performance compared with active data only32,35. Notably, in one study, events including suicide attempts and emergency department visits were predicted in the short-term with a good model performance using activity profiles from passive smartphone sensing34. Other studies found correlations of post-test STB with specific app usage in adolescents27, sleep characteristics26, and the high-frequency component in heart rate variability33. There was a high risk of bias overall, with severe shortcomings in the study design and transparency in reporting.

Nevertheless, preliminary feasibility studies suggest that passive sensing for STB is feasible in high-risk populations. Patients with STB and/or mental disorders reported moderate to high levels of comfort while wearing wrist devices40,42,43 and using a sensing app44, with sufficient adherence to these devices, including a smart ring39,40,42,43. However, participants from the general population seem to be more willing to share active data compared to passive data28,37,41. Potentially, individuals might be reluctant to share passive data under some circumstances such as a non-clinical setting or the possibility to opt out of specific passive data.

The findings of this review hold important implications for future research. First and foremost, studies should address general methodological limitations by adhering to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist56 and by avoiding potential sources of bias according to the PROBAST tool24. Studies should rigorously report the following aspects both in pre-registrations and research reports: eligibility criteria and reasons for exclusion from analyses, missing data handling, details regarding the selection of predictors and feature extraction algorithms, pre-processing steps of passive sensing data, a detailed list of sensors, the sampling rate and method for each sensor, as well as the model performance. Concerning the sampling strategy, studies should select participants with the aim of achieving a balanced sample containing a substantial proportion of participants or time points with and without the outcome of STB, which is necessary for model development and comparison to a control/naive predictor24. Authors should report how they dealt with potential class imbalance in classification tasks. In addition, studies should avoid choosing arbitrary cut-offs for differentiating participants into high- and low-risk24. Finally, it is crucial to evaluate models using external data24.

Second, it is crucial to prioritize the valid and reliable assessment of STB. Single-item measures, as frequently used in the reviewed studies, may not have sufficient psychometric properties53. In addition, suicidal ideation and suicidal behaviors should be differentiated (e.g., the subscales of the CSSR-S) instead of using a composite score including both of them, as they are likely to have different risk trajectories57. Furthermore, most studies have assessed STB at single time points. Given the high fluctuations in STB8, it might be fruitful to additionally investigate the short-term prediction by assessing these outcomes using EMA5.

Third, research on passive sensing in the context of STB should consider different application settings and target groups. Target groups identified in this review included acutely suicidal patients32,33, patients with a history of STB34, groups at risk of developing STB (i.e., people in mental health treatment26,29,36 or medical interns30,31), as well as the general population27,28. Each application area presents unique challenges. The short-term prediction of STB within individuals with acute STB seems promising both clinically and methodologically. Clinically, intensive monitoring is desirable in this vulnerable target group, while methodologically, a high frequency of time points with STB is necessary for developing predictive algorithms. Developing algorithms for predicting suicide attempts might however prove challenging because of the low rates of occurrence of this outcome and the low signal-to-noise ratio of passive sensing data. Transfer-learning on models initially developed for a different but closely related source domain58 might be one of the possible workarounds to this methodological hurdle. Another application area of passive sensing could be universal screening for STB; again, studies should address sample imbalance in samples from the general population. Another aspect to consider in the application setting is the use of passive sensing variables as the sole predictors versus combining them with EMA data for predicting STB. Passive data alone might be especially useful for longer monitoring periods due to the low participant effort, but it remains unclear whether passive data alone has sufficient predictive value. Combining EMA and passive sensing holds promise for the longitudinal prediction of STB, making use of the advantages of both assessment strategies, including additional data such as nonresponse59 or response latency to questionnaire items60. Regarding the choice of devices for assessing predictors, it is important to consider their real-world applicability in different settings. Participants from the general population seem more willing to share actively assessed data compared to passive smartphone usage data28,37,41; however, this might not generalize to indicated or selective prevention, such as risk monitoring in individuals with acute suicidal ideation or a recent suicide attempt. Furthermore, the applicability of passive monitoring may be influenced by the availability of devices (i.e., higher availability of smartphones compared with wearables), requiring a balance between data quality and scalability. It is crucial to further consider the applicability of passive monitoring for STB beyond research settings, particularly with regards to how an individual’s awareness of STB monitoring via passive devices could influence behaviors in ways that would affect the reliability of risk detection in real-world settings.

Several limitations of this review should be considered. First, in line with the protocol, the search string aimed to identify prediction studies and might have missed some relevant feasibility studies. Second, the heterogeneity of reviewed papers did not allow a meta-analytic pooling of results. Third, the current results on social signals are limited to quantitative features, such as number of calls or social media application usage. However, contents of social media posts might be highly relevant for the prediction of STB, as indicated in language processing models61–63.

In conclusion, the field of passive sensing of STB is currently in its infancy, with the first studies published in recent years. The selection of predictors has been largely explorative, and existing studies – whilst pioneering in their own right - have major methodological shortcomings consistent with first-in-field research. The identified study protocols indicate that studies conducted in the following years may be designed with further rigor to hopefully reveal whether passive data is suitable to predict STB. Future studies should rigorously follow reporting guidelines and pre-register their procedures. Alongside this, the clinical implications of applying such approaches in real-world settings should be considered, to develop tools with maximal utility for helping to detect and ultimately prevent STB among vulnerable populations.

Supplementary information

Supplementary Information

Supplementary information

The online version contains supplementary material available at 10.1038/s44184-024-00089-4.

Acknowledgements

We thank Annika Hartmann for helping with the manuscript preparation. In addition, we thank Mascha von der Lippe, Abrama E. Nikrant, and Fatma Dilvin Abaci for supporting the study team with the search update, including screening, data extractions, and data checks. There was no external funding for this study.

Author contributions

L.B.S. initialized the study and supervised the procedures. R.B., T.W., M.E.L., J.T., J.F., and L.B.S. designed the study methods. T.W., S.K., M.L., A.E.N., F.D.A., and R.B. screened the records for eligibility; any conflicts were resolved in discussion with L.B.S.; Data extractions were conducted by T.W., S.H., J.M., M.L., A.E.N., and N.J.; any conflicts were resolved by R.B.; T.W., J.M., R.B., and N.J. conducted the risk of bias ratings; any conflicts were resolved in discussion with R.B. and L.B.S.; M.C. and W.V.B. provided input for the interpretation of performance metrics. R.B. and T.W. drafted the paper. All authors critically revised the paper.

Funding

Open Access funding enabled and organized by Projekt DEAL.

Data availability

The extracted data will be available via OSF with publication (https://osf.io/psk85/files/).

Competing interests

J.T. is a scientific advisor of Precision Mental Wellness. J.F. is supported by a UK Research and Innovation Future Leaders Fellowship (MR/T021780/1) and has received honoraria/consultancy fees from Atheneum, Informa, Gillian Kenny Associates, Bayer, Big Health, Hedonia, Strive Coaching, Wood For Trees, Nutritional Medicine Institute, Angelini, ParachuteBH, Richmond Foundation and Nirakara, independent of this work. All other authors declare no conflicts of interest.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. World Health Organization. Suicide. Available at https://www.who.int/news-room/fact-sheets/detail/suicide (2021).
2. U.S.A. suicide: 2022 Official final data. Available at https://www.thencsp.org/_files/ugd/a0415f_e503f577e700439e8cc9f33771d813af.pdf (2024).
3. Joseph VA Temporal trends in suicide methods among adolescents in the US JAMA Netw. Open 2022 5 e2236049 10.1001/jamanetworkopen.2022.36049 36223121
Joseph, V. A. et al. Temporal trends in suicide methods among adolescents in the US. JAMA Netw. Open 5, e2236049 (2022).36223121
4. Nock MK Cross-national prevalence and risk factors for suicidal ideation, plans and attempts Br. J. Psychiatry. J. Ment. Sci. 2008 192 98 105 10.1192/bjp.bp.107.040113
Nock, M. K. et al. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br. J. Psychiatry. J. Ment. Sci. 192, 98–105 (2008).
5. Franklin JC Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research Psychol. Bull. 2017 143 187 232 10.1037/bul0000084 27841450
Franklin, J. C. et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol. Bull. 143, 187–232 (2017).27841450
6. Turecki G Suicide and suicide risk Nat. Rev. Dis. Prim. 2019 5 74 10.1038/s41572-019-0121-0 31649257
Turecki, G. et al. Suicide and suicide risk. Nat. Rev. Dis. Prim. 5, 74 (2019).31649257
7. Kleiman EM Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies J. Abnorm. Psychol. 2017 126 726 738 10.1037/abn0000273 28481571
Kleiman, E. M. et al. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J. Abnorm. Psychol. 126, 726–738 (2017).28481571
8. Kivelä L van der Does WAJ Riese H Antypa N Don’t miss the moment: a systematic review of ecological momentary assessment in suicide research Front. Digit. Health 2022 4 876595 10.3389/fdgth.2022.876595 35601888
Kivelä, L., van der Does, W. A. J., Riese, H. & Antypa, N. Don’t miss the moment: a systematic review of ecological momentary assessment in suicide research. Front. Digit. Health 4, 876595 (2022).35601888
9. Millner AJ Lee MD Nock MK Describing and measuring the pathway to suicide attempts: a preliminary study Suicide Life Threat. Behav. 2017 47 353 369 10.1111/sltb.12284 27477787
Millner, A. J., Lee, M. D. & Nock, M. K. Describing and measuring the pathway to suicide attempts: a preliminary study. Suicide Life Threat. Behav. 47, 353–369 (2017).27477787
10. Hawton K Pirkis J Suicide is a complex problem that requires a range of prevention initiatives and methods of evaluation Br. J. Psychiatry 2017 210 381 383 10.1192/bjp.bp.116.197459 28572430
Hawton, K. & Pirkis, J. Suicide is a complex problem that requires a range of prevention initiatives and methods of evaluation. Br. J. Psychiatry 210, 381–383 (2017).28572430
11. Beurs Dde A network perspective on suicidal behavior: understanding suicidality as a complex system Suicide Life Threat. Behav. 2021 51 115 126 10.1111/sltb.12676 33624872
Beurs, Dde et al. A network perspective on suicidal behavior: understanding suicidality as a complex system. Suicide Life Threat. Behav. 51, 115–126 (2021).33624872
12. Bryan CJ Nonlinear change processes and the emergence of suicidal behavior: a conceptual model based on the fluid vulnerability theory of suicide New Ideas Psychol. 2020 57 100758 10.1016/j.newideapsych.2019.100758
Bryan, C. J. et al. Nonlinear change processes and the emergence of suicidal behavior: a conceptual model based on the fluid vulnerability theory of suicide. New Ideas Psychol. 57, 100758 (2020).
13. Woodford R Accuracy of clinician predictions of future self-harm: a systematic review and meta-analysis of predictive studies Suicide Life Threat. Behav. 2019 49 23 40 10.1111/sltb.12395 28972271
Woodford, R. et al. Accuracy of clinician predictions of future self-harm: a systematic review and meta-analysis of predictive studies. Suicide Life Threat. Behav. 49, 23–40 (2019).28972271
14. Torous J The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality World Psychiatry. Off. J. World Psychiatr. Assoc. (WPA) 2021 20 318 335
Torous, J. et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry. Off. J. World Psychiatr. Assoc. (WPA) 20, 318–335 (2021).
15. Shiffman S Stone AA Hufford MR Ecological momentary assessment Annu. Rev. Clin. Psychol. 2008 4 1 32 10.1146/annurev.clinpsy.3.022806.091415 18509902
Shiffman, S., Stone, A. A. & Hufford, M. R. Ecological momentary assessment. Annu. Rev. Clin. Psychol. 4, 1–32 (2008).18509902
16. Baumeister, H. & Montag, C. Digital Phenotyping and Mobile Sensing (Springer International Publishing, 2019).
17. Huckvale K Venkatesh S Christensen H Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety NPJ Digit. Med. 2019 2 88 10.1038/s41746-019-0166-1 31508498
Huckvale, K., Venkatesh, S. & Christensen, H. Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety. NPJ Digit. Med. 2, 88 (2019).31508498
18. Abdullah S Choudhury T Sensing technologies for monitoring serious mental illnesses IEEE Multimed. 2018 25 61 75 10.1109/MMUL.2018.011921236
Abdullah, S. & Choudhury, T. Sensing technologies for monitoring serious mental illnesses. IEEE Multimed. 25, 61–75 (2018).
19. Allen NB Nelson BW Brent D Auerbach RP Short-term prediction of suicidal thoughts and behaviors in adolescents: Can recent developments in technology and computational science provide a breakthrough? J. Affect. Disord. 2019 250 163 169 10.1016/j.jad.2019.03.044 30856493
Allen, N. B., Nelson, B. W., Brent, D. & Auerbach, R. P. Short-term prediction of suicidal thoughts and behaviors in adolescents: Can recent developments in technology and computational science provide a breakthrough? J. Affect. Disord. 250, 163–169 (2019).30856493
20. Ballard ED Gilbert JR Wusinich C Zarate CA New methods for assessing rapid changes in suicide risk Front. Psychiatry 2021 12 598434 10.3389/fpsyt.2021.598434 33574775
Ballard, E. D., Gilbert, J. R., Wusinich, C. & Zarate, C. A. New methods for assessing rapid changes in suicide risk. Front. Psychiatry 12, 598434 (2021).33574775
21. Torous J Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: A review of current progress and next steps Curr. Psychiatry Rep. 2018 20 51 10.1007/s11920-018-0914-y 29956120
Torous, J. et al. Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: A review of current progress and next steps. Curr. Psychiatry Rep. 20, 51 (2018).29956120
22. Moher D Liberati A Tetzlaff J Altman DG Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement PLoS Med. 2009 6 e1000097 10.1371/journal.pmed.1000097 19621072
Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. G. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6, e1000097 (2009).19621072
23. Winkler T Passive sensing in the prediction of suicidal thoughts and behaviors: protocol for a systematic review JMIR Res. Protoc. 2022 11 e42146 10.2196/42146 36445737
Winkler, T. et al. Passive sensing in the prediction of suicidal thoughts and behaviors: protocol for a systematic review. JMIR Res. Protoc. 11, e42146 (2022).36445737
24. Moons KG PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration Ann. Intern Med 2019 170 W1 10.7326/M18-1377 30596876
Moons, K. G. et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann. Intern Med 170, W1 (2019).30596876
25. Ciharova, M. et al. Use of machine learning algorithms based on text, audio, and video data in the prediction of anxiety and posttraumatic stress in general and clinical populations: a systematic review. Biol. Psychiatry10.1016/j.biopsych.2024.06.002 (2024).
26. Bertrand L Bourguignon C Beaulieu S Storch KF Linnaranta O Suicidal ideation and insomnia in bipolar disorders Can. J. Psychiatry 2020 65 802 810 10.1177/0706743720952226 32856463
Bertrand, L., Bourguignon, C., Beaulieu, S., Storch, K. F. & Linnaranta, O. Suicidal ideation and insomnia in bipolar disorders. Can. J. Psychiatry 65, 802–810 (2020).32856463
27. Coyne SM Suicide risk in emerging adulthood: associations with screen time over 10 years J. Youth Adolesc. 2021 50 2324 2338 10.1007/s10964-020-01389-6 33528704
Coyne, S. M. et al. Suicide risk in emerging adulthood: associations with screen time over 10 years. J. Youth Adolesc. 50, 2324–2338 (2021).33528704
28. Dogrucu A Moodable: on feasibility of instantaneous depression assessment using machine learning on voice samples with retrospectively harvested smartphone and social media data Smart Health 2020 17 100118 10.1016/j.smhl.2020.100118
Dogrucu, A. et al. Moodable: on feasibility of instantaneous depression assessment using machine learning on voice samples with retrospectively harvested smartphone and social media data. Smart Health 17, 100118 (2020).
29. Haines-Delmont A Testing suicide risk prediction algorithms using phone measurements with patients in acute mental health settings: feasibility study JMIR mHealth uHealth 2020 8 e15901 10.2196/15901 32442152
Haines-Delmont, A. et al. Testing suicide risk prediction algorithms using phone measurements with patients in acute mental health settings: feasibility study. JMIR mHealth uHealth 8, e15901 (2020).32442152
30. Horwitz A Utilizing daily mood diaries and wearable sensor data to predict depression and suicidal ideation among medical interns J. Affect. Disord. 2022 313 1 7 10.1016/j.jad.2022.06.064 35764227
Horwitz, A. et al. Utilizing daily mood diaries and wearable sensor data to predict depression and suicidal ideation among medical interns. J. Affect. Disord. 313, 1–7 (2022).35764227
31. Horwitz AG Using machine learning with intensive longitudinal data to predict depression and suicidal ideation among medical interns over time Psychol. Med. 2022 53 5778 5785 10.1017/S0033291722003014 36177889
Horwitz, A. G. et al. Using machine learning with intensive longitudinal data to predict depression and suicidal ideation among medical interns over time. Psychol. Med. 53, 5778–5785 (2022).36177889
32. Kleiman EM Can passive measurement of physiological distress help better predict suicidal thinking? Transl. Psychiatry 2021 11 611 10.1038/s41398-021-01730-y 34857731
Kleiman, E. M. et al. Can passive measurement of physiological distress help better predict suicidal thinking? Transl. Psychiatry 11, 611 (2021).34857731
33. Sheridan DC Heart rate variability and its ability to detect worsening suicidality in adolescents: A pilot trial of wearable technology Psychiatry Investig. 2021 18 928 935 10.30773/pi.2021.0057 34555890
Sheridan, D. C. et al. Heart rate variability and its ability to detect worsening suicidality in adolescents: A pilot trial of wearable technology. Psychiatry Investig. 18, 928–935 (2021).34555890
34. Barrigon ML One-week suicide risk prediction using real-time smartphone monitoring: prospective cohort study J. Med. Internet Res. 2023 25 e43719 10.2196/43719 37656498
Barrigon, M. L. et al. One-week suicide risk prediction using real-time smartphone monitoring: prospective cohort study. J. Med. Internet Res. 25, e43719 (2023).37656498
35. Czyz EK Ecological momentary assessments and passive sensing in the prediction of short-term suicidal ideation in young adults JAMA Netw. open 2023 6 e2328005 10.1001/jamanetworkopen.2023.28005 37552477
Czyz, E. K. et al. Ecological momentary assessments and passive sensing in the prediction of short-term suicidal ideation in young adults. JAMA Netw. open 6, e2328005 (2023).37552477
36. Salvatore P Circadian activity rhythms and psychopathology in major depressive episodes Psychopathology 2024 57 1 9 10.1159/000530768 37499644
Salvatore, P. et al. Circadian activity rhythms and psychopathology in major depressive episodes. Psychopathology 57, 1–9 (2024).37499644
37. Tlachac, M. L. et al. EMU: early mental health uncovering framework and dataset. 20th IEEE international conference on machine learning and applications (ICMLA), 1311–1318, https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9680143 (2021).
38. Tlachac ML StudentSADD: rapid mobile depression and suicidal ideation screening of college students during the coronavirus pandemic Proc. ACM Interact., Mob. Wearable Ubiquitous Technol. 2022 6 1 32
Tlachac, M. L. et al. StudentSADD: rapid mobile depression and suicidal ideation screening of college students during the coronavirus pandemic. Proc. ACM Interact., Mob. Wearable Ubiquitous Technol. 6, 1–32 (2022).
39. Ortiz A Predictors of adherence to electronic self-monitoring in patients with bipolar disorder: a contactless study using Growth Mixture Models Int. J. Bipolar Disord. 2023 11 18 10.1186/s40345-023-00297-5 37195477
Ortiz, A. et al. Predictors of adherence to electronic self-monitoring in patients with bipolar disorder: a contactless study using Growth Mixture Models. Int. J. Bipolar Disord. 11, 18 (2023).37195477
40. Jiang A Acceptability and feasibility of ecological momentary assessment with augmentation of passive sensor data in young adults at high risk for suicide Psychiatry Res. 2023 326 115347 10.1016/j.psychres.2023.115347 37487460
Jiang, A. et al. Acceptability and feasibility of ecological momentary assessment with augmentation of passive sensor data in young adults at high risk for suicide. Psychiatry Res. 326, 115347 (2023).37487460
41. Bruen AJ Wall A Haines-Delmont A Perkins E Exploring suicidal ideation using an innovative mobile app-Strength Within Me: The usability and acceptability of setting up a trial involving mobile technology and mental health service users JMIR Ment. Health 2020 7 e18407 10.2196/18407 32985995
Bruen, A. J., Wall, A., Haines-Delmont, A. & Perkins, E. Exploring suicidal ideation using an innovative mobile app-Strength Within Me: The usability and acceptability of setting up a trial involving mobile technology and mental health service users. JMIR Ment. Health 7, e18407 (2020).32985995
42. Glenn CR Feasibility and acceptability of ecological momentary assessment with high-risk suicidal adolescents following acute psychiatric care J. Clin. Child Adolesc. Psychol. 2022 51 32 48 10.1080/15374416.2020.1741377 32239986
Glenn, C. R. et al. Feasibility and acceptability of ecological momentary assessment with high-risk suicidal adolescents following acute psychiatric care. J. Clin. Child Adolesc. Psychol. 51, 32–48 (2022).32239986
43. Kleiman E Using wearable physiological monitors with suicidal adolescent inpatients: feasibility and acceptability study JMIR mHealth uHealth 2019 0 e0 31586364
Kleiman, E. et al. Using wearable physiological monitors with suicidal adolescent inpatients: feasibility and acceptability study. JMIR mHealth uHealth 0, e0 (2019).31586364
44. Porras-Segovia A Real-world feasibility and acceptability of real-time suicide risk monitoring via smartphones: a 6-month follow-up cohort J. Psychiatr. Res. 2022 149 145 154 10.1016/j.jpsychires.2022.02.026 35276631
Porras-Segovia, A. et al. Real-world feasibility and acceptability of real-time suicide risk monitoring via smartphones: a 6-month follow-up cohort. J. Psychiatr. Res. 149, 145–154 (2022).35276631
45. Porras-Segovia A Smartphone-based ecological momentary assessment (EMA) in psychiatric patients and student controls: a real-world feasibility study J. Affect. Disord. 2020 274 733 741 10.1016/j.jad.2020.05.067 32664009
Porras-Segovia, A. et al. Smartphone-based ecological momentary assessment (EMA) in psychiatric patients and student controls: a real-world feasibility study. J. Affect. Disord. 274, 733–741 (2020).32664009
46. Werner‐Seidler, A. et al. The future proofing study: design, methods and baseline characteristics of a prospective cohort study of the mental health of Australian adolescents. Int. J. Methods Psychiatric Res.10.1002/mpr.1954 (2022).
47. Barzilay S Real-time real-world digital monitoring of adolescent suicide risk during the six months following emergency department discharge: protocol for an intensive longitudinal study JMIR Res. Protoc. 2023 12 e46464 10.2196/46464 37358906
Barzilay, S. et al. Real-time real-world digital monitoring of adolescent suicide risk during the six months following emergency department discharge: protocol for an intensive longitudinal study. JMIR Res. Protoc. 12, e46464 (2023).37358906
48. Berrouiguet S Combining mobile-health (mHealth) and artificial intelligence (AI) methods to avoid suicide attempts: the Smartcrises study protocol BMC Psychiatry 2019 19 277 10.1186/s12888-019-2260-y 31493783
Berrouiguet, S. et al. Combining mobile-health (mHealth) and artificial intelligence (AI) methods to avoid suicide attempts: the Smartcrises study protocol. BMC Psychiatry 19, 277 (2019).31493783
49. Brown LA Digital phenotyping to improve prediction of suicidal urges in treatment: study protocol Aggress. Violent Behav. 2022 66 101733 10.1016/j.avb.2022.101733
Brown, L. A. et al. Digital phenotyping to improve prediction of suicidal urges in treatment: study protocol. Aggress. Violent Behav. 66, 101733 (2022).
50. Dewa LH The association between sleep disturbance and suicidality in psychiatric inpatients transitioning to the community: Protocol for an ecological momentary assessment study JMIR Res. Protoc. 2022 11 e33817 10.2196/33817 35579920
Dewa, L. H. et al. The association between sleep disturbance and suicidality in psychiatric inpatients transitioning to the community: Protocol for an ecological momentary assessment study. JMIR Res. Protoc. 11, e33817 (2022).35579920
51. Sels L SIMON: a digital protocol to monitor and predict suicidal ideation Front. Psychiatry 2021 12 554811 10.3389/fpsyt.2021.554811 34276427
Sels, L. et al. SIMON: a digital protocol to monitor and predict suicidal ideation. Front. Psychiatry 12, 554811 (2021).34276427
52. Victor SE Christensen K Johnson SL van Allen J Brick LA Dynamic regulatory processes in the transition from suicidal ideation to action in adults leaving inpatient psychiatric care: Protocol for an intensive longitudinal study JMIR Res. Protoc. 2022 11 e38582 10.2196/38582 35771618
Victor, S. E., Christensen, K., Johnson, S. L., van Allen, J. & Brick, L. A. Dynamic regulatory processes in the transition from suicidal ideation to action in adults leaving inpatient psychiatric care: Protocol for an intensive longitudinal study. JMIR Res. Protoc. 11, e38582 (2022).35771618
53. Na PJ The PHQ-9 item 9 based screening for suicide risk: a validation study of the Patient Health Questionnaire (PHQ)-9 Item 9 with the Columbia Suicide Severity Rating Scale (C-SSRS) J. Affect. Disord. 2018 232 34 40 10.1016/j.jad.2018.02.045 29477096
Na, P. J. et al. The PHQ-9 item 9 based screening for suicide risk: a validation study of the Patient Health Questionnaire (PHQ)-9 Item 9 with the Columbia Suicide Severity Rating Scale (C-SSRS). J. Affect. Disord. 232, 34–40 (2018).29477096
54. Jacobson NS Truax P Clinical significance: a statistical approach to defining meaningful change in psychotherapy research J. Consult. Clin. Psychol. 1991 59 12 19 10.1037/0022-006X.59.1.12 2002127
Jacobson, N. S. & Truax, P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 59, 12–19 (1991).2002127
55. Kusuma K The performance of machine learning models in predicting suicidal ideation, attempts, and deaths: a meta-analysis and systematic review J. Psychiatr. Res. 2022 155 579 588 10.1016/j.jpsychires.2022.09.050 36206602
Kusuma, K. et al. The performance of machine learning models in predicting suicidal ideation, attempts, and deaths: a meta-analysis and systematic review. J. Psychiatr. Res. 155, 579–588 (2022).36206602
56. Collins GS Reitsma JB Altman DG Moons KGM Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement BMJ (Clin. Res. ed.) 2015 350 g7594
Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. M. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ (Clin. Res. ed.) 350, g7594 (2015).
57. Klonsky ED Saffer BY Bryan CJ Ideation-to-action theories of suicide: a conceptual and empirical update Curr. Opin. Psychol. 2018 22 38 43 10.1016/j.copsyc.2017.07.020 30122276
Klonsky, E. D., Saffer, B. Y. & Bryan, C. J. Ideation-to-action theories of suicide: a conceptual and empirical update. Curr. Opin. Psychol. 22, 38–43 (2018).30122276
58. Al-Stouhi S Reddy CK Transfer learning for class imbalance problems with inadequate data Knowl. Inf. Syst. 2016 48 201 228 10.1007/s10115-015-0870-3 27378821
Al-Stouhi, S. & Reddy, C. K. Transfer learning for class imbalance problems with inadequate data. Knowl. Inf. Syst. 48, 201–228 (2016).27378821
59. Stanley IH Marx BP Nichter B Pietrzak RH Non-response to questions about suicide ideation and attempts among veterans: Results from the National Health and Resilience in Veterans Study Suicide Life Threat. Behav. 2022 52 763 772 10.1111/sltb.12860 35362636
Stanley, I. H., Marx, B. P., Nichter, B. & Pietrzak, R. H. Non-response to questions about suicide ideation and attempts among veterans: Results from the National Health and Resilience in Veterans Study. Suicide Life Threat. Behav. 52, 763–772 (2022).35362636
60. Henson P Torous J Feasibility and correlations of smartphone meta-data toward dynamic understanding of depression and suicide risk in schizophrenia Int. J. Methods Psychiatr. Res. 2020 29 e1825 10.1002/mpr.1825 32333471
Henson, P. & Torous, J. Feasibility and correlations of smartphone meta-data toward dynamic understanding of depression and suicide risk in schizophrenia. Int. J. Methods Psychiatr. Res. 29, e1825 (2020).32333471
61. Lekkas D Klein RJ Jacobson NC Predicting acute suicidal ideation on Instagram using ensemble machine learning models Internet Interv. 2021 25 100424 10.1016/j.invent.2021.100424 34401383
Lekkas, D., Klein, R. J. & Jacobson, N. C. Predicting acute suicidal ideation on Instagram using ensemble machine learning models. Internet Interv. 25, 100424 (2021).34401383
62. Glenn JJ Nobles AL Barnes LE Teachman BA Can text messages identify suicide risk in real time? A within-subjects pilot examination of temporally sensitive markers of suicide risk Clin. Psychol. Sci. 2020 8 704 722 10.1177/2167702620906146 35692890
Glenn, J. J., Nobles, A. L., Barnes, L. E. & Teachman, B. A. Can text messages identify suicide risk in real time? A within-subjects pilot examination of temporally sensitive markers of suicide risk. Clin. Psychol. Sci. 8, 704–722 (2020).35692890
63. Diniz EJS Boamente: a natural language processing-based digital phenotyping tool for smart monitoring of suicidal ideation Healthcare 2022 10 698 10.3390/healthcare10040698 35455874
Diniz, E. J. S. et al. Boamente: a natural language processing-based digital phenotyping tool for smart monitoring of suicidal ideation. Healthcare 10, 698 (2022).35455874
