
==== Front
Ann Med
Ann Med
Annals of Medicine
0785-3890
1365-2060
Taylor & Francis

39297306
10.1080/07853890.2024.2405077
2405077
Version of Record
Research Article
Psychiatry
Predicting sleep based on physical activity, light exposure, and Heart rate variability data using wearable devices
K. M. Park et al.
https://orcid.org/0000-0002-2416-2683
Park Kyung Mee ab
Lee Sang Eun c
Lee Changhee d #
Hwang Hyun Duck c
Yoon Do Hoon c
Choi Eunchae b
https://orcid.org/0000-0002-7462-0144
Lee Eun be
a Department of Hospital Medicine, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
b Department of Psychiatry, Institute of Behavioral Science in Medicine, and Institute for Innovation in Digital Healthcare, Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea
c Health IT center, Yonsei University Health System, Yonsei College of Medicine, Seoul, Republic of Korea
d Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea
e Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Republic of Korea
Supplemental data for this article can be accessed online at https://doi.org/10.1080/07853890.2024.2405077.

# Present Address: Changhee Lee is now affiliated to Korea University, Seoul, Republic of Korea

CONTACT Eun Lee leeeun@yuhs.ac Department of Psychiatry, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea.
19 9 2024
2024
19 9 2024
56 1 24050777 5 2024
29 7 2024
29 8 2024
KnowledgeWorks Global Ltd.18 9 2024
published online in a building issue18 9 2024
© 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
2024
The Author(s)
https://creativecommons.org/licenses/by/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.

Abstract

Objective

We aimed to improve the performance of sleep prediction algorithms by increasing the data amount, adding variables reflecting psychological state, and adjusting the data length.

Materials and Methods

We used ActiGraph GT3X+® and Galaxy Watch Active2™ to collect physical activity and light exposure data. We collected heart rate variability (HRV) data with the Galaxy Watch. We evaluated the performance of sleep prediction algorithms based on different data sources (wearable devices only, sleep diary only, or both), data lengths (1, 2, or 3 days), and analysis methods. We defined the target outcome, ‘good sleep’, as ≥90% sleep efficiency.

Results

Among 278 participants who denied having sleep disturbance, we used data including 2136 total days and nights from 230 participants. The performance of the sleep prediction algorithms improved with an increased amount of data and added HRV data. The model with the best performance was the extreme gradient boosting model; XGBoost, using both sources combined data with HRV, and 2-day data (accuracy=.85, area under the curve =.80).

Conclusions

The results show that the performance of the sleep prediction models improved by increasing the data amount and adding HRV data. Further studies targeting insomnia patients and applied researches on non-pharmacological insomnia treatment are needed.

Keywords

Sleep
sleep prediction
actigraphy
wearable device
machine learning
deep learning
Basic Science Research Program through the National Research Foundation of Korea of the Ministry of Science, ICT & Future Planning, Republic of Korea 2022R1A2B5B03002611 Intelligence Information Expansion Support System for Private Organizations supervised by the National IT Industry Promotion Agency A0602-19-1020 Ministry of Trade, Industry and Energy Grant funded by the Korean government 10060085 This study was funded by the Basic Science Research Program through the National Research Foundation of Korea of the Ministry of Science, ICT & Future Planning, Republic of Korea (Grant number: 2022R1A2B5B03002611 to Eun Lee); the Intelligence Information Expansion Support System for Private Organizations supervised by the National IT Industry Promotion Agency (grant number A0602-19-1020 to Eun Lee); and a Ministry of Trade, Industry and Energy Grant funded by the Korean government (Medical Device R&D Platform Project, project number 10060085 to Sang Eun Lee). There were no other sources of financial support for this study. The funding sources were not involved in study design, data collection, or manuscript preparation.
==== Body
pmcIntroduction

The digitalization of cognitive behavior therapy for insomnia (CBTi) has proven to be an irreversible trend as digital CBTi has become one of the most used digital therapeutics worldwide [1]. However, sleep diaries continue to be essential for prescribing behavioral therapy, including digital CBTi, and determining the treatment’s effectiveness and patient adherence. We previously developed sleep prediction models intending to eliminate the need for keeping sleep diaries by predicting the next sleep using only information from wearable devices [2]. Three other studies have predicted future sleep using data automatically collected from wearable devices or smartphones [3–5]. Two of these used only physical activity data to predict sleep [4,5], and the other used physical activity and light exposure data but predicted subjective sleep quality [3]. In our previous study, we used actigraphy data, including physical activity and light exposure, to predict good sleep, defined as ≥90% sleep efficiency.

Our previously developed sleep prediction model has several limitations. First, even though we collected data from 109 people for 14 consecutive days, the model performance was not excellent due to the relatively small amount of data and significant data loss (38% of the total data) due to battery depletion and non-contact with the body, among other causes. Including sufficient data in prediction model development is known to improve the model’s accuracy [6,7]. In this study, we aimed to solve this problem by using commercially available wearable devices and sending data to the cloud in real time, which could prevent data loss.

Second, sleep predictions based on physical activity and light exposure alone are limited. Existing research suggests that psychological stress also has a significant impact on sleep [8,9]. Although psychological factors are known to have an important impact on sleep, they have not been included in previous models, which may explain the models’ limited performance. Thus, we added a new variable to the sleep prediction model that reflects psychological state, as measured by wearable devices. Heart rate variability (HRV) is a physiological indicator of psychological stress that has gained attention in sleep research [10,11]. As it has not yet been used in sleep prediction models, we decided to investigate whether using HRV as an indicator of psychological state related to sleep would improve the performance of our sleep prediction models.

Finally, we considered the length of the data analyzed for sleep prediction. Three previous studies that used wearable devices to predict sleep developed models based on 1-day data [3–5]. We previously compared a model based on 2-day data with a model based on 1-day data and found that the performance of the 2-day data model was better [2]. Sleep is neither fragmented nor independent on a day-to-day basis. Instead, it exists in a dynamic and interconnected relationship with continuous lifestyle habits and the surrounding environment. Changes in lifestyle habits and the environment can influence sleep over short periods of several days and also over longer spans lasting months [12]. Therefore, 1 or 2 days of data may not be sufficient to predict future sleep. We aimed to determine whether we could improve the performance of our models with longer periods of data.

The purpose of this study was to improve the performance of our sleep prediction models by increasing the amount of data, adding indicators of psychological stress to physical activity and light exposure, and adjusting the data length. We developed several sleep prediction models using different data sources (i.e. exclusively wearable device data, exclusively sleep diary data, and combined data from both sources), analysis methods (machine learning, deep learning, and classical logistic regression), and data lengths (1, 2, and 3 days).

Materials and methods

Participants

Healthy participants who denied having sleep disturbance were enrolled in this study. We recruited participants via advertisements posted at Severance Hospital and Yonsei University College of Medicine. The exclusion criteria were as follows: poorly controlled medical condition (i.e. a condition resulting in a need for hospitalization within the last 3 months), regular use of medication for sleep, any current or previous major psychiatric disorder (e.g. schizophrenia or a related psychotic disorder, bipolar disorder, major depressive disorder, anxiety disorder), and substance use disorder. We determined the presence of psychiatric disorders, including substance use disorder, using the structured clinical interview for DSM-IV [13]. We invited individuals who agreed to participate by providing written informed consent to a screening interview in which the researchers (EC and DY) collected baseline characteristics, including sex, age, height, weight, education level, smoking status, alcohol consumption, caffeine use, and current psychiatric disorder diagnoses. The researchers also obtained baseline sleep-related information from the participants through the Pittsburgh Sleep Quality Index and Insomnia Severity Index [14,15].

Ethical considerations

This study was conducted according to the Declaration of Helsinki principles. The institutional review board of Severance Hospital, Yonsei University Health System, approved this study (nos. 4-2017-0384 and 1-2019-0038). All participants provided written informed consent. All the collected data was anonymized before analysis.

Data collection

Our previous preliminary study was conducted using only the Actigraph [2]. For the current study, we switched to a consumer wearable device for clinical application; however, we had 33 participants (11.9% of the total enrolled population) wear both devices simultaneously to verify the compatibility of data obtained from the two devices (Supplementary Figure 1). We found that the data from the two devices were compatible, thus we used data from both the preliminary study and the current study in our analysis. The ActiGraph GT3X+® is a triaxial accelerometer with a light sensor, that has been validated against polysomnography and other activity monitors in several studies [16,17]. The Galaxy Watch Active2™ (Samsung) can monitor physical activity, light exposure, and HRV. Recent studies on consumer wearable devices, including the Galaxy Watch, have assessed and validated their use for research in medical fields [18–21].

After the screening interview, we asked participants to wear a device (either the ActiGraph or Galaxy Watch) on their non-dominant wrist continuously for 14 days, except while washing or participating in strenuous activities that could damage the device. We exported and assessed the collected data through the ActiLife software (version 6.13.3, ActiGraph) for the ActiGraph or an in-house program for the Galaxy Watch. We gathered data from the ActiGraph after the end of the 14 days of the experiment, whereas the Galaxy Watch sent data to a cloud server in real time.

We used sleep diaries based on the international consensus format to assess in-bed time, sleep onset latency, the number of awakenings after sleep, sleep time, wake time, out-of-bed time, caffeine consumption (defined as >1 cup within 6 h before sleep), alcohol consumption (defined as >10 g within 6 h before sleep), and whether the participant took naps [22]. We encouraged the participants to record the time whenever they removed the wearable device.

Wearable device data processing

We collected physical activity and light exposure data every 10 seconds from both devices. The raw data are generally comparable across accelerometer brands [23], and previous studies have validated the comparability between consumer wearable devices and the ActiGraph [20,24]. Physical activity data from an accelerometer can be estimated using vector magnitude, which allows measurement of the extent of movement in three-dimensional space. The vector magnitude can be calculated using the coordinate values on the x-, y-, and z-axis measured by a triaxial accelerometer. This incorporates the vertical axis (z-axis: up–down) as well as the longitudinal (y-axis: forward–backward) and lateral (x-axis: left–right) axes [25]. We calculated the vector magnitude using the positional data from the ActiGraph and the Galaxy Watch to measure physical activity. Both devices use a photodetector to detect the light of their surroundings in lux. Light exposure data were gathered from the participants who wore both devices for 14 days, and the results showed no significant difference in this data between the devices.

We referred to previous study that developed a sleep prediction model using three different devices [4]. Previous studies demonstrated that the Galaxy watch can gather data at a level comparable to that of the Actigraph for overground walking and daily physical activity in free-living conditions [20,24]. In addition, we analyzed physical activity data in various situation (sitting, walking, and running) and light exposure data in indoor/outdoor environment from randomly selected participants who wore both devices simultaneously.

We excluded physical activity data or light exposure data from the analysis if there were any incomplete measurements due to issues such as device malfunction, battery depletion, or obstruction of the light sensor by long sleeves. It is not entirely possible to distinguish between genuine participant inactivity and depletion of the device battery, or between participants being in a dark environment and obstruction of the light sensor by long sleeves. We attempted to differentiate as much as possible by excluding data if there was continuous non-measurement of physical activity for 30 minutes or light exposure for 3 minutes. These criteria are consistent with our previous research [2].

We also measured HRV, which is an indicator of several conditions which affects the autonomic nervous system, including psychological stress [26]. HRV can be measured by electrocardiogram, cardiac belt, Holter monitor, or consumer wearable devices with a pulse monitor [27]. In this study, we used the R-R interval (RRI), measured with photoplethysmography, a pulse monitor on the Galaxy Watch, for every 2 consecutive minutes within the 30 min before sleep to calculate pNN50, defined as the proportion of successive RRIs greater than 50 milliseconds divided by the total number of RRIs [28]. Among the possible HRV indicators, pNN50 is considered to effectively reflect psychological states of relaxation or anxiety by measuring parasympathetic activity [29,30]. The ActiGraph does not have photoplethysmography, thus we developed new prediction models with HRV data from only the Galaxy Watch users.

When assessing sleep, we converted 10-second epochs into 60-second epochs using the ActiLife software (version 6.13.3, ActiGraph) and Python 3.6.4 (Python Software Foundation), which is consensus when measuring sleep by actigraphy [31,32]. This reflects the relatively stable movement patterns that occur during sleep compared with the active movement patterns that occur during wakefulness [31]. We used the Cole–Kripke algorithm, which identifies sleep/wake status from the mean value of y-axis actigraphy data, to measure in-bed time, sleep time, and wake time for both devices [32,33]. We used manual scoring in addition to the algorithm to avoid underestimating sleep latency or waking after sleep onset based on the Actigraphy Scoring Hierarchy Manual [34]. Furthermore, we used sleep diaries to assess the times when the devices were not worn. We followed the Society of Behavioral Sleep Medicine guidelines during actigraphy data scoring [35].

Target outcome

The target outcome for ‘good sleep’ was ≥90% sleep efficiency, which matches the sleep efficiency target of 90% for sleep restriction therapy in CBTi [36,37]. Based on the data from the wearable devices, this study defined and calculated sleep efficiency as the ratio of total sleep time (sleep time to wake time) to the total time spent in bed (in-bed time to out-of-bed time) [38].

Statistical analysis

In our previous study, we developed 24 different sleep prediction models with different data sources (wearable device, sleep diary, and combined data), data lengths (1-day and 2-day data), and analysis methods (machine learning, deep learning, and classical regression). We used extreme gradient boosting (XGBoost) for machine learning and convolutional neural network (CNN) and long short-term memory (LSTM) for deep learning. XGBoost is a scalable machine learning system for gradient tree boosting. It is based on a tree ensemble model, which combines decision trees to provide better predictive performance by creating more variance within the model [39]. CNN is a type of artificial neural network. Every layer of CNN transforms the input data to an output result of neuron activation, eventually leading to the final connected layers showing synthesized results [40]. LSTM is a recurrent neural network (RNN) model. RNN models are effective learning models for sequential data that use memory cells characterized by state maintenance over time [41]. All analysis methods were same as our previous preliminary study [2]. Here, we combined ActiGraph and Galaxy Watch data to create 24 sleep prediction models as before (Supplementary Figure 1). We developed prediction models with HRV and 3-day data from the Galaxy Watch data (Figure 1).

Figure 1. Data partitioning for the development of the sleep prediction models.

Morning is the period from out-of-bed time to noon. Afternoon is the period from noon to 6 PM. Evening is the period from 6 PM to in-bed time. Sleep time indicates the period from in-bed time of the previous day to out-of-bed time of the same day.

We measured model performance by accuracy and area under the curve (AUC). Accuracy referred to the number of correct predictions (both true positives and true negatives) divided by the total number of predictions. AUC values were used to show the combined predictability and validity of each model.

All statistical analyses were performed using R Studio version 1.1.463 (RStudio, Inc.) and Python 3.6.4 (Python Software Foundation).

Results

Characteristics of participants

Two hundred seventy-eight participants were enrolled in this study, including 91 of the 109 participants from the previous study who wore the ActiGraph [2]. The remaining 169 participants were newly recruited; one of these participants was screened out, 33 wore both the ActiGraph and Galaxy Watch at the same time, and 135 wore the Galaxy Watch only. Data from 29 participants were excluded from analysis (23 used both devices and 6 used the Galaxy Watch only) due to device errors (n = 18) or excessive time not wearing the device (n = 11). As 139 people completed the second 2-week study, the analysis ultimately included 2-week experiment data from a total of 230 participants (91 used the ActiGraph only, 129 used the Galaxy Watch only, and 10 used both devices). We defined one data unit as one day and one night of data; thus, because there were 13 possible data units per participant during the 2-week experiment, we could have theoretically collected 3120 units. The actual collected data included 2883 data units because 237 data units were lost during data collection (n = 132, battery depletion; n = 105, early study termination without consent withdrawal). We also excluded 282 data units related to physical activity and 465 data units related to light exposure due to insufficient measurements as described in the methods (Figure 2). Therefore, 2136 data units from 230 participants were used in the analysis. The analysis included data from 10 participants who wore both devices, 101 (897 data units) who wore only the ActiGraph, and 139 who wore only the Galaxy Watch (1239 data units). The 1-day sleep prediction models used 2136 units of data. For the 2-day prediction models, we combined two consecutive data units into a single unit, generating 1103 two-day units. The models based on consumer wearable device data were developed with 1239 data units for the 1-day model, 572 units for the 2-day model, and 308 units for the 3-day model.

Figure 2. Flow diagram of participant enrollment and data selection.

AC, ActiGraph; G, Galaxy Watch

aConsent withdrawal (n = 4), whole data lost (n = 3), incomplete data (n = 11); bBattery depletion (n = 70), early study termination (n = 52); cLack of physical activity data more than 30 minutes (n = 109), lack of light exposure data more than 3 minutes (n = 219); dExcluded due to narcolepsy; eWhole data lost (n = 18), incomplete data (n = 11); f13 units of data for AC (n = 10) and G (n = 139); gBattery depletion (n = 62), early study termination (n = 53); hLack of physical activity data more than 30 minutes (n = 173), lack of light exposure data more than 3 minutes (n = 246).

The participant characteristics are presented in Table 1.

Table 1. Participant characteristics.

Variable	All participants (n = 230)	Participants with ActiGraph (n = 101)	Participants with Galaxy Watch (n = 139)	P value	
Age, years, mean ± SD	33.5 ± 9.1	30.9 ± 8.8	35.5 ± 8.8	<.001	
Sex, female, n (%)	146 (63.5)	65 (64.4)	90 (64.7)	.95	
BMI, kg/m2, mean ± SD	22.6 ± 3.5	22.7 ± 3.6	22.6 ± 3.4	.93	
Education, years, mean ± SD	16.9 ± 2.5	16.8 ± 1.9	16.9 ± 2.8	.77	
Regular alcohol consumption, yesa, n (%)	142 (61.7)	56 (55.4)	92 (66.2)	.54	
Current smoking, yes, n (%)	16 (7.0)	8 (7.9)	8 (5.8)	.28	
Regular caffeine consumption, yesb, n (%)	192 (83.9)	81 (80.2)	122 (87.8)	.61	
PSQI global score, mean ± SD	5.6 ± 2.6	5.9 ± 2.6	5.3 ± 2.6	.004	
ISI, mean ± SD	6.1 ± 4.5	6.0 ± 4.2	6.3 ± 4.9	.55	
Ten participants used both the Actigraph and Galaxy Watch.

PSQI, Pittsburgh Sleep Quality Index; ISI, Insomnia Severity Index.

a >300 mL per week. b>1 cup per day.

Performance of the sleep prediction models

The performance of all the sleep prediction models (Tables 2 and 3; mean accuracy ± SD=.66±.00, area under the curve [AUC]=.69±.00) improved compared with the performance of the prediction models from our previous study (accuracy=.61±.00, AUC=.62±.01) [2]. The performance of the sleep prediction models with added HRV data (accuracy=.71±.01, AUC= .66±.01) also improved compared with the previous models. The 2-day models performed better (accuracy=.73±.06, AUC=.70±.08) than the 1-day models (accuracy=.68±.08, AUC=.64±.08) and the 3-day models (accuracy=.70±.07, AUC=.67±.08). The model with the best performance was the extreme gradient boosting model, XGBoost, with 2-day data units and combined actigraphy and sleep diary data (accuracy=.85, AUC=.80).

Table 2. Performance of sleep prediction models.

Analysis method	Parameter	Wearable devicea	Sleep diaryb	Combinedc	
1-day	2-day	1-day	2-day	1-day	2-day	
XGBoost	Accuracy	.67	.66	.60	.62	.73	.74	
AUC	.69	.72	.61	.64	.77	.79	
CNN	Accuracy	.63	.62	.65	.64	.68	.69	
AUC	.65	.57	.69	.68	.70	.75	
LSTM	Accuracy	.63	.64	.58	.60	.69	.71	
AUC	.65	.68	.62	.62	.76	.77	
Logistic regression	Accuracy	.62	.69	.64	.64	.69	.72	
AUC	.64	.66	.65	.66	.74	.74	
XGBoost, extreme gradient boost; CNN, convolutional neural network; LSTM, long short-term memory; AUC, area under the curve.

a Models developed from only wearable device data. bModels developed from only sleep diary data. cModels developed from both actigraphy and sleep diary data.

Table 3. Performance of sleep prediction models with the addition of heart rate variability.

Analysis method	Parameter	Wearable devicea	Sleep diaryb	Combinedc	
1-day	2-day	3-day	1-day	2-day	3-day	1-day	2-day	3-day	
XGBoost	Accuracy	.60	.76	.70	.73	.74	.72	.82	.85	.85	
AUC	.52	.60	.57	.70	.68	.70	.75	.80	.79	
CNN	Accuracy	.64	.59	.63	.62	.70	.61	.64	.72	.65	
AUC	.53	.55	.56	.58	.67	.60	.64	.70	.63	
LSTM	Accuracy	.56	.76	.65	.71	.72	.72	.80	.83	.78	
AUC	.55	.60	.61	.70	.73	.71	.73	.77	.78	
Logistic regression	Accuracy	.63	.68	.67	.69	.71	.69	.72	.73	.76	
AUC	.56	.62	.61	.68	.73	.71	.70	.77	.75	
These models were developed from consumer wearable device (Galaxy Watch) data.

XGBoost, extreme gradient boost; CNN, convolutional neural network; LSTM, long short-term memory; AUC, area under the curve.

a Models developed from only wearable device data. bModels developed from only sleep diary data. cModels developed from both actigraphy and sleep diary data.

Discussion

This study aimed to improve sleep prediction models based on physical activity and light exposure by increasing the amount of data, adding a psychological stress indicator, and adjusting the data length. The XGBoost model developed from 2-day combined data and including HRV showed the best performance of all the models with 80% AUC and 85% accuracy. Although the 3-day data model did not outperform the 2-day model, this study proved for the second time that the existing sleep prediction model is valid and demonstrated its evolutionary potential.

It has been more than 20 years since actigraphy was first used in sleep research [42]. Unlike polysomnography, which usually measures sleep in a laboratory environment, actigraphy can evaluate sleep in everyday environments and can even monitor some activities during wake time. However, it cannot provide variables other than those approved by the manufacturers, and researchers cannot identify errors before the device is returned because the data are stored in the device. Furthermore, it is very expensive compared with consumer wearable devices, inevitably limiting the expansion of research results. To increase the amount of data and apply the results to patients with insomnia, a study using a popular consumer wearable device was needed. Sleep research using consumer wearable devices, including the Galaxy Watch, is becoming more common [43,44]. We conducted an experiment to compare data from a research device and data from a consumer wearable device, referring to previous studies that used two or more wearable devices to collect activity data [4,21,45,46]. We received Galaxy Watch data from the cloud in real time through the in-house program and tried to reduce data loss by monitoring the data uploads and contacting participants in case of errors. Although there was still significant data loss, we reduced the rate by 10% compared with previous studies, which may have improved the model performance. The availability of real-time data transmission and, therefore, the opportunity to reduce data loss are important developments that will allow sleep researchers to collect more data in the future.

We developed the models with HRV using only the data from the Galaxy Watch. Although the amount of data was consequently small (65% of the data volume of the models that used both ActiGraph data and Galaxy Watch data), the overall performance was better than that of the models without HRV regardless of the source, analysis method, or data length. HRV is used in sleep research because it reflects psychological arousal due to stress exposure. Estrela and colleagues found that lower high-frequency HRV (HF-HRV), reflecting increased sleep reactivity to stress, is associated with poor sleep quality [10]. MacNeil and colleagues also found that greater HF-HRV reactivity to worry correlates with sleep disturbance [11]. Instead of using HF-HRV, which is a frequency-domain index of HRV, we used pNN50, a time-domain index, to measure HRV. Both HF-HRV and pNN50 are known to reflect the activity of the parasympathetic nervous system, which in turn reflects stress reactivity [47]. The use of a time-domain indicator of HRV has the advantage of easy calculation; because pNN50 is calculated using RRI, resting HRV can be easily obtained from consumer wearable devices. HRV reflects changes in autonomic function, which means that it can be affected by various factors including age, sex, genetic factors, and medical conditions such as metabolic syndrome or pain [48–51]. Hence, HRV can be used as an objective assessment of stress and mental health, but it is important to consider other potential influencing factors [47]. In this study, we conducted research on healthy participants without significant psychiatric or medical history. Future studies including patients with various conditions can potentially improve the accuracy of sleep prediction. This is the first study to predict the quality of the next sleep by analyzing both physical and psychological state affecting sleep, and the results show that psychological state has a significant impact on the next future sleep.

We hypothesized that longer data collection periods would improve predictive power for the next sleep because we found that a 2-day model performed better than a 1-day model in our previous study. However, the results of the current study did not support this hypothesis. Previous studies that predicted future sleep with artificial intelligence used data from only a single day [3–5], so we can only speculate as to why the performance of our 3-day models was not better than that of the 1-day or 2-day models. Because many heterogeneous variables influence sleep, information about physical activity, light exposure, and HRV from more than two nights ago may not be enough to predict the next sleep. We assumed that the 2-day model performed better than the 1-day model because the 2-day data contained information about the previous night’s sleep; however, the information about the sleep from two nights before may have less of an impact on the next sleep prediction than the information from the previous night. Similarly, physical activity, light exposure, and psychological arousal from more than 2 days ago may have less of an effect on the next sleep. It is also possible that unmeasured variables may have long-term effect on sleep, potentially causing prediction errors. Another possible reason is that the 3-day models were based on a reduced amount of data compared with the 2-day and 1-day models because the data for the three consecutive days were grouped into one unit. To the best of our knowledge, no other studies have compared the effects of different data lengths on sleep prediction. Further studies are needed to verify the most efficient data length for sleep prediction.

This study has some limitations. First, the definition of ‘good sleep’ was not subjective but rather an objective measure of sleep efficiency. There has been controversy in clinical insomnia research over whether pharmacological/non-pharmacological treatment goals should be measured with objective indicators, such as sleep efficiency, or subjective sleep quality improvement [52]. This study’s definition of good sleep was based on sleep efficiency in consideration of CBTi, as improved sleep efficiency is often set as the goal of CBTi sleep restriction therapy and stimulation control therapy [53]. To apply the sleep prediction model developed in this study to CBTi in the future, additional validation in patients with insomnia is required. Second, although no participants reported sleep disturbances, 56.1% (n = 129) had a global Pittsburgh Sleep Quality Index score of at least 5 and 22.2% (n = 51) had a score of at least 8. A PSQI score above 5 generally indicates poor sleep quality, although some researchers have proposed that a cut-off score of 8 or higher is more appropriate to accurately screen for sleep problems [14,54]. The average PSQI score among our participants was high despite our recruitment of a healthy population of individuals who denied having sleep disturbance. This may be related to the fact that the researchers posted recruitment advertisements inside medical schools and hospitals. Among participants, 46.1% (n = 106) were medical practitioners or students, and individuals studying or working in medicine are often too busy to sleep. Lastly, the sleep prediction model was developed for participants who have not experienced sleep disturbances. Further research on insomnia patients is needed to demonstrate its clinical usefulness.

Conclusion

Despite the above limitations, we have improved the performance of our previously developed sleep prediction models by increasing the amount of data and adding HRV as an indicator of psychological state. We expected improved sleep prediction by increasing the data length to 3 days, but the 3-day models failed to perform better than the 2-day models. We believe that these results, including the reliable performance of automated sleep prediction models and information about the factors that improve predictive performance, may provide a basis for further research in the field of non-pharmacological treatment for insomnia.

Supplementary Material

Supplemental Material

Authors contributions

Kyung Mee Park and Eun Lee conceived, designed, wrote the initial draft, and revised the manuscript and were primarily responsible for the final content. Kyung Mee Park, Sang Eun Lee, Changhee Lee and Eun Lee conceptualized statistical analysis and interpreted the results. Kyung Mee Park, Eunchae Choi, Eun Lee enrolled participants and performed experiments. Sang Eun Lee, Changhee Lee, Hyun Duck Hwang, and Do Hoon Yoon performed data collection, processing, and the analyses including machine learning and deep learning. Kyung Mee Park and Eun Lee wrote original drafts, and all authors reviewed and edited the manuscript. All authors contributed to the interpretation of the results. All authors have read and approved the final manuscript.

Disclosure statement

The authors declare no financial arrangements or connections and other potential conflicts of interest that are relevant to the content of this paper.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
==== Refs
References

1 Lee E. Cognitive behavioral treatment as a digital therapeutic for insomnia. Chronobiol Med. 2020;2 (1 ):10–15. doi: 10.33069/cim.2020.0007.
2 Park KM, Lee SE, Lee C, et al. Prediction of good sleep with physical activity and light exposure: a preliminary study. J Clin Sleep Med. 2022;18 (5 ):1375–1383. doi: 10.5664/jcsm.9872.34989333
3 Xu B, Ma Y, Sun G, et al. Will you have a good sleep tonight? sleep quality prediction with mobile phone. 7th International Conference on Body Area Networks; 2012. Citeseer. doi: 10.4108/icst.bodynets.2012.250091.
4 Fellger A, Sprint G, Weeks D, et al. Wearable device-independent next day activity and next night sleep prediction for rehabilitation populations. IEEE J Transl Eng Health Med. 2020;8 :2700509–2700509. doi: 10.1109/JTEHM.2020.3014564.32802598
5 Sathyanarayana A, Joty S, Fernandez-Luque L, et al. Sleep quality prediction from wearable data using deep learning. JMIR Mhealth Uhealth. 2016;4 (4 ):e125. doi: 10.2196/mhealth.6562.27815231
6 Ajiboye A, Abdullah-Arshah R, Qin H, et al. Evaluating the effect of dataset size on predictive model using supervised learning technique. ijsecs. 2015;1 (1 ):75–84. doi: 10.15282/ijsecs.1.2015.6.0006.
7 Juba B, Le HS. Precision-recall versus accuracy and the role of large data sets. Proceedings of the AAAI conference on artificial intelligence. 2019;33(1):4039–4048. doi: 10.1609/aaai.v33i01.33014039.
8 Âkerstedt T. Psychosocial stress and impaired sleep. Scand J Work Environ Health. 2006;32(6):493–501. http://www.jstor.org/stable/40967601 17173205
9 Åkerstedt T, Orsini N, Petersen H, et al. Predicting sleep quality from stress and prior sleep–a study of day-to-day covariation across six weeks. Sleep Med. 2012;13 (6 ):674–679. doi: 10.1016/j.sleep.2011.12.013.22621983
10 da Estrela C, McGrath J, Booij L, et al. Heart rate variability, sleep quality, and depression in the context of chronic stress. Ann Behav Med. 2021;55 (2 ):155–164. doi: 10.1093/abm/kaaa039.32525208
11 MacNeil S, Deschênes SS, Caldwell W, et al. High-frequency heart rate variability reactivity and trait worry interact to predict the development of sleep disturbances in response to a naturalistic stressor. Ann Behav Med. 2017;51 (6 ):912–924. doi: 10.1007/s12160-017-9915-z.28527014
12 Chung K-F, Lee C-T, Yeung W-F, et al. Sleep hygiene education as a treatment of insomnia: a systematic review and meta-analysis. Fam Pract. 2018;35 (4 ):365–375. doi: 10.1093/fampra/cmx122.29194467
13 First MB, Williams JBW, Karg RS, et al. Structured clinical interview for DSM-5—research version. Arlington, VA: American Psychiatric Association; 2015.
14 Buysse DJ, Reynolds CF, 3rd, Monk TH, et al. The pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28 (2 ):193–213. doi: 10.1016/0165-1781(89)90047-4.2748771
15 Bastien CH, Vallières A, Morin CM. Validation of the insomnia severity index as an outcome measure for insomnia research. Sleep Med. 2001;2 (4 ):297–307. doi: 10.1016/s1389-9457(00)00065-4.11438246
16 Quante M, Kaplan ER, Cailler M, et al. Actigraphy-based sleep estimation in adolescents and adults: a comparison with polysomnography using two scoring algorithms. Nat Sci Sleep. 2018;10 :13–20. doi: 10.2147/NSS.S151085.29403321
17 Santos-Lozano A, Santin-Medeiros F, Cardon G, et al. Actigraph GT3X: validation and determination of physical activity intensity cut points. Int J Sports Med. 2013;34 (11 ):975–982. doi: 10.1055/s-0033-1337945.23700330
18 Lewis ZH, Cannon M, Rubio G, et al. Analysis of the behavioral change and utility features of electronic activity monitors. Technologies. 2020;8 (4 ):75. doi: 10.3390/technologies8040075.
19 Saganowski S, Kazienko P, Dziezyc M, et al. Review of consumer wearables in emotion, stress, meditation, sleep, and activity detection and analysis. 2020; arXiv preprint arXiv:.00093.
20 Hartung V, Sarshar M, Karle V, et al. Validity of consumer activity monitors and an algorithm using smartphone data for measuring steps during different activity types. Int J Environ Res Public Health. 2020;17 (24 ):9314. doi: 10.3390/ijerph17249314.33322833
21 Arora A, Chakraborty P, Bhatia M. Analysis of data from wearable sensors for sleep quality estimation and prediction using deep learning. Arab J Sci Eng. 2020;45 (12 ):10793–10812. doi: 10.1007/s13369-020-04877-w.
22 Carney CE, Buysse DJ, Ancoli-Israel S, et al. The consensus sleep diary: standardizing prospective sleep self-monitoring. Sleep. 2012;35 (2 ):287–302. doi: 10.5665/sleep.1642.22294820
23 Rowlands AV, Mirkes EM, Yates TE, et al. Accelerometer-assessed physical activity in epidemiology: are monitors equivalent? University of Leicester. Journal Contribution. 2017;50(2):357–265. doi: 10.1249/MSS.0000000000001435.
24 Viciana J, Casado-Robles C, Guijarro-Romero S, et al. Are wrist-worn activity trackers and mobile applications valid for assessing physical activity in high school students? Wearfit study. J Sports Sci Med. 2022;21 (3 ):356–375. doi: 10.52082/jssm.2022.356.36157395
25 Leeger-Aschmann CS, Schmutz EA, Zysset AE, et al. Accelerometer-derived physical activity estimation in preschoolers–comparison of cut-point sets incorporating the vector magnitude vs the vertical axis. BMC Public Health. 2019;19 (1 ):513. doi: 10.1186/s12889-019-6837-7.31060538
26 Kaur B, Durek Jj, O’Kane Bl, et al. Heart rate variability (HRV): an indicator of stress. Proceedings of Independent Component Analyses, Compressive Sampling, Wavelets, Neural Net, Biosystems, and Nanoengineering XII; 2014. SPIE. doi: 10.1117/12.2051148.
27 Rodrigues E, Lima D, Barbosa P, et al. HRV monitoring using commercial wearable devices as a health indicator for older persons during the pandemic. Sensors. 2022;22 (5 ):2001. doi: 10.3390/s22052001.35271148
28 Shaffer F, Ginsberg JP. An overview of heart rate variability metrics and norms. Front Public Health. 2017;5 :258. doi: 10.3389/fpubh.2017.00258.29034226
29 Spiegelhalder K, Fuchs L, Ladwig J, et al. Heart rate and heart rate variability in subjectively reported insomnia. J Sleep Res. 2011;20 (1 Pt 2 ):137–145. doi: 10.1111/j.1365-2869.2010.00863.x.20626615
30 Kunikata H, Watanabe K, Miyoshi M, et al. The effects measurement of hand massage by the autonomic activity and psychological indicators. J Med Invest. 2012;59 (1–2 ):206–212. doi: 10.2152/jmi.59.206.22450009
31 Migueles JH, Cadenas-Sanchez C, Ekelund U, et al. Accelerometer data collection and processing criteria to assess physical activity and other outcomes: a systematic review and practical considerations. Sports Med. 2017;47 (9 ):1821–1845. doi: 10.1007/s40279-017-0716-0.28303543
32 Cole RJ, Kripke DF, Gruen W, et al. Automatic sleep wake identification from wrist activity. Sleep. 1992;15 (5 ):461–469. doi: 10.1093/sleep/15.5.461.1455130
33 de Souza L, Benedito-Silva AA, Pires ML, et al. Further validation of actigraphy for sleep studies. Sleep. 2003;26 (1 ):81–85. doi: 10.1093/sleep/26.1.81.12627737
34 Rijsketic JM, Dietch JR, Wardle-Pinkston S, et al. Actigraphy (Actiware) Scoring Hierarchy Manual; 2020. Available from: https://insomnia.arizona.edu/actigraphy.
35 Ancoli-Israel S, Martin JL, Blackwell T, et al. The SBSM guide to actigraphy monitoring: clinical and research applications. Behav Sleep Med. 2015;13 (sup1 ):S4–S38. doi: 10.1080/15402002.2015.1046356.26273913
36 Pigeon WR. Treatment of adult insomnia with cognitive–behavioral therapy a. J Clin Psychol. 2010;66 (11 ):1148–1160. doi: 10.1002/jclp.20737.20853442
37 Babson KA, Feldner MT, Badour CL. Cognitive behavioral therapy for sleep disorders. Psychiatr Clin North Am. 2010;33 (3 ):629–640. doi: 10.1016/j.psc.2010.04.011.20599137
38 Reed DL, Sacco WP. Measuring sleep efficiency: what should the denominator be? J Clin Sleep Med. 2016;12 (2 ):263–266. doi: 10.5664/jcsm.5498.26194727
39 Chen T, Guestrin G. Xgboost: a scalable tree boosting system. Proceedings of Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining; 2016:785–794. ACM. doi: 10.1145/2939672.2939785.
40 Voulodimos A, Doulamis N, Doulamis A, et al. Deep learning for computer vision: a brief review. Comput Intell Neurosci. 2018;2018 :7068349–7068313. doi: 10.1155/2018/7068349.29487619
41 Greff K, Srivastava RK, Koutnik J, et al. LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst. 2016;28 (10 ):2222–2232. doi: 10.1109/TNNLS.2016.2582924.27411231
42 Brown AC, Smolensky MH, D’Alonzo GE, et al. Actigraphy: a means of assessing circadian patterns in human activity. Chronobiol Int. 1990;7 (2 ):125–133. doi: 10.3109/07420529009056964.2242506
43 Byun J-I, Noh KC, Shin WC. Performance of the fitbit charge 2 and galaxy watch 2 compared with polysomnography in assessing patients with obstructive sleep apnoea. Chronobiol Int. 2023;40 (5 ):596–602. doi: 10.1080/07420528.2023.2191720.36971253
44 Gaiduk M, Seepold R, Martínez Madrid N, et al. Assessing the feasibility of replacing subjective questionnaire-based sleep measurement with an objective approach using a Smartwatch. Sensors. 2023;23 (13 ):6145. doi: 10.3390/s23136145.37447992
45 Moore L, McDevitt B, Akhtar N, et al. Preliminary investigations of the agreement between two wearable accelerometers for use in clinical studies. Proceedings of 2020 31st Irish Signals and Systems Conference (ISSC); 2020:1–4. IEEE. doi: 10.1109/ISSC49989.2020.9180169.
46 Sundararajan K, Georgievska S, Te Lindert BH, et al. Sleep classification from wrist-worn accelerometer data using random forests. Sci Rep. 2021;11 (1 ):24. doi: 10.1038/s41598-020-79217-x.33420133
47 Kim H-G, Cheon E-J, Bai D-S, et al. Stress and heart rate variability: a meta-analysis and review of the literature. Psychiatry Investig. 2018;15 (3 ):235–245. doi: 10.30773/pi.2017.08.17.
48 Garavaglia L, Gulich D, Defeo MM, et al. The effect of age on the heart rate variability of healthy subjects. PLoS One. 2021;16 (10 ):e0255894. doi: 10.1371/journal.pone.0255894.34624048
49 Zhang J. Effect of age and sex on heart rate variability in healthy subjects. J Manipulative Physiol Ther. 2007;30 (5 ):374–379. doi: 10.1016/j.jmpt.2007.04.001.17574955
50 Stuckey MI, Tulppo MP, Kiviniemi AM, et al. Heart rate variability and the metabolic syndrome: a systematic review of the literature. Diabetes Metab Res Rev. 2014;30 (8 ):784–793. doi: 10.1002/dmrr.2555.24816921
51 Koenig J, Jarczok M, Ellis R, et al. Heart rate variability and experimentally induced pain in healthy adults: a systematic review. Eur J Pain. 2014;18 (3 ):301–314. doi: 10.1002/j.1532-2149.2013.00379.x.23922336
52 Spruyt K, Ferri R. Objective or subjective test? The answer lies in its psychometric property! Sleep. 2022;45(10). doi: 10.1093/sleep/zsac169.
53 Edinger JD, Carney CE. Overcoming insomnia: a cognitive-behavioral therapy approach, therapist guide. NY, USA: Oxford University Press; 2014.
54 Mollayeva T, Thurairajah P, Burton K, et al. The Pittsburgh sleep quality index as a screening tool for sleep dysfunction in clinical and non-clinical samples: a systematic review and meta-analysis. Sleep Med Rev. 2016;25 :52–73. doi: 10.1016/j.smrv.2015.01.009.26163057
