==== Front Front Neurosci Front Neurosci Front. Neurosci. Frontiers in Neuroscience 1662-4548 1662-453X Frontiers Media S.A. 10.3389/fnins.2023.1199150 Neuroscience Methods Mapping the time-varying functional brain networks in response to naturalistic movie stimuli Song Limei 1 † Ren Yudan 1 * † Wang Kexin 1 Hou Yuqing 1 Nie Jingsi 2 He Xiaowei 1 1School of Information Science and Technology, Northwest University, Xi’an, China 2School of Foreign Studies, Xi’an Jiaotong University, Xi’an, China Edited by: Bin Jing, Capital Medical University, China Reviewed by: Zhen Zhou, University of Pennsylvania, United States; Xuan Li, Helmholtz Association of German Research Centres (HZ), Germany *Correspondence: Yudan Ren, yudan.ren@nwu.edu.cn †These authors have contributed equally to this work and share first authorship 16 6 2023 2023 17 119915003 4 2023 30 5 2023 Copyright © 2023 Song, Ren, Wang, Hou, Nie and He. 2023 Song, Ren, Wang, Hou, Nie and He https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. One of human brain’s remarkable traits lies in its capacity to dynamically coordinate the activities of multiple brain regions or networks, adapting to an externally changing environment. Studying the dynamic functional brain networks (DFNs) and their role in perception, assessment, and action can significantly advance our comprehension of how the brain responds to patterns of sensory input. Movies provide a valuable tool for studying DFNs, as they offer a naturalistic paradigm that can evoke complex cognitive and emotional experiences through rich multimodal and dynamic stimuli. However, most previous research on DFNs have predominantly concentrated on the resting-state paradigm, investigating the topological structure of temporal dynamic brain networks generated via chosen templates. The dynamic spatial configurations of the functional networks elicited by naturalistic stimuli demand further exploration. In this study, we employed an unsupervised dictionary learning and sparse coding method combing with a sliding window strategy to map and quantify the dynamic spatial patterns of functional brain networks (FBNs) present in naturalistic functional magnetic resonance imaging (NfMRI) data, and further evaluated whether the temporal dynamics of distinct FBNs are aligned to the sensory, cognitive, and affective processes involved in the subjective perception of the movie. The results revealed that movie viewing can evoke complex FBNs, and these FBNs were time-varying with the movie storylines and were correlated with the movie annotations and the subjective ratings of viewing experience. The reliability of DFNs was also validated by assessing the Intra-class coefficient (ICC) among two scanning sessions under the same naturalistic paradigm with a three-month interval. Our findings offer novel insight into comprehending the dynamic properties of FBNs in response to naturalistic stimuli, which could potentially deepen our understanding of the neural mechanisms underlying the brain’s dynamic changes during the processing of visual and auditory stimuli. dynamic functional brain network time-varying dictionary learning and sparse coding naturalistic stimuli fMRI National Natural Science Foundation of China 10.13039/501100001809 62006187 61971350 12271434 Youth Innovation Team Foundation of Education Department of Shaanxi Province Government21JP119 China Postdoctoral Science Foundation Funded Project2021M702650 Natural Science Basic Research Program of Shaanxi2023-JC-JQ-57 Key Research and Development Program Project of Shaanxi Province2020SF-036 section-at-acceptanceBrain Imaging Methods ==== Body pmc1. Introduction The study of functional brain networks (FBNs) can reveal the mechanisms and properties of brain functions, which is significant for elucidating the cognitive, sensory, and emotional functions of the brain (Rubinov and Sporns, 2010; Barrett and Satpute, 2013). The key characteristic of FBNs is their dynamic change across time for adapting to the continuously complex external environment (Hutchison et al., 2013; Calhoun et al., 2014; Ma et al., 2014; Lurie et al., 2020). Research on dynamic functional networks (DFNs) using functional magnetic resonance imaging (fMRI) have largely advanced our understanding of dynamic brain activity in responding to external sensory information (Tononi et al., 1996; Park and Friston, 2013; Calhoun et al., 2014). Current studies on dynamic FBNs mainly rely on the resting-state paradigm (Hutchison et al., 2013; Allen et al., 2014; Liegeois et al., 2017; Savva et al., 2019). However, the resting-state is challenging to use when investigating specific cognitive processes due to its unrestrained nature and undesired behavioral disturbances, such as head movements and microsleep (Van Dijk et al., 2012; Buckner et al., 2013; Tagliazucchi and Laufs, 2014). In addition, electrophysiological and neuroimaging studies suggest that neural responses under the resting-state paradigm show general reliability and reproducibility (Belitski et al., 2008; Wang et al., 2017). Naturalistic paradigms have been found to be more reliable and effective than the resting-state paradigm in exploring FBNs by providing cognitive constraints and high reliability (Sonkusare et al., 2019). These paradigms involve rich multimodal dynamic stimuli that reflect our everyday experience, resulting in more intricate patterns of functional brain activity and more diverse FBNs. Movies, as a typical representative of the passive viewing naturalistic paradigm, provide continuous audiovisual experiences that elicit stronger emotions than brief and isolated emotion-inducing events (Hasson et al., 2004; Meer et al., 2020; Saarimaki, 2021). Hence, using movies as stimuli in fMRI studies can better induce higher-order and complex FBNs related to cognition and emotion, thus leading to a more comprehensive understanding of DFNs and their relationship with cognition, sensation, and emotion. However, most of the current research on DFNs focuses on the topology of time-varying connectivity, which limits the regions or nodes of the network to the selected template or the region of interest (ROI) (Hutchison et al., 2013; Calhoun et al., 2014). Less attention has been paid to the dynamic spatial patterns of the large-scale complex FBNs themselves induced by natural stimuli. To fully understand FBNs derived from fMRI data, it is necessary to investigate the spatio-temporal dynamics of these FBNs (Ge et al., 2020). In addition, while recent studies have analyzed the test–retest reliability of dynamic functional connectivity constrained by selected brain parcellation under the naturalistic paradigm (Tian et al., 2021; Zhang et al., 2021), the reliability of large-scale dynamic spatial patterns of FBNs remains unclear. Therefore, further research is required to explore the dynamic spatial patterns of FBNs and their relationship with cognition and perception, as well as their reliability during naturalistic conditions. Inspired by the effectiveness of dictionary learning and sparse coding (DLSC) method in detecting static and dynamic FBNs (Lv, 2013; Ren et al., 2017a; Ge et al., 2020), we developed a data-driven method that combines group-wise DLSC approach with sliding window strategy, to identify and quantify the dynamic spatial patterns of time-varying FBNs from naturalistic fMRI data (NfMRI). Our method successfully identified several higher-order and complex FBNs, such as cerebellum-related networks, and revealed the significant correlations between movie annotations and detected DFNs. Additionally, we observed that specific individual DFNs were correlated with individuals’ subjective emotional perceptions to the movie. Furthermore, we validated the reliability of DFNs derived from two scanning sessions with 3 months intervals by evaluating their ICCs. In general, our study provides novel insights into the dynamic characteristics of FBNs under naturalistic stimuli. 2. Results 2.1. Group-wise static FBNs We first identified seven consistent and representative group-wise static FBNs for both session A and session B via the DLSC approach. Figure 1 shows the representative FBNs of session A. These networks include either typically activated simple networks or complex networks. The simple networks involve the visual network (Figure 1A) and the auditory network (Figure 1B). The complex networks consist of multiple co-activated brain networks/regions, including auditory and cerebellar network (AC) (Figure 1C), the audiovisual and sensorimotor network (VAS) (Figure 1D), the partial default mode network (DMN), the salience and cerebellar network (pDSC) (Figure 1E), the DMN and cerebellar network (DC) (Figure 1F), and the dorsal attention network (DAN) (Figure 1G). Specifically, the AC network is primarily composed of auditory, cerebellar posterior crus 1,2 and vermis (Figure 1C). The VAS network is composed of visual, auditory, and sensorimotor cortex (Figure 1D). The pDSC encompasses the posterior cingulate cortex, medial prefrontal cortex, angular gyrus, anterior insula, dorsal anterior cingulate cortex, cerebellar posterior crus1,2, cerebellums 9 and vermis. Notably, the pDSC network excludes the precuneus (Figure 1E). The DC network mainly consists posterior cingulate cortex, medial prefrontal cortex angular gyrus, precuneus, cerebellar posterior crus1, 2, cerebellums 9 and vermis (Figure 1F). The DAN network includes intraparietal sulcus and the frontal eye fields (Figure 1G). A comparison between these identified FBNs and well-established resting-state templates or networks from previous studies conducted under natural stimulation is presented in Supplementary Figure S8. Figure 1 Group-wise static functional brain networks (FBNs) of session A, including (A) visual network, (B) auditory network, (C) auditory and cerebellar network (AC), (D) audiovisual and sensorimotor network (VAS), (E) partial default mode network (DMN), salience, and cerebellar network (pDSC), (F) DMN and cerebellar network (DC), (G) dorsal attention network (DAN). The FBNs derived from session B showed a high degree of spatial consistency with those observed in session A (Supplementary Figure S1), as demonstrated by the relatively high overlap rate and Pearson correlation coefficient (PCC) values between the two sets of FBNs (Table 1). Specifically, the mean overlapping rate and the mean PCC of the seven FBNs were 0.44 ± 0.11 [Mean ± standard deviation (SD)] and 0.82 ± 0.18 (Mean ± SD), respectively, suggesting the consistency and stability of the DLSC framework in detecting FBNs across two scanning sessions. Table 1 Overlapping rate and Pearson correlation coefficient (PCC) across two sessions for seven representative brain functional networks (FBNs). Visual Auditory AC VAS pDSC DC DAN Mean ± SD Overlap 0.34 0.58 0.5 0.52 0.27 0.5 0.39 0.44 ± 0.11 PCC 0.9 0.95 0.97 0.93 0.47 0.87 0.68 0.82 ± 0.18 2.2. Dynamic spatial patterns of FBNs We applied the sliding time window method with a window length of 60 repetition time (TR) units and step size of 1TR, resulting in 470 available windows. Correspondingly, 470 FBNs were obtained by applying the DLSC method, which could reflect the dynamics of time-varying large-scale networks. To provide representative visualization of these FBNs, we selected and displayed the FBNs from the first window among every 50 windows. For example, the first brain activation map in Figure 2 represents the visual network obtained during the first window (1TR to 60TR), corresponding to a time period of 1 s to 132 s. Figure 2 Dynamic evolution of the number of activated voxels (NAV) of seven brain function networks (FBNs) (session A). The corresponding FBNs of the first window among every 50 windows are displayed at the bottom. To quantitatively explore the dynamic spatial patterns of FBNs, we assessed the dynamic temporal changes of the number of activated voxels (NAV) (Figure 2) and the intensity of activated voxels (IAV) (Figure 3) for the seven FBNs. Our results revealed that for each network, both NAV and IAV exhibited temporal variations, with relatively consistent trends between the two metrics. The IAV showed less variability compared to NAV due to the calculation method of averaging activation intensities of all voxels exceeding a predefined threshold, resulting in a relatively narrow range of variation in the overall activation strength of the whole network. The peaks of NAV and IAV curves corresponded to the FBNs that displayed more pronounced and widespread patterns of activation, whereas the troughs of these curves responded to FBNs with diminished or even absent activation patterns. These findings highlight that FBNs were dynamic and evolved temporally in response to the unfolding plot of the movie, which is also consistent with the underlying neural basis of complex perception and behavior (Calhoun et al., 2014). Additionally, the lower-order perceptual networks, including visual network, auditory network, and VAS network, exhibited relatively stable level of activation over time, whereas the higher-order networks, such as pDSC, DC, and DAN networks, showed greater fluctuations in activation curves. The AC network, specifically, comprising both lower-order network (i.e., auditory network) and higher-order networks (i.e., cerebellar network), also displayed substantial fluctuations in its activation curves (Figures 2, 3). These results suggest that different FBNs exhibit distinct temporal dynamics in response to external stimuli, which may reflect their respective roles in higher-level cognitive and attentional processes. Figure 3 Dynamic evolution of the intensity of activated voxels (IAV) of seven FBNs (session A). The corresponding FBNs of the first window among every 50 windows are displayed at the bottom. The results of session B were generally consistent with those of session A, as evidenced by the visual comparison of the results depicted in Supplementary Figures S2, S3. Moreover, the PCC values for both NAV and IAV curves across session A and session B were relatively high for most FBNs (Supplementary Table S1), suggesting that the identified dynamic spatial patterns of FBNs induced by the movie viewing are reproducible and consistent across two scanning sessions. 2.3. Correlation between DFNs and movie annotations To investigate the relationship between DFNs and the unfolding of the movie, we assessed the spearman correlation between dynamic changes of NAV/IAV and movie annotations, which include language use, changepoints, the presence of positive valence of scenes (scenes_p), the presence of negative valence of scenes (scenes_n), the presence of faces with positive (face_p), and presence of faces with negative (face_n). The results showed that two DMN-related networks, i.e., the DC and pDSC networks, were significantly correlated with movie annotations. Specifically, both NVA and IVA metrics of the DC network showed statistically significant correlations with the appearance of positive facial expressions based on permutation-based testing (p < 0.05) (see Methods) (Tables 2, 3). Notably, the value of p for the IAV metric was less than 0.01 (Table 3). Additionally, the changes in INV of the pDSC network were significantly and positively correlated with the appearance of the changepoint in the movie scenes (permutation 5,000 times, p < 0.05) (Table 3). Table 2 The Spearman correlation between the changes in the number of active voxels (NAV) and movie annotations. Visual Auditory AC VAS pDSC DC DAN language −0.03 0.00 −0.09 −0.12 −0.17 −0.11 −0.14 changepoint −0.15 −0.10 −0.04 0.11 0.02 0.14 0.06 scenes_p 0.00 −0.11 0.07 0.03 −0.05 0.08 −0.11 scenes_n −0.10 0.01 −0.09 −0.19 −0.08 0.02 −0.12 face_p −0.07 −0.11 0.08 0.00 −0.02 0.16* −0.09 face_n −0.05 0.02 −0.08 −0.28 −0.07 −0.01 −0.20 Bold font indicates significant correlation (*p < 0.05). Permutation test with 5,000 iterations. Table 3 The Spearman correlation between the changes in the intensity of active voxels (IAV) and movie annotations. Visual Auditory AC VAS pDSC DC DAN language −0.14 0.03 −0.12 0.00 −0.17 −0.08 −0.08 changepoint −0.18 −0.14 0.02 0.01 0.15* 0.12 −0.01 scenes_p −0.18 −0.04 0.03 −0.10 0.04 0.13 −0.07 scenes_n −0.01 0.07 −0.08 −0.06 −0.15 −0.01 −0.07 face_p −0.06 −0.01 −0.02 −0.09 0.05 0.19** −0.07 face_n −0.12 0.04 −0.07 −0.19 −0.19 −0.06 −0.11 Bold font indicates significant correlation (*p < 0.05, **p < 0.01). Permutation test with 5,000 iterations. 2.4. Dynamic inter-subject correlation analyses The neural response evoked by the naturalistic stimuli exhibit not only high consistency across individuals, but also inter-subject variability and uniqueness reflecting personal experiences and intrinsically-driven processes under natural viewing condition, which varies across different brain regions/networks (Golland et al., 2007; Ren et al., 2017b). Hence, to quantify these group consistency and individual variations in defined DFNs, we adopted their corresponding group-wise static FBNs as templates to calculate the dynamic inter-subject correlation (ISC) (see Methods). Accordingly, the group-level dynamic ISC can represent the degree of temporal consistency across subjects in different FBNs (the thick blue line in Figure 4). The average values of group-level dynamic ISC during the entire period for seven FNBs (including visual, auditory, AC, VAS, pDSC, DC, and DAN networks) were 0.33 ± 0.12, 0.51 ± 0.13, 0.58 ± 0.12, 0.39 ± 0.12, 0.25 ± 0.10, 0.31 ± 0.08, and 0.34 ± 0.08 (Mean ± SD), respectively. While relatively high ISC values were observed in networks encompassing lower-level perceptual regions, especially those related to auditory processing, such as auditory and AC networks, the higher-order networks demonstrated lower ISC values that can indicate the occurrence of intrinsically-driven processes during individual movie viewing, including pDSC and DC networks, consistent with previous research (Ren et al., 2017b). Moreover, individual-level dynamic ISC also showed inter-subject variations especially in those higher-order networks under movie stimuli (colorful thin lines in Figure 4). Figure 4 Dynamic inter-subject correlation (ISC) (session A): group-wise and individual dynamic ISC. The thick blue line represents the group-wise dynamic ISC, and the thin colorful lines depict the dynamic ISC of 16 different individuals. The dynamic ISC of session B were largely consistent with session A (Supplementary Figure S4). Quantitatively, the PCC values for seven FBNs across two sessions were relatively high (Table 4), with an average PCC value of 0.82 ± 0.18 (Mean ± SD), thereby reaffirming the consistency of DFNs identified by our DLSC framework across two sessions. Table 4 The PCC of group-wise dynamic inter-subject correlation (ISC) across two sessions. Visual Auditory AC VAS pDSC DC DAN Mean ± SD PCC 0.89 0.63 0.87 0.79 0.76 0.86 0.83 0.81 ± 0.09 2.5. Correlations between movie ratings and individual differences in dynamic ISC of DFNs In the preceding section, there was relatively lower consistency in individual neural responses observed in higher-order brain networks, such as the pDSC and DC networks. This variability may be indicative of individual differences and unique experiences during natural viewing conditions. To investigate this assumption further, that is, exploring the potential relationship between subjective movie viewing experiences and the dynamics of DFNs, we examined whether the individual dynamic ISCs were correlated with their personal ratings of the movie. We applied an inter-subject representational similarity analysis (IS-RSA) (see Methods) to explore whether participants with similar subjective ratings also exhibited similar dynamic neural response patterns. Specifically, we employed a multidimensional scaling approach (MDS) (Carroll and Arabie, 1998) to characterize the answers to the post-movie questionnaire. Our result revealed that participants had varying experiences while watching the movie, with some reporting high engagement characterized by low boredom, high enjoyment, high emotion, and high audio quality, while others had low engagement (Figure 5A). The distances between movie ratings were measured by the Euclidean distance matrix of questionnaire answers across all individuals (Figure 5B). We computed Pearson distance to represent the inter-subject distances of dynamic ISC values for seven representative FBNs, respectively (Supplementary Figure S5). By evaluating the correlation between the movie rating distances and the inter-subject distances of the dynamic ISC, we found significant positive correlations (permutation 5,000 times, p < 0.05) in three cerebellum-related networks, that is, AC, pDSC, and DC networks. The distance matrices of dynamic ISC for these three networks are presented in Figure 5C, and their simple linear regressions are shown in Figure 5D. However, the other DFNs did not show statistically significant associations (p > 0.05) (Table 5). We did not repeat this experiment in session B as it involved a repeated viewing of the same movie, and the post-viewing questionnaire was not conducted. Figure 5 Correlation between the movie ratings and difference of individual dynamic ISC. (A) The inter-subject distances of the movie ratings were mapped onto a two-dimensional plane, with movie ratings shown in the inset and coded accordingly. The arrangement of movie ratings from left to right signifies participants’ engagement with the movie, as those who were more engaged reported higher levels of enjoyment, emotion, and audio quality and lower levels of boredom. The top-to-bottom scale reflects the participants’ ratings of evoked emotions. (B) Inter-subject distance matrix of the movie ratings. (C) Distance matrices of dynamic ISC for AC, pDSC, and DC networks. (D) The correlation between the movie rating distances and the inter-subject distances of dynamic ISC. Table 5 The Spearman correlation between movie rating distances and inter-subject distances of dynamic ISC for seven representative FBNs. Visual Auditory AC VAS pDSC DC DAN Correlation −0.02 0.07 0.15* 0.13 0.20* 0.17* 0.05 Bold font indicates significant correlation (*p < 0.05). Permutation test with 5,000 iterations. 2.6. Test–retest reliability of DFNs It was assumed that similar sensory experiences would lead to the emergence of DFNs in a consistent and reliable manner. Therefore, we assessed the level of reliability of the DFNs that develop in response to the movie storyline across two sessions. Specifically, we first calculated the scan-wise intra-group correlation coefficient (ICC) values for seven static FBNs. The results showed that the visual, AC, and VAS networks exhibited excellent reliability, the auditory and DAN networks possessed good reliability levels, and the pDSC and DC networks had moderate reliability, indicating that the networks associated with primary perceptual processes were relatively more reliable, while the higher-level networks showed less reliability, consistent with previous studies (Choe et al., 2017) (Figure 6B). Figure 6 Intra-group correlation coefficient (ICC) for seven representative FBNs. (A) Dynamic ICC. The red line represents the group-wise dynamic ICC. (B) Static ICC. We further analyzed the reliability of the DFNs (see Methods). The reliability of the auditory, AC, VAS, and DC networks, triggered by this touching movie, gradually increased during the mid to late period (about 300–390 window, corresponding to 600–1,000 s time period), and reached its peak in the near-end period. This is consistent with the narrative structure of the movie (Supplementary Table S2), wherein the plot also reaches its climax near the end (at around 17 min or 1,020 s) (Nguyen et al., 2017). The findings suggest that as the storyline develops, individuals may experience greater behavioral constraints and engagements, leading to an increase in the test–retest reliability of brain activities. 3. Discussion Brain is a complex and dynamic system, composed of different brain regions forming functional brain networks that perform different cognitive functions (Raichle, 2006; Allen et al., 2014; Monti et al., 2014; Shine et al., 2016). This study explored the dynamic functional brain networks (DFNs) involved in higher-order cognitive processes, sensory perception, and emotional responses to naturalistic stimuli. Utilizing the proposed method, the study revealed rich and complex higher-order FBNs, including cerebellum-related networks, which are challenging to detect by conventional resting-state paradigm. The spatial patterns of these FBNs were time-varying with the movie storylines, and were correlated with the movie annotations and the subjective experience of the participants. Specifically, our findings showed that two cerebellum-related networks, the DC network, and the pDSC network, were significantly connected to movie annotations. The DC network, which involves the cerebellum, posterior cingulate cortex, and precuneus region, was significantly and positively correlated with the appearance of positive facial expression during movie viewing. This finding is consistent with previous studies, which have shown that the cerebellum and posterior cingulate cortex are involved in facial emotion recognition, and that the precuneus is activated during the appearance of happy faces (Pelletier-Baldelli et al., 2015). The pDSC network, including partial DMN, salience network, and cerebellum, showed a significant positive correlation with the appearance of changepoints in movie scenes. This finding is also in line with previous studies, which have demonstrated that the changepoints in movie scenes are related to bottom-up attention, and that the salience network (SN) provides effective control of DMN activity when external event stimuli require an attentional response (Kelly et al., 2008; Menon and Uddin, 2010; Jilka et al., 2014). In addition, the cerebellum also plays a part in bottom-up attention (Gottwald et al., 2003; Kellermann et al., 2012). Overall, our results add weight to the notion that cerebellum-related DFNs are involved in higher-order cognitive and emotional processes. Furthermore, the dynamic ISC analyses demonstrate relatively low consistency in the neural responses of higher-order brain networks across individuals. This variability suggests individual differences and unique experiences during natural viewing conditions, as evidenced by the strong correlations between the subjective ratings of the movie and dynamic ISC distances of DC and pDSC networks revealed by IS-RSA analyses. Exceptionally, the AC network exhibited relatively high temporal consistency across subjects but also existed a significant correlation with individuals’ subjective movie ratings, where the high ISC values were probably caused by the involvement of large auditory regions in the AC network. This could also explain the relatively weaker correlation between the AC network and movie ratings in comparison to that of the DC and pDSC networks (Figure 5 and Table 5). Intriguingly, all these three networks involve cerebellar posterior crus1,2 and vermis regions. Previous functional connectivity studies have confirmed that there are strong links between the posterior cerebellum and the temporal lobes, suggesting these regions share neural similarities and are involved in sensory integration and emotional processing (Yeo et al., 2011; Chan et al., 2019; Heleven et al., 2019; Van Overwalle et al., 2020b). In addition, several positron emission tomography (PET) studies suggest that the posterior cerebellum is involved in various emotional responses, such as fear, sadness, and happiness (Schwartz and Davidson, 1997; Turner et al., 2007; Verger et al., 2020). Additionally, Nguyen et al. (2017) have shown that the crus1,2 areas in the posterior cerebellum exhibit peak activities during important moments embedded in the movie, and Van Overwalle et al. (2020a) have shown that the cerebellar posterior crus 2 is specialized for mentalizing appraisal processes. Our study extends these previous findings by demonstrating that the dynamic nature of cerebellum-related FBNs is significantly correlated with individual-specific emotional responses. Finally, our results also demonstrated that the DFNs elicited by ecologically valid sensory experiences were reproducible and reliable. Seven representative FBNs identified by our method were consistent across two scanning sessions with relatively-long interval (Figure 2, Supplementary Figure S1, and Table 1), and the changes in NVA and IVA for DFNs also showed high PCC values across two sessions, suggesting substantial consistency (Figures 2, 3, Supplementary Figures S2, S3, and Table 2). These results indicate that DFNs are reproducible during repeated movie viewing, further demonstrating that the naturalistic paradigm provides reliable experimental conditions for measuring DFNs. Furthermore, our results suggest that DFNs show good test–retest reliability, and the development of the movie plot enhances the test–retest reliability of the auditory, AC, VAS, and DC networks. This may reflect an increase in cognitive engagement as the storyline progresses, where the positive influences of cognitive participation on reliability appear to exceed the negative effect of familiarity from potential repeated viewings (Wang et al., 2017). Overall, our study of time-varying spatial patterns of FBNs in the context of naturalistic paradigm improves our understanding of human perception, emotion, and subjective cognition. The results highlight the reliable correlations between cerebellum-related DFNs and sensory, cognitive, emotional, and subjective senses, which could motivate further research on the neural mechanisms underlying ecologically valid sensory experiences. Thus, our study provides valuable insights into the dynamic nature of brain networks and their role in higher-order cognitive and emotional processes, with potential applications in both basic and clinical neuroscience. 4. Limitations and future directions Linking neural activity to higher cognitive and emotional functions in a dynamic and complex natural environment remains a challenge. In this work, we selected a relatively long time window of 60TRs to capture the accumulation of higher-order complex emotions and to improve the reproducibility of the FBNs (Savva et al., 2019). However, the relatively slow temporal resolution of fMRI with a large window size hinders the assessment of the responses of the brain to the perception of transient movie features. In the future, we expect to address this limitation by using electroencephalography (EEG) or magnetoencephalogram (MEG) with higher temporal resolution. While the dataset used in this study is relatively small, all individuals watched a complete movie (20 min), which has been shown to strongly stimulate higher-order cognitions and emotions (Jaaskelainen et al., 2021). To increase the accuracy and reliability of our results, we performed a second acquisition after 3 months, despite the considerable expenses incurred for the acquisition of the complete movie. Nevertheless, we acknowledge that an abundance of subjects would further strengthen our findings, and we plan to apply our model to NfMRI datasets with a larger sample size in future studies. 5. Materials and methods 5.1. Experimental paradigm The experiment consisted of two scanning sessions. Following a first session (session A) conducted 3 months earlier, participants underwent a second scanning session (session B) employing the same experimental paradigm. In each session, participants freely watched the 20-min short film “Butterfly Circus.” In addition, all participants completed a questionnaire immediately after session A. The short film, “Butterfly Circus,” depicts a touching story of a limbless man who is encouraged by the showman of a renowned circus to discover his true potential. The narrative architecture of the film follows three distinctive drama acts that feature significant developments, complications, and turning points (Supplementary Table S2). Additionally, basic movie annotations were provided, including: the use of language, changepoints, the presence of positive valence of scenes, the presence of negative valence of scenes, the presence of faces with positive, and the presence of faces with negative (Supplementary Figure S6). Further details regarding the participants can be found in the Supplementary material (1.2). 5.2. Data acquisition and preprocessing Sixteen right-handed (ages 27 ± 2.7) healthy participated in this study. FMRI images were acquired from a whole-body 3 T Siemens Trio MRI scanner with the following scanning parameters: repetition time (TR) 2,200 ms, echo time (TE) 30 ms, flip angle (FA) 79°, the field of view (FOV) 192 × 192 mm, pixel bandwidth 2,003 Hz, a 64 × 64 acquisition matrix, 44 axial slices, and 3 × 3 × 3 mm 3 voxel resolution. Functional images were preprocessed using FMRI Expert Analysis Tool (FEAT) from FMRIB’s Software Library (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki), which involved motion correction, slice timing correction, spatial smoothing with 6-mm full width at half maximum Gaussian kernel, band pass filtering (0.0085 ± 0.15 Hz), linear registration to the standard Montreal Neurological Institute space (2 mm MNI152 standard template), and masking. 5.3. Dynamic sparse representation To discover and characterize DFNs, we proposed a computational framework comprised of two stages: (A) using group-wise dictionary learning and sparse coding (DLSC) to represent static FBNs (Figure 7A), (B) sliding-window method applying for the representation of dynamic spatial patterns of FBNs (Figure 7B). Figure 7 The overview of the proposed framework. (A) Using group-wise dictionary learning and sparse coding to represent static FBNs. (B) Sliding-window method applying for the representation of dynamic individual FBNs. p, the subject number; n, the number of voxels in the brain; K, the number of atoms in the dictionary; t, the time points; L, the length of each window; w, the total number of windows generated by the sliding-window method. In stage A, first, the whole-brain fMRI signals of each subject were extracted and stacked into a 2D matrix Si(Si∈ℝt×n)), followed by spatial concatenation of the individual matrix Si into a group-wise matrix 𝑺 (Si∈ℝt×(p×n)) (top panel in Figure 7A), where t represents the time length of fMRI signals, n refers to the number of the individual brain voxels, p stands for the number of subjects. Second, we applied the dictionary learning algorithm to the group-wise matrix 𝑺 to learn a meaningful group-wise dictionary 𝑫 (D∈ℝt×K). This dictionary 𝑫 consists of K atoms that can well represent the temporal features embedded in naturalistic fMRI data and are commonly shared by all the subjects (Ren et al., 2017a; Ge et al., 2020). Hence the group-wise dictionary 𝑫 can be used to sparsely represent individual fMRI signals Si, resulting in the individual spatial patterns αi(αi∈ℝK×n) (K <  0.8), good (ICC 0.6–0.79), moderate (ICC 0.4–0.59), fair (ICC 0.2–0.39), and poor (ICC < 0.2). The test–retest reliability was assessed at the scan-wise level, and the methodology for this process was carried out in accordance with the previous study (Guo et al., 2012; Wang et al., 2017). Data availability statement The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author. Ethics statement Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study. Author contributions LS and YR contributed to the conception and design of the study. LS, YR, and JN drafted the manuscript and performed the research. LS and KW contributed to analyzing the data. YH and XH contributed to supervision, writing review, and editing. All authors contributed to the article and approved the submitted version. Funding This work was supported by the National Natural Science Foundation of China (Grant Nos. 62006187, 61971350, and 12271434), the Youth Innovation Team Foundation of Education Department of Shaanxi Province Government (Grant No. 21JP119), the China Postdoctoral Science Foundation Funded Project (Grant No. 2021M702650), the Natural Science Basic Research Program of Shaanxi (Grant No. 2023-JC-JQ-57), and the Key Research and Development Program Project of Shaanxi Province (Grant No. 2020SF-036). Conflict of interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Publisher’s note All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher. Supplementary material The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2023.1199150/full#supplementary-material Click here for additional data file. ==== Refs References Allen E. A. Damaraju E. Plis S. M. Erhardt E. B. Eichele T. Calhoun V. D. (2014). Tracking whole-brain connectivity dynamics in the resting state. Cereb. Cortex 24 , 663–676. doi: 10.1093/cercor/bhs352, PMID: 23146964 Barrett L. F. Satpute A. B. (2013). Large-scale brain networks in affective and social neuroscience: towards an integrative functional architecture of the brain. Curr. Opin. Neurobiol. 23 , 361–372. doi: 10.1016/j.conb.2012.12.012, PMID: 23352202 Belitski A. Gretton A. Magri C. Murayama Y. Montemurro M. A. Logothetis N. K. . (2008). Low-frequency local field potentials and spikes in primary visual cortex convey independent visual information. J. Neurosci. 28 , 5696–5709. doi: 10.1523/JNEUROSCI.0009-08.2008, PMID: 18509031 Benesty J. Chen J. Huang Y. Cohen I. (2009). “Pearson correlation coefficient” in Noise reduction in speech processing. eds. Cohen I. Huang Y. Chen J. Benesty J. (Berlin, Heidelberg: Springer), 1–4. Buckner R. L. Krienen F. M. Yeo B. T. (2013). Opportunities and limitations of intrinsic functional connectivity MRI. Nat. Neurosci. 16 , 832–837. doi: 10.1038/nn.3423 23799476 Calhoun V. D. Miller R. Pearlson G. Adali T. (2014). The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron 84 , 262–274. doi: 10.1016/j.neuron.2014.10.015, PMID: 25374354 Carroll J. D. Arabie P. (1998). “Multidimensional scaling” in Measurement, judgment and decision making. ed. Birnbaum M. H. (Cambridge, MA: Academic Press), 179–250. Chan H. Y. Smidts A. Schoots V. C. Dietvorst R. C. Boksem M. A. S. (2019). Neural similarity at temporal lobe and cerebellum predicts out-of-sample preference and recall for video stimuli. NeuroImage 197 , 391–401. doi: 10.1016/j.neuroimage.2019.04.076, PMID: 31051296 Choe A. S. Nebel M. B. Barber A. D. Cohen J. R. Xu Y. Pekar J. J. . (2017). Comparing test-retest reliability of dynamic functional connectivity methods. NeuroImage 158 , 155–175. doi: 10.1016/j.neuroimage.2017.07.005, PMID: 28687517 Di X. Biswal B. B. (2020). Intersubject consistent dynamic connectivity during natural vision revealed by functional MRI. NeuroImage 216 :116698. doi: 10.1016/j.neuroimage.2020.116698, PMID: 32130972 Finn E. S. Glerean E. Khojandi A. Y. Nielson D. Molfese P. J. Handwerker D. A. . (2020). Idiosynchrony: from shared responses to individual differences during naturalistic neuroimaging. NeuroImage 215 :116828. doi: 10.1016/j.neuroimage.2020.116828, PMID: 32276065 Friston K. J. Holmes A. P. Worsley K. J. Poline J. P. Frith C. D. Frackowiak R. S. (1994). Statistical parametric maps in functional imaging: a general linear approach. Hum. Brain Mapp. 2 , 189–210. doi: 10.1002/hbm.460020402 Ge F. Lv J Hu X. Guo L. Han J. Zhao S. . (2018). Exploring intrinsic networks and their interactions using group wise temporal sparse coding. In International Symposium on Biomedical Imaging (ISBI 2018) (Washington, D.C., USA, IEEE Xplore). Ge B. Wang H. Wang P. Tian Y. Zhang X. Liu T. (2020). Discovering and characterizing dynamic functional brain networks in task FMRI. Brain Imaging Behav. 14 , 1660–1673. doi: 10.1007/s11682-019-00096-6, PMID: 31011948 Golland Y. Bentin S. Gelbard H. Benjamini Y. Heller R. Nir Y. . (2007). Extrinsic and intrinsic systems in the posterior cortex of the human brain revealed during natural sensory stimulation. Cereb. cortex 17 , 766–777. doi: 10.1093/cercor/bhk030, PMID: 16699080 Gottwald B. Mihajlovic Z. Wilde B. Mehdorn H. M. (2003). Does the cerebellum contribute to specific aspects of attention? Neuropsychologia 41 , 1452–1460. doi: 10.1016/S0028-3932(03)00090-3 12849763 Guo C. C. Kurth F. Zhou J. Mayer E. A. Eickhoff S. B. Kramer J. H. . (2012). One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. NeuroImage 61 , 1471–1483. doi: 10.1016/j.neuroimage.2012.03.027, PMID: 22446491 Hasson U. Nir Y. Levy I. Fuhrmann G. Malach R. (2004). Intersubject synchronization of cortical activity during natural vision. Science 303 , 1634–1640. doi: 10.1126/science.1089506 15016991 Heleven E. van Dun K. Van Overwalle F. (2019). The posterior cerebellum is involved in constructing social action sequences: an fMRI study. Sci. Rep. 9 , 1–11. doi: 10.1038/s41598-019-46962-7 30626917 Hutchison R. M. Womelsdorf T. Allen E. A. Bandettini P. A. Calhoun V. D. Corbetta M. . (2013). Dynamic functional connectivity: promise, issues, and interpretations. NeuroImage 80 , 360–378. doi: 10.1016/j.neuroimage.2013.05.079, PMID: 23707587 Jaaskelainen I. P. Sams M. Glerean E. Ahveninen J. (2021). Movies and narratives as naturalistic stimuli in neuroimaging. NeuroImage 224 :117445. doi: 10.1016/j.neuroimage.2020.117445, PMID: 33059053 Jilka S. R. Scott G. Ham T. Pickering A. Bonnelle V. Braga R. M. . (2014). Damage to the salience network and interactions with the default mode network. J. Neurosci. 34 , 10798–10807. doi: 10.1523/JNEUROSCI.0518-14.2014 25122883 Kellermann T. Regenbogen C. De Vos M. Mößnang C. Finkelmeyer A. Habel U. (2012). Effective connectivity of the human cerebellum during visual attention. J. Neurosci. 32 , 11453–11460. doi: 10.1523/JNEUROSCI.0678-12.2012, PMID: 22895727 Kelly A. C. Uddin L. Q. Biswal B. B. Castellanos F. X. Milham M. P. (2008). Competition between functional brain networks mediates behavioral variability. NeuroImage 39 , 527–537. doi: 10.1016/j.neuroimage.2007.08.008, PMID: 17919929 Kriegeskorte N. Mur M. Bandettini P. A. (2008). Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2 :4. doi: 10.3389/neuro.06.004.2008 19104670 Liegeois R. Laumann T. O. Snyder A. Z. Zhou J. Yeo B. T. T. (2017). Interpreting temporal fluctuations in resting-state functional connectivity MRI. NeuroImage 163 , 437–455. doi: 10.1016/j.neuroimage.2017.09.012, PMID: 28916180 Lurie D. J. Kessler D. Bassett D. S. Betzel R. F. Breakspear M. Kheilholz S. . (2020). Questions and controversies in the study of time-varying functional connectivity in resting fMRI. Netw. Neurosci. 4 , 30–69. doi: 10.1162/netn_a_00116, PMID: 32043043 Lv J. (2013). Identifying functional networks via sparse coding of whole brain FMRI signals. In Annual International IEEE EMBS Conference on Neural Engineering (San Diego, California, IEEE), pp. 778–781. Lv J. Jiang X. Li X. Zhu D. Chen H. Zhang T. . (2015a). Sparse representation of whole-brain fMRI signals for identification of functional networks. Med. Image Anal. 20 , 112–134. doi: 10.1016/j.media.2014.10.011, PMID: 25476415 Lv J. Jiang X. Li X. Zhu D. Zhang S. Zhao S. . (2015b). Holistic atlases of functional networks and interactions reveal reciprocal organizational architecture of cortical function. IEEE Trans. Biomed. Eng. 62 , 1120–1131. doi: 10.1109/TBME.2014.2369495 25420254 Ma S. Calhoun V. D. Phlypo R. Adalı T. (2014). Dynamic changes of spatial functional network connectivity in healthy individuals and schizophrenia patients using independent vector analysis. NeuroImage 90 , 196–206. doi: 10.1016/j.neuroimage.2013.12.063, PMID: 24418507 McGraw K. O. Wong S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1 , 30–46. doi: 10.1037/1082-989X.1.1.30 Meer J. N. V. Breakspear M. Chang L. J. Sonkusare S. Cocchi L. (2020). Movie viewing elicits rich and reliable brain state dynamics. Nat. Commun. 11 :5004. doi: 10.1038/s41467-020-18717-w, PMID: 33020473 Menon V. Uddin L. Q. (2010). Saliency, switching, attention and control: a network model of insula function. Brain Struct. Funct. 214 , 655–667. doi: 10.1007/s00429-010-0262-0, PMID: 20512370 Monti R. P. Hellyer P. Sharp D. Leech R. Anagnostopoulos C. Montana G. (2014). Estimating time-varying brain connectivity networks from functional MRI time series. NeuroImage 103 , 427–443. doi: 10.1016/j.neuroimage.2014.07.033 25107854 Nguyen V. T. Sonkusare S. Stadler J. Hu X. Breakspear M. Guo C. C. (2017). Distinct cerebellar contributions to cognitive-perceptual dynamics during natural viewing. Cereb. Cortex 27 , 5652–5662. doi: 10.1093/cercor/bhw334, PMID: 29145671 Park H. J. Friston K. (2013). Structural and functional brain networks: from connections to cognition. Science 342 :1238411. doi: 10.1126/science.1238411 24179229 Pelletier-Baldelli A. Bernard J. A. Mittal V. A. (2015). Intrinsic functional connectivity in salience and default mode networks and aberrant social processes in youth at ultra-high risk for psychosis. PLoS One 10 :e0134936. doi: 10.1371/journal.pone.0134936, PMID: 26252525 Ponce J. M. F. B. J. Sapiro G. (2010). Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11 , 19–60. doi: 10.48550/arXiv.0908.0050 Preti M. G. Bolton T. A. W. Van De Ville D. (2017). The dynamic functional connectome: state-of-the-art and perspectives. NeuroImage 160 , 41–54. doi: 10.1016/j.neuroimage.2016.12.061, PMID: 28034766 Raichle M. E. (2006). The brain's dark energy. Science 314 , 1249–1250. doi: 10.1126/science.1134405, PMID: 17124311 Ren Y. Lv J. Guo L. Fang J. Guo C. C. (2017a). Sparse coding reveals greater functional connectivity in female brains during naturalistic emotional experience. PLoS One 12 :e0190097. doi: 10.1371/journal.pone.0190097, PMID: 29272294 Ren Y. Nguyen V. T. Guo L. Guo C. C. (2017b). Inter-subject functional correlation reveal a hierarchical organization of extrinsic and intrinsic systems in the brain. Sci. Rep. 7 :10876. doi: 10.1038/s41598-017-11324-8, PMID: 28883508 Rubinov M. Sporns O. (2010). Complex network measures of brain connectivity: uses and interpretations. NeuroImage 52 , 1059–1069. doi: 10.1016/j.neuroimage.2009.10.003, PMID: 19819337 Saarimaki H. (2021). Naturalistic stimuli in affective neuroimaging: a review. Front. Hum. Neurosci. 15 :675068. doi: 10.3389/fnhum.2021.675068, PMID: 34220474 Savva A. D. Mitsis G. D. Matsopoulos G. K. (2019). Assessment of dynamic functional connectivity in resting-state fMRI using the sliding window technique. Brain Behav. 9 :e01255. doi: 10.1002/brb3.1255, PMID: 30884215 Schwartz G. E. Davidson R. J. (1997). Neuroanatomical correlates of happiness, sadness, and disgust. Am. J. Psychiatry 154 , 926–933. doi: 10.1176/ajp.154.7.926 9210742 Shine J. M. Bissett P. G. Bell P. T. Koyejo O. Balsters J. H. Gorgolewski K. J. . (2016). The dynamics of functional brain networks: integrated network states during cognitive task performance. Neuron 92 , 544–554. doi: 10.1016/j.neuron.2016.09.018, PMID: 27693256 Shrout P. E. Fleiss J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 86 , 420–428. doi: 10.1037/0033-2909.86.2.420 18839484 Simony E. Honey C. J. Chen J. Lositsky O. Yeshurun Y. Wiesel A. . (2016). Dynamic reconfiguration of the default mode network during narrative comprehension. Nat. Commun. 7 :12141. doi: 10.1038/ncomms12141, PMID: 27424918 Sonkusare S. Breakspear M. Guo C. (2019). Naturalistic stimuli in neuroscience: critically acclaimed. Trends Cogn. Sci. 23 , 699–714. doi: 10.1016/j.tics.2019.05.004 31257145 Tagliazucchi E. Laufs H. (2014). Decoding wakefulness levels from typical fMRI resting-state data reveals reliable drifts between wakefulness and sleep. Neuron 82 , 695–708. doi: 10.1016/j.neuron.2014.03.020, PMID: 24811386 Tian L. Ye M. Chen C. Cao X. Shen T. (2021). Consistency of functional connectivity across different movies. NeuroImage 233 :117926. doi: 10.1016/j.neuroimage.2021.117926, PMID: 33675997 Tononi G. Sporns O. Edelman G. M. (1996). A complexity measure for selective matching of signals by the brain. Proc. Natl. Acad. Sci. 93 , 3422–3427. doi: 10.1073/pnas.93.8.3422, PMID: 8622951 Turner B. M. Paradiso S. Marvel C. L. Pierson R. Ponto L. L. B. Hichwa R. D. . (2007). The cerebellum and emotional experience. Neuropsychologia 45 , 1331–1341. doi: 10.1016/j.neuropsychologia.2006.09.023, PMID: 17123557 Tzachor I. Hoshen Y. (2022). Window projection features are all you need for time series anomaly detection. In ICLR 2023 Conference Withdrawn Submission. Van Dijk K. R. Sabuncu M. R. Buckner R. L. (2012). The influence of head motion on intrinsic functional connectivity MRI. NeuroImage 59 , 431–438. doi: 10.1016/j.neuroimage.2011.07.044, PMID: 21810475 Van Overwalle F. Ma Q. Heleven E. (2020a). The posterior crus II cerebellum is specialized for social mentalizing and emotional self-experiences: a meta-analysis. Soc. Cogn. Affect. Neurosci. 15 , 905–928. doi: 10.1093/scan/nsaa124, PMID: 32888303 Van Overwalle F. Van de Steen F. van Dun K. Heleven E. (2020b). Connectivity between the cerebrum and cerebellum during social and non-social sequencing using dynamic causal modelling. NeuroImage 206 :116326. doi: 10.1016/j.neuroimage.2019.116326 31678499 Verger A. Rousseau P. F. Malbos E. Chawki M. B. Nicolas F. Lancon C. . (2020). Involvement of the cerebellum in EMDR efficiency: a metabolic connectivity PET study in PTSD. Eur. J. Psychotraumatol. 11 :1767986. doi: 10.1080/20008198.2020.1767986, PMID: 33029312 Wang J. Ren Y. Hu X. Nguyen V. T. Guo L. Han J. . (2017). Test-retest reliability of functional connectivity networks during naturalistic fMRI paradigms. Hum. Brain Mapp. 38 , 2226–2241. doi: 10.1002/hbm.23517, PMID: 28094464 Yeo B. T. Krienen F. M. Sepulcre J. Sabuncu M. R. Lashkari D. Hollinshead M. . (2011). The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106 , 1125–1165. doi: 10.1152/jn.00338.2011 21653723 Zhang S. Li X. Guo L. Liu T. (2017). Exploring human brain activation via nested sparse coding and functional operators. In International Symposium on Biomedical Imaging (ISBI 2017) (Melbourne, VIC, Australia, IEEE). Zhang X. Liu J. Yang Y. Zhao S. Guo L. Han J. . (2021). Test–retest reliability of dynamic functional connectivity in naturalistic paradigm functional magnetic resonance imaging. Hum. Brain Mapp. 43 , 1463–1476. doi: 10.1002/hbm.25736 34870361