
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39266687
72112
10.1038/s41598-024-72112-9
Article
Brain activities in the auditory area and insula represent stimuli evoking emotional response
Tsushima Yoshiaki tsushima@nict.go.jp

1
Nakayama Koharu 2
Okuya Teruhisa 3
Koiwa Hiroko 4
Ando Hiroshi 15
Watanabe Yoshiaki 2
1 https://ror.org/016bgq349 grid.28312.3a 0000 0001 0590 0962 Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), 1-4 Yamadaoka, Suita, Osaka 565-0871 Japan
2 https://ror.org/01fxdkm29 grid.255178.c 0000 0001 2185 2753 Faculty of Life and Medical Sciences, Doshisha University, 1-3 Tataramiyakodani, Kyotanabe, Kyoto 610-0321 Japan
3 grid.410834.a 0000 0004 0447 7842 Panasonic Holdings Corporation, 3-1-1 Yagumo-Naka-Machi, Moriguchi City, Osaka 570-8501 Japan
4 grid.410834.a 0000 0004 0447 7842 Electric Works Company, Panasonic Corporation, Kadoma, Osaka 571-8686 Japan
5 https://ror.org/016bgq349 grid.28312.3a 0000 0001 0590 0962 Universal Communication Research Institue, National Institute of Information and Communications Technology (NICT), 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0289 Japan
12 9 2024
12 9 2024
2024
14 2133520 3 2024
3 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Cinema, a modern titan of entertainment, holds power to move people with the artful manipulation of auditory and visual stimuli. Despite this, the mechanisms behind how sensory stimuli elicit emotional responses are unknown. Thus, this study evaluated which brain regions were involved when sensory stimuli evoke auditory- or visual-driven emotions during film viewing. Using functional magnetic resonance imaging (fMRI) decoding techniques, we found that brain activities in the auditory area and insula represent the stimuli that evoke emotional response. The observation of brain activities in these regions could provide further insights to these mechanisms for the improvement of film-making, as well as the development of novel neural techniques in neuroscience. In near feature, such a “neuro-designed” products/ applications might gain in popularity.

Subject terms

Perception
Human behaviour
http://dx.doi.org/10.13039/501100012389 National Institute of Information and Communications Technology Doshisha University, Doshisha University, JapanPanasonic Holdings CorporationElectric Works Company, Panasonic Corporation, Japanissue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

As one of the most influential mediums of auditory and visual art, cinema holds power to evoke emotions. In fact, the use of natural sceneries with appropriate melodies can already evoke an emotional response, even without relying on dialogue. The mechanisms behind this response begs the question on how to improve emotional response in film-making. Thus, studies have been conducted to evaluate which stimuli primarily evokes emotions during film viewing, whether auditory- or visual-driven1,2.

Recent studies utilizing functional magnetic resonance imaging (fMRI), particularly voxel-wise encoding and decoding models, have revealed the relationship between brain activities and comprehensive cognitive states3,4. For one, Kragel et al. demonstrated that certain emotional states can be decoded from brain patterns3. Numerous studies have similarly showed that visual and auditory experiences can be observed from the brain activities of the audience using fMRI decoding techniques5–8. Nishimoto et al. further revealed that cinema-evoked visual images can be reconstructed from specific brain patterns6. Given these advancements in fMRI decoding techniques, it is possible to determine which sensory stimulus drives emotion, audition, and vision during film viewing.

In this study, we explored whether fMRI decoding techniques can identify which sensory stimulus evokes emotional responses during film viewing. We hypothesized that the neural processing of underlying emotional responses would exhibit distinct patterns in specific brain regions depending on whether the emotion was auditory- or visual-driven. To test the hypothesis, we conducted the two main experiments—one inside the fMRI scanner and outside (Fig. 1). First, participants watched movies featuring natural sceneries and piano music while undergoing fMRI. To ensure alertness, that is keeping their eyes and ears open, participants also performed a simple audio-visual task (See Methods). Once outside the fMRI scanner, participants revisited the same movies and reported their evoked emotional feelings using Russell’s Circumplex model9. In addition, they indicated if the emotions were auditory- or visual-driven. Based on these results, we categorized brain activity as auditory- or visual-driven emotion (See Methods). Certain portions of the data were selected for fMRI model training, and remaining data were used to test if the fMRI decoder model can distinguish stimuli that evoke auditory- and visual-driven emotions (Figs. 1 and 2). Furthermore, we investigated what brain regions successfully decoded the emotional responses, allowing use to observe the neural processing of audiovisual emotions10; See Methods.Fig. 1 Experimental scheme. Participants watched selected movies inside and outside the fMRI scanner. Based on the subjective film-viewing evaluations outside the fMRI scanner, data were initially categorized as either auditory-driven (AD) or visual-driven (VD), and subsequently divided into the training and testing datasets for the fMRI decoder model10–13. The accuracy and predictive ability of the model to distinguish AD and VD emotional states was then evaluated.

Fig. 2 Experimental results and analyses. (a) The mean ratio of visual-driven emotions for all movies. Error bars show standard errors. Data were sorted in ascending order and categorized at the center as either auditory- or visual-driven. (b) Schematic illustration of cross-validation for creating the fMRI decoder in this study (four-fold crass-validation).

Results

Prior to emotional decoding, we analyzed the behavioral data to verify the alertness of participants during the fMRI experiment (See Methods). Utilizing data from the simple audiovisual task, we calculated mean accuracy as detecting target signals (hearing a beep or seeing a change in color) within a 2-s window. Data revealed a mean accuracy of 88.2 ± 2.9% across 288 trials (4 trials/movie × 24 movies × 3 times repetition), indicating that participants were sufficiently engaged and attentive throughout the session.

We then analyzed the subjective ratio of emotional driven stimulus that was taken after the fMRI experiment. Interestingly, one-way ANOVA showed that the distribution of these subjective ratings in the behavioral experiment outside the fMRI scanner differed significantly from the pre-experiment movie assignment (F [23,264], 1.57; p, 0.04; partial η2, 0.11) (Fig. 2a, top bar graph). Therefore, we divided the movies into two categories, AD and VD (Fig. 2a, bottom bar graph), which served as labels for training the fMRI decoding models (Fig. 1).

fMRI decoding analyses showed significantly higher mean decoding accuracies in the auditory area (trained and chance-level: t(11), 4.73; p, 0.00031; r, 0.82) and insula trained and chance-level: t(11), 3.55; p, 0.0023; r, 0.73), as compared to chance-level. Conversely, the mean accuracies in the visual areas and amygdala were not significant (Fig. 3).Fig. 3 Box plots for decoding accuracies in the visual areas, auditory area, insular cortices, and amygdala. The boxes represent 25 ~ 75% of the data distribution, and vertical bars indicate the lowest and highest values. Red plots represent the mean decoding accuracies of the trained decoding model (Mean data points: 50.6, 61.9*, 57.7*, and 49.1| Visual areas, Auditory area, Insula, and Amygdala respectively). Green plots show the mean accuracies with the same set of training and testing data, but training data were randomly labeled randomly as auditory- or visual-driven. Asterisks indicate that the decoding accuracies of the trained model were significantly higher than that of the random model and chance-level. The blue dashed line represents the chance-level accuracy (50%).

To rule out the possibility of chance, we compared the mean accuracies of the trained model with that of random model, which was made using random labels. As compared to the random model, significantly higher mean accuracies in the auditory area (trained and random: t(11), 2.81; p, 0.0085; r, 0.65) and insula (trained and random: t(11), 2.36; p, 0.020; r, 0.58) were observed in the trained model. For further confirmation in the off-chance that using only the mean subjective evaluations influenced the results (Fig. 2a, bottom bar graph), we also calculated the mean accuracies for both brain regions using individual subjective evaluations, revealing similar tendencies as with mean subjective evaluations (See Supplementary Analysis 1 at Supplementary Information). Given these findings, brain activities in the auditory area and insula appeared to represent which stimuli (auditory or visual) evoked emotional responses during film viewing.

Discussion

In this study, we utilized fMRI decoding techniques to examine brain regions that predicted auditory- or vision-driven emotions during film viewing. Results showed that brain activities in both the auditory area and insula represented relatively which stimulus (auditory or visual) relatively elicited an emotional response, suggesting their involvement in neural processing during film viewing. Our findings were consistent with previous studies on human and non-human primates that showed wide reciprocal connections between the two regions14–18. For instance, Zhang et al. showed and argued significant roles of the insula and auditory areas in emotional processing, and also suggested that insula play a crucial role in conscious perception of emotions14,15. These suggest that observing brain activity in the auditory area and insula allows us to objectively determine if the emotional response evoked was auditory- or visual-driven. In the cinematic arts, directors would be able to utilize these findings to decide on what music and sceneries to present in their films. In contrast, the present results revealed that brain activities in the visual areas and amygdala did not represent which stimulus mainly elicits the emotion, audition or vision. This is likely due to the specialized functions of these regions, such as the visual areas (V1-V4) which basically process visual information (e.g. luminance-contrast, shape) without making connections to emotional value. Moreover, auditory stimuli have been associated with greater influence over emotional response, as compared to visual stimuli2,19. Therefore, it seems logical that the visual areas were not relatively involved in the processing of stimulus-driven emotion. It may be also convincing that the brain activities in the amygdala did not represent the stimulus-driven emotion, mean decoding accuracies. The lack of decoding accuracy in the amygdala may also be logical, since neural responses in this region correlate with fundamental emotional responses, including fear and anxiety20–24, rather than the stimulus of the emotion.

One might think the reason why brain activity in the auditory area represented the stimulus in emotional response is that the auditory area was just stimulated by basic sound features, such as sound volume, tempo, and tonality25,26. To confirm this possibility, we analyzed and compared these sound features (sound volume, tempo, and tonality) using the MIRtoolbox (https://www.mathworks.com/matlabcentral/fileexchange/24583-mirtoolbox), consequently identifying the ratio of stimulus-driven emotions (Fig. 2a; “AD” and “VD” labels for decoding analyses). Results of this analysis revealed no significant correlations, indicating that basic sound features did not influence the emotional response of the participants (See Supplementary Analysis 2 at Supplementary Information).

The present study offers valuable insights for personalizing and enhancing the audio-visual experiences of people. For instance, monitoring the brain activity using electroencephalography would allow the creation of advanced and comfortable audio-visual environments. Studies have shown the growing popularity of such “neuro-designed” services and products in near future27,28. Someday, our brain might select our favorites instead of “us”. This idea would also contribute to the controversial aspect of consciousness29,30.

Despite the significant findings of this study, the decoding accuracy values highest values only reached 70–75%. Further research is required to establish more effective fMRI decoding methods to address this. Increasing the number of training sessions or using functional ROI may be helpful in constructing a more accurate decoder and performing more detailed analyses. Furthermore, we should reconsider scientific approach to complex human emotion as well as the technical methods of fMRI brain decoding. In addition, we here explained the limitations of generalizability in the current study. Although we had statistical and neuroscientific important results, we tested only some specific people: They were twelve participants from twenty to thirty-nine years old. We should test more numbers of and more variety of people if we want much more generalized findings.

In conclusion, the current results highlight fMRI decoding techniques as an excellent tool for determining which stimulus evoked an emotional response during film viewing, providing new insights into the mechanism of stimulus-driven emotion.

Methods

Participants

A total of twelve individuals (six males, six females| It was assumed to be no effect of gender difference in this kind of emotion and perception experiment, but in case it doesn’t, we collected equal number of men and women.) with normal/corrected vision and normal hearing participated in the study (mean age, 26.58; SD, 3.23; range, 20–39| It was not predicted to be effect of aging in this perception experiment, but we avoided the possibility of perceptually developing or declining by aging.). All participants gave written informed consent, and our study was approved by the Ethics Committee of the National Institute of Information and Communications Technology, Japan in compliance with the Declaration of Helsinki.

Stimuli

The experiment employed 24 movies presented on a screen inside the fMRI scanner. Each movie comprised a grayscale natural scene paired with piano music. The selection of the scenes and music were based on the results of the preliminary experiment (See some samples in Supplementary Information). We avoided using too familiar or/and too bizarre stimuli, because they might have a possibility of bias affecting the participants’ performances. Also, we used these stimuli in our previous studies and pilot testing, because it seemed to be no biases. Multimedia elements were sourced from the following royalty-free libraries: PIXTA (https://pixta.jp/), Panasonic (https://www2.panasonic.biz/jp/lighting/shop/effect-projector/spaceplayer/contents/free/), and SOUND-JUKE (https://www.sound-juke.com/). Each movie was presented for 40 s and repeated three times in a random order. Between movies, a white fixation point on a gray background was presented (Fig. 4a). While inside the fMRI scanner, participants viewed the movies through a mirror mounted on the head coil, ensuring their gaze remained centered on the fixation point. For audio reception, they wore MRI compatible headphones throughout the experiment.Fig. 4 Experiments inside and outside of the fMRI scanner. (a) Inside the fMRI scanner, each movie was presented for 40 s with a 4-s interval in between. While watching, participants were required to press a button when seeing a red circle in the center or hearing a beep sound. These audio-visual targets randomly appeared every 10 s. (b) Outside the fMRI scanner, participants rewatched the movies and were asked regarding their evoked emotional responses using Russell’s Circumplex model. Participants also reported which sensory stimulus (auditory or visual) evoked that emotional response.

While outside of fMRI, they just viewed and listened to the audio-visual stimuli that were identical to the stimuli inside the fMRI (Fig. 4b). They reported their evoked emotional feelings using Russell’s Circumplex model9 and indicated if the emotions were auditory- or visual-driven in in 100% scale (Fig. 2a).

Behavioral task inside the fMRI scanner

To ensure alertness (keeping their eyes and ears open) while minimizing direct influence in their viewing, participants were tasked to perform a simple audio-visual task. Specifically, participants were instructed to press a button if they observed any of the target signals—color change in the central fixation point (white to red) or a brief beeping sound. For each 40-s film viewing, four randomly timed target signals were shown (Fig. 4a).

Behavioral task outside the fMRI scanner

Following testing inside the fMRI scanner, participants rewatched the same movies. At this time, they were asked to answer their emotional response using Russell’s Circumplex model9 and report which sensory stimulus (auditory or visual) evoked that emotion in a scale of 1–100% (e.g., 70% audition and 30% vision) (Fig. 2a).

fMRI data acquisition

MRI data were acquired using a 3T MRI scanner (Magnetom Prisma; Siemens, Erlangen, Germany) equipped with a 12-channel head coil at the Center for Information and Neural Networks (Osaka, Japan). To achieve high spatial resolution whole-brain fMRI with a standard time of repetition (TR), multi-band echo planar imaging by CMRR (https://www.cmrr.umn.edu/multiband/) was utilized. Interleaved T2*-weighted gradient-echo planar imaging (EPI) was specifically performed to acquire functional images covering the entire brain (TR, 2000 ms; TE, 30 ms; flip angle, 75°; interleaved; multi-band acceleration factor = 2, partial Fourier = 6/8; voxel size, 3 × 3 × 3 mm; number of slices, 50). Furthermore, T1-weighted anatomical images by MP-RAGE were collected (TR, 1900 ms; TE, 2.48 ms; flip angle, 9°; voxel size, 1 × 1 × 1 mm). Afterwards, field map imaging was performed with a double-echo spoiled gradient echo sequence (gre_field_map; TR, 753.0 ms; TE, 4.92/7.38 ms; voxel size, 3 × 3 × 3 mm with 0.8-mm gaps; flip angle, 90°), generating a magnitude and two-phase images, which were essential for field map computation.

fMRI data analyses

Image pre-processing and statistical analyses were conducted using SPM12 (Wellcome Trust Centre for Neuroimaging, UCL, SPM), and field map correction was applied, subsequently discarding the first 8 s of each scan to account for instability at the start of data acquisition. Acquired fMRI data underwent slice-timing and motion correction, and results were co-registered to the within-session, high-resolution, anatomical images (matched slides used for EPI) and the whole-head, high-resolution, anatomical image. Images were then spatially normalized to the Montreal Neurological Institute template, resampling the voxels to a uniform size of 3 × 3 × 3 mm, and underwent linear trend removal within each run. Following normalization and smoothing, amplitude normalization relative to the mean amplitude in each run was determined to minimize the baseline difference across runs. To compensate for hemodynamic delays, fMRI signals of each voxel were averaged within each stimulus block after shifting the data by 4 s. Specifically, data samples were created by averaging the fMRI volumes within each 40-s stimulus block (average, 20 volumes) and by sorting voxels into each region of interest (ROI). Anatomical labels for ROI analysis were defined using the SPM anatomy toolbox (SPM Anatomy Toolbox, version 3.0)).

fMRI decoding analyses

We used "svc", as a support vector machine (SVM) implementation of the decoder, with the default value of a soft margin parameter (C = 1) in the parameter settings, in which the scikit-learn Python toolbox was employed for the decoding (https://scikit-learn.org/stable/). We attached the sample code in Supplementary Information. In this study, fMRI decoders were trained to classify the sensory stimuli that evoke emotional response by categorizing data samples as “auditory-driven (AD)” or “visual-driven (VD)” based on participants’ subjective evaluations outside the fMRI scanner. To compensate for variability in the specific training and test dataset choices, the decoding performance was computed following the four-fold cross-validation method, which used 25% of the data left out in the study (Fig. 2b) (Because the previous study suggested that 20% ~ of the left data enables better estimates with less computation13). This was repeated 10 times with the shuffled data splits to ensure generalizability (Fig. 2b). In case, some readers are interested in the other possible cross-validation in this study (we basically could test only 12-fold (8.3% of the data left), sixfold (16.7%), fourfold (25%| Fig. 2), and twofold (50%), because we have to choose equal numbers of data as the Training data from each AD and VD), we presented them at Supplementary Analysis 3 in Supplementary Information.

To evaluate the success of the decoder in predicting the primary stimulus of the evoked emotion, separate decoding accuracies were calculated for each participant and specific ROIs as defined by the SPM anatomy toolbox (version 3.0)10, including the visual areas (from visual area 1 to 4), auditory area (from TE1 to 5), insula, and amygdala. These brain regions have been indicated to play significant roles in the perceptual and emotional processing of audio-visual stimuli during film viewing5–8,14–18.

Supplementary Information

Supplementary Information.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-72112-9.

Acknowledgements

We would like to thank Shizuko Hiryu for her academic supports, and the medical staff members of Koganei Rehabilitation hospital and Nishi-Tokyo central hospital for their amazing professionalism.

Author contributions

All authors conceived of the research idea and designed the experiment. T.O., K.N., and Y.T. conducted the experiments and analyzed the obtained data. All authors discussed the results and commented on the manuscript.

Data availability

The all experimental and analytical datasets used and/or generated during the current study available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Smith GM Film Structure and the Emotion System 2003 Cambridge University Press
Smith, G. M. Film Structure and the Emotion System (Cambridge University Press, 2003).
2. Buhler J Neumeyer D Deemer R Hearing the movies: Music and sound in film history J. Film Music 2011 3 2 183 185 10.1558/jfm.v3i2.183
Buhler, J., Neumeyer, D. & Deemer, R. Hearing the movies: Music and sound in film history. J. Film Music 3(2), 183–185. 10.1558/jfm.v3i2.183 (2011).10.1558/jfm.v3i2.183
3. Kragel PA Knodt AR Hariri AR LaBar KS Decoding spontaneous emotional states in the human brain PLoS Biol. 2016 14 9 e2000106 10.1371/journal.pbio.2000106 27627738
Kragel, P. A., Knodt, A. R., Hariri, A. R. & LaBar, K. S. Decoding spontaneous emotional states in the human brain. PLoS Biol. 14(9), e2000106. 10.1371/journal.pbio.2000106 (2016).27627738 10.1371/journal.pbio.2000106
4. Nakai T Nishimoto S Quantitative models reveal the organization of diverse cognitive functions in the brain Nat. Commun. 2020 11 1 1142 10.1038/s41467-020-14913-w 32123178
Nakai, T. & Nishimoto, S. Quantitative models reveal the organization of diverse cognitive functions in the brain. Nat. Commun. 11(1), 1142. 10.1038/s41467-020-14913-w (2020).32123178 10.1038/s41467-020-14913-w
5. Naselaris T Kay KN Nishimoto S Gallant JL Encoding and decoding in fMRI Neuroimage 2011 56 2 400 410 10.1016/j.neuroimage.2010.07.073 20691790
Naselaris, T., Kay, K. N., Nishimoto, S. & Gallant, J. L. Encoding and decoding in fMRI. Neuroimage 56(2), 400–410. 10.1016/j.neuroimage.2010.07.073 (2011).20691790 10.1016/j.neuroimage.2010.07.073
6. Nishimoto S Vu AT Naselaris T Benjamini Y Yu B Gallant JL Reconstructing visual experiences from brain activity evoked by natural movies Curr. Biol. 2011 21 19 1641 1646 10.1016/j.cub.2011.08.031 21945275
Nishimoto, S. et al. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 21(19), 1641–1646. 10.1016/j.cub.2011.08.031 (2011).21945275 10.1016/j.cub.2011.08.031
7. Vetter P Smith FW Muckli L Decoding sound and imagery content in early visual cortex Curr. Biol. 2014 24 11 1256 1262 10.1016/j.cub.2014.04.020 24856208
Vetter, P., Smith, F. W. & Muckli, L. Decoding sound and imagery content in early visual cortex. Curr. Biol. 24(11), 1256–1262. 10.1016/j.cub.2014.04.020 (2014).24856208 10.1016/j.cub.2014.04.020
8. Bailey KM Giordano BL Kaas AL Smith FW Decoding sounds depicting hand-object interactions in primary somatosensory cortex Cereb. Cortex 2023 33 7 3621 3635 10.1093/cercor/bhac296 36045002
Bailey, K. M., Giordano, B. L., Kaas, A. L. & Smith, F. W. Decoding sounds depicting hand-object interactions in primary somatosensory cortex. Cereb. Cortex 33(7), 3621–3635. 10.1093/cercor/bhac296 (2023).36045002 10.1093/cercor/bhac296
9. Russell JA A circumplex model of affect J. Pers. Soc. Psychol. 1980 39 6 1161 1178 10.1037/h0077714
Russell, J. A. A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980).10.1037/h0077714
10. Tsushima Y Sawahata Y Komine K Task-dependent fMRI decoder with the power to extend Gabor patch results to natural images Sci. Rep. 2020 10 1 1382 10.1038/s41598-020-58241-x 31992812
Tsushima, Y., Sawahata, Y. & Komine, K. Task-dependent fMRI decoder with the power to extend Gabor patch results to natural images. Sci. Rep. 10(1), 1382. 10.1038/s41598-020-58241-x (2020).31992812 10.1038/s41598-020-58241-x
11. Breiman L Bagging predictors Mach. Learn. 1996 24 123 140 10.1007/BF00058655
Breiman, L. Bagging predictors. Mach. Learn. 24, 123–140 (1996).10.1007/BF00058655
12. Pedregosa F Varoquaux G Gramfort A Scikit-learn: Machine learning in Python JLMR 2011 12 2825
Pedregosa, F. et al. Scikit-learn: Machine learning in Python. JLMR 12, 2825 (2011).
13. Varoquaux G Raamana PR Engemann DA Hoyos-Idrobo A Schwartz Y Thirion B Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines Neuroimage 2017 145 Pt B 166 179 10.1016/j.neuroimage.2016.10.038 27989847
Varoquaux, G. et al. Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. Neuroimage 145(Pt B), 166–179. 10.1016/j.neuroimage.2016.10.038 (2017).27989847 10.1016/j.neuroimage.2016.10.038
14. Zhang Y Zhou W Huang J Hong B Wang X Neural correlates of perceived emotions in human insula and amygdala for auditory emotion recognition Neuroimage 2022 15 260 119502 10.1016/j.neuroimage.2022.119502
Zhang, Y., Zhou, W., Huang, J., Hong, B. & Wang, X. Neural correlates of perceived emotions in human insula and amygdala for auditory emotion recognition. Neuroimage 15(260), 119502. 10.1016/j.neuroimage.2022.119502 (2022).10.1016/j.neuroimage.2022.119502
15. Zhang Y Zhou W Wang S Zhou Q Wang H Zhang B Huang J Hong B Wang X The roles of subdivisions of human insula in emotion perception and auditory processing Cereb. Cortex 2019 29 2 517 528 10.1093/cercor/bhx334 29342237
Zhang, Y. et al. The roles of subdivisions of human insula in emotion perception and auditory processing. Cereb. Cortex 29(2), 517–528. 10.1093/cercor/bhx334 (2019).29342237 10.1093/cercor/bhx334
16. Augustine JR The insular lobe in primates including humans Neurol. Res. 1985 7 1 2 10 10.1080/01616412.1985.11739692 2860583
Augustine, J. R. The insular lobe in primates including humans. Neurol. Res. 7(1), 2–10. 10.1080/01616412.1985.11739692 (1985).2860583 10.1080/01616412.1985.11739692
17. Mesulam MM Mufson EJ Peters A Jones EG The insula of Reil in man and monkey: Architectonics, connectivity, and function Cereb. Cortex 1985 4 179 226 10.1007/978-1-4757-9619-3_5
Mesulam, M. M., Mufson, E. J., Peters, A. & Jones, E. G. The insula of Reil in man and monkey: Architectonics, connectivity, and function. Cereb. Cortex 4, 179–226 (1985).10.1007/978-1-4757-9619-3_5
18. Augustine JR Circuitry and functional aspects of the insular lobe in primates including humans Brain Res. Rev. 1996 22 3 229 244 10.1016/s0165-0173(96)00011-2 8957561
Augustine, J. R. Circuitry and functional aspects of the insular lobe in primates including humans. Brain Res. Rev. 22(3), 229–244. 10.1016/s0165-0173(96)00011-2 (1996).8957561 10.1016/s0165-0173(96)00011-2
19. Costabile KA Terman AW Effects of film music on psychological transportation and narrative persuasion Basic Appl. Soc. Psychol. 2013 3 316 324 10.1080/01973533.2013.785398
Costabile, K. A. & Terman, A. W. Effects of film music on psychological transportation and narrative persuasion. Basic Appl. Soc. Psychol. 3, 316–324. 10.1080/01973533.2013.785398 (2013).10.1080/01973533.2013.785398
20. Davis M The role of the amygdala in fear and anxiety Annu. Rev. Neurosci. 1992 15 353 375 10.1146/annurev.ne.15.030192.002033 1575447
Davis, M. The role of the amygdala in fear and anxiety. Annu. Rev. Neurosci. 15, 353–375. 10.1146/annurev.ne.15.030192.002033 (1992).1575447 10.1146/annurev.ne.15.030192.002033
21. Morris JS Frith CD Perrett DI Rowland D Young AW Calder AJ Dolan RJ A differential neural response in the human amygdala to fearful and happy facial expressions Nature 1996 383 6603 812 815 10.1038/383812a0 8893004
Morris, J. S. et al. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383(6603), 812–815. 10.1038/383812a0 (1996).8893004 10.1038/383812a0
22. Morris JS Ohman A Dolan RJ Conscious and unconscious emotional learning in the human amygdala Nature 1998 393 6684 467 470 10.1038/30976 9624001
Morris, J. S., Ohman, A. & Dolan, R. J. Conscious and unconscious emotional learning in the human amygdala. Nature 393(6684), 467–470. 10.1038/30976 (1998).9624001 10.1038/30976
23. Phillips ML Young AW Scott SK Calder AJ Andrew C Giampietro V Williams SC Bullmore ET Brammer M Gray JA Neural responses to facial and vocal expressions of fear and disgust Proc. Biol. Sci. 1998 265 1408 1809 1817 10.1098/rspb.1998.0506 9802236
Phillips, M. L. et al. Neural responses to facial and vocal expressions of fear and disgust. Proc. Biol. Sci. 265(1408), 1809–1817. 10.1098/rspb.1998.0506 (1998).9802236 10.1098/rspb.1998.0506
24. Fitzgerald DA Angstadt M Jelsone LM Nathan PJ Phan KL Beyond threat: Amygdala reactivity across multiple expressions of facial affect Neuroimage. 2006 30 4 1441 1448 10.1016/j.neuroimage.2005.11.003 16368249
Fitzgerald, D. A., Angstadt, M., Jelsone, L. M., Nathan, P. J. & Phan, K. L. Beyond threat: Amygdala reactivity across multiple expressions of facial affect. Neuroimage. 30(4), 1441–1448. 10.1016/j.neuroimage.2005.11.003 (2006).16368249 10.1016/j.neuroimage.2005.11.003
25. Zatorre RJ Pitch perception of complex tones and human temporal-lobe function J. Acoust. Soc. Am. 1988 84 2 566 572 10.1121/1.396834 3170948
Zatorre, R. J. Pitch perception of complex tones and human temporal-lobe function. J. Acoust. Soc. Am. 84(2), 566–572. 10.1121/1.396834 (1988).3170948 10.1121/1.396834
26. Warrier CM Zatorre RJ Right temporal cortex is critical for utilization of melodic contextual cues in a pitch constancy task Brain 2004 127 1616 1625 10.1093/brain/awh183 15128620
Warrier, C. M. & Zatorre, R. J. Right temporal cortex is critical for utilization of melodic contextual cues in a pitch constancy task. Brain 127, 1616–1625 (2004).15128620 10.1093/brain/awh183
27. Bridger, D. Neuro Design: Neuromarketing Insights to Boost Engagement and Profitability (2017).
28. Bayle-Tourtoulou, A. S. & Badoc M. The Neuro-Consumer: Adapting Marketing and Communication Strategies for the Subconscious, Instinctive and Irrational Consumer's Brain (2020).
29. Koch C The Quest for Consciousness: A Neurobiological Approach 2004 Roberts & Company Publishers
Koch, C. The Quest for Consciousness: A Neurobiological Approach (Roberts & Company Publishers, 2004).
30. Tsushima Y Sasaki Y Watanabe T Greater disruption due to failure of inhibitory control on an ambiguous distractor Science 2006 314 5806 1786 1788 10.1126/science.1133197 17170308
Tsushima, Y., Sasaki, Y. & Watanabe, T. Greater disruption due to failure of inhibitory control on an ambiguous distractor. Science 314(5806), 1786–1788. 10.1126/science.1133197 (2006).17170308 10.1126/science.1133197
