==== Front Chin Med J (Engl)Chin. Med. JCMJChinese Medical Journal0366-6999Medknow Publications & Media Pvt Ltd India 30082523CMJ-131-192610.4103/0366-6999.238144Original ArticleVisual Dominance Effect upon Passing the Central Bottleneck of Information Processing 中央瓶颈加工中的视觉主导效应 Yao Xing-Qi 1Yang Yu-Qian 2Chen Shi-Yong 2Sun Wei 3Chen Qi 241 Department of Neurology, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China2 Center for Studies of Psychological Application and School of Psychology, South China Normal University, Guangzhou, Guangdong 510631, China3 Department of Neurology, Beijing Xuanwu Hospital, Capital Medical University, Beijing 101149, China4 Department of Psychology, Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong 510631, ChinaAddress for correspondence: Prof. Qi Chen, Center for Studies of Psychological Application and School of Psychology, South China Normal University, Guangzhou, Guangdong 510631, China E-Mail: qi.chen27@gmail.comXing-Qi Yao and Yu-Qian Yang contributed equally to this work. 20 8 2018 131 16 1926 1935 18 4 2018 Copyright: © 2018 Chinese Medical Journal2018This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.Background: In the classical psychological refractory period (PRP) paradigm, two stimuli are presented in brief succession, and participants are asked to make separate speeded responses to both stimuli. Due to a central cognitive bottleneck, responses to the second stimulus are delayed, especially at short stimulus-onset asynchrony (SOA) between the two stimuli. Although the mechanisms of dual-task interference in the classical PRP paradigm have been extensively investigated, specific mechanisms underlying the cross-modal PRP paradigm are not well understood. In particular, it remains unknown whether the dominance of vision over audition manifests in the cross-modal PRP tasks. The present study aimed to investigate whether the visual dominance effect manifests in the cross-modal PRP paradigm. Methods: We adapted the classical PRP paradigm by manipulating the order of a visual and an auditory task: the visual task could either precede the auditory task or vice versa, at either short or long SOAs. Twenty-five healthy participants took part in Experiment 1, and thirty-three new participants took part in Experiment 2. Reaction time and accuracy data were calculated and further analyzed by repeated-measures analysis of variance. Results: The results showed that visual precedence in the Visual-Auditory condition caused larger impairments to the subsequent auditory processing than vice versa in the Auditory-Visual condition: a larger delay of second response was revealed in the Visual-Auditory condition (135 ± 10 ms) than the Auditory-Visual condition (88 ± 9 ms). This effect was found only at the short SOAs under the existence of the central bottleneck, but not at the long SOAs. Moreover, this effect occurred both when the single visual and the single auditory task were of equal difficulty in Experiment 1 and when the single auditory task was more difficult than the single visual task in Experiment 2. Conclusion: Results of the two experiments suggested that the visual dominance effect occurred under the central bottleneck of cognitive processing. 摘要 背景： 在经典的心理不应期（PRP）范式中，被试要求对相继呈现的两个任务分别进行反应。由于中央瓶颈加工的限制，对 第二个任务的反应时（简称RT2）会随着两个刺激间隔时间（SOA）的缩短而延长。当两个任务的刺激来自不同的感觉通道 时（如，一个视觉任务和一个听觉任务），这种跨通道心理不应期范式（cross-modal PRP paradigm）背后的认知加工机制仍 然缺乏深入关注。本研究旨在探讨在视听跨通道心理不应期范式中是否会出现视觉通道主导听觉通道的现象。 方法： 本研究采用视听跨通道心理不应期范式，其中包含一个视觉数字大小判断任务和一个听觉高低音辨别任务，并操纵两 个任务呈现的顺序（视觉任务先呈现、听觉任务先呈现）和SOA（长、短）。因变量为被试的反应时和正确率，采用重复测 量多元方差分析进行统计分析。实验一中视觉任务和听觉任务难度平衡，包含22名健康被试。通过增加背景噪音，实验二中 听觉任务比视觉任务更难，另包含33名健康被试。 结果： 相较于听觉任务先呈现的条件（88 ± 9 ms），视觉任务先呈现条件下RT2延长的程度显著更大（135 ± 10 ms）。结果 说明了，当视觉任务先于听觉任务呈现，视觉通道信息在加工过程的主导地位会对随后的听觉加工产生更大的干扰。这种效 应仅仅出现在短SOA条件下，即当两个任务需要同时进行中央瓶颈加工时。此外，这种视觉主导效应不受视觉和听觉任务的 难度是否平衡的影响。 结论： 当视觉和听觉任务同时需要进行中央瓶颈加工时存在视觉主导效应。 Multisensory CompetitionPsychological Refractory PeriodSensory DominanceVisual and Auditory Systems ==== Body INTRODUCTION When two stimuli (S1 and S2) are presented in rapid succession to which participants are asked to make separate speed responses, reaction times to the second stimulus (RT2) show a significant increase with decreasing stimulus-onset asynchrony (SOA) between the two stimuli.[1] This phenomenon is termed as the psychological refractory period (PRP) effect.[2] The dominant theory with regard to the mechanisms underlying the PRP effect is the response-selection bottleneck model proposed by Welford and Pashler:[1345] the central response-selection stage constitutes a single-channel bottleneck, and thus can be only executed serially, whereas the other stages, for example, perceptual and motor stages, can be performed in parallel with concurrent processes of another task. Therefore, at short SOAs, the selection of the S2 response had to be postponed until after the S1 response has been selected, resulting in delayed responses to S2. At the neural level, the PRP effect has been associated with the delay of the P3 component elicited by the second target (T2–P3),[67] which is related to postperceptual processes, such as updating in working memory.[89] Moreover, prefrontal and parietal executive areas have been reported to be involved in the PRP effect.[7101112] Upon facing streams of information from multiple sensory modalities, our brain does not give equal weight to different modalities. Rather, visual information more frequently receives preferential processing and eventually dominates consciousness and behavior. One intriguing example of the dominance of vision over audition is the Colavita effect, which refers to the phenomenon that participants often fail in responding to the auditory component of bimodal audiovisual targets.[131415] The striking pattern of the dominance of vision over audition is almost as if the simultaneous presentation of the visual stimulus leads to the “extinction” of the participants’ conscious awareness of the auditory stimulus.[16171819] Even when human participants are able to make explicit behavioral responses to both the visual and the auditory components, they cannot always respond to them strictly simultaneously. Behaviorally, visual responses precede auditory responses more frequently than vice versa,[20] indicating visual dominance. At the neural level, visual information preferentially accesses to the corresponding sensorimotor representations via enhanced functional connectivity between the dorsal visual stream and the sensorimotor cortex.[20] The cross-modal PRP paradigm adopts stimuli of different sensory modalities for S1 and S2, respectively, and shows that information from different sensory modalities competes for the limited processing resources of the human brain.[52122232425] It remains unknown, however, whether the visual dominance effect manifests in the cross-modal PRP situation as well. Specifically speaking, in the cross-modal PRP paradigm, the order of S1 and S2 is 2-fold, i.e., either visual stimuli precede auditory stimuli or vice versa. The temporal order of the two tasks is always fixed in the classical cross-modal PRP paradigm, i.e., the Auditory-Visual and the Visual-Auditory conditions were always blocked, rather than randomly mixed together.[722232526] In contrast to the classical PRP effect with fixed task orders, an additional executive order control mechanism will be introduced in the PRP paradigm with random task orders. The order control mechanism is involved in the planning of the appropriate sequence of actions and the online control of the serial processing order of tasks and operates after the perceptual stage and before the response-selection stage.[272829] In the classical PRP paradigm with fixed task order, the order control process is activated and completed before the onset of the first stimuli in a top-down manner, so that the participants are able to predict the particular tasks for the upcoming trial, and thus prepare for performing the two tasks in a specific order, causing an increase in the top-down weight of limited processing resources for the sensory modality of the first task, especially at short SOAs.[2730] On the contrary, when the task order is randomized, the current order set can only be activated upon the actual presentation of the first task in each trial. Such a random task-order context can thus balance the preparation between the first and the second tasks and prevent the participants from systematically prioritizing one sensory modality over the other.[27313233] Therefore, in contrast to the classical PRP paradigm, we randomly mixed the dual-task trials with different orders (i.e., Visual-Auditory and Auditory-Visual conditions), so that participants could not predict the task order of the upcoming trial to avoid unbalanced top-down bias toward one specific modality. We investigated whether the visual dominance effect manifests in the cross-modal PRP paradigm with random task orders, by manipulating the order of the auditory and visual task. We hypothesized that if visual information is more preferentially selected than auditory information, it will be easier for the second visual response to overcome the interference caused by the first auditory response than vice versa, especially at the short SOAs when multisensory information competes to pass through the central bottleneck. Therefore, with difficulty of the single visual task and the single auditory task being equivalent when they are separately performed, we expected different sizes of the interresponse interval (IRI = RT2 – RT1), depending on the order of the visual and the auditory tasks, when the two tasks are paired with short SOAs. Specifically speaking, the IRI should be significantly smaller in the Auditory-Visual condition when the second visual task needs to overcome the interferences caused by the first auditory task than in the Visual-Auditory condition when the second auditory task needs to overcome the interferences induced by the first visual task. This effect should manifest only at the short, rather than long, SOAs when the two tasks compete for limited resources of the central bottleneck. In addition, we calculated the ratio of the error trials in which the response order is reversed, i.e. the Auditory-Visual responses in the Visual-Auditory condition and the Visual-Auditory responses in the Auditory-Visual condition. We predicted that if visual information tends to be more preferentially selected than auditory information, participants may show a tendency to first respond to the visual stimulus even when it is preceded by an auditory stimulus in the Auditory-Visual condition. Accordingly, more error trials with reversed response orders should be found in the Auditory-Visual than the Visual-Auditory condition. Moreover, it remains unclear whether the potential sensory dominance effect depends on the difficulty of processing in different sensory modalities. For example, whether a similar visual dominance effect will still occur if the visual processing becomes less demanding (easier) than the auditory processing. Therefore, in contrast to the Experiment 1, in which difficulty of single modality visual and auditory processing is matched based on the parameters from previous psychophysics pilots, we will break the balanced difficulty between the single modality visual and auditory processing, by adding background noises throughout the experiment and thus increasing the difficulty of auditory processing in the Experiment 2. METHODS Ethical approval This study was approved by the Ethics Committee of the School of Psychology, South China Normal University. Informed written consent was obtained from all participants before their enrollment in this study. Participants Twenty-five healthy participants (16 females and 9 males; aged 21.0 ± 1.4 years, ranging from 18 to 24 years) participated in Experiment 1, and thirty-three new participants (18 females and 15 males; aged 20.0 ± 2.2 years, ranging from 18 to 25 years) participated in Experiment 2. They were all right-handed, with normal hearing and normal or corrected-to-normal visual acuity. None of them had a history of neurological or psychiatric disorders. All the participants were paid for their participation after the experiment. The apparatus, procedures, experimental design, and stimuli were the same between Experiments 1 and 2, except that a continuously played background noise (echo planar imaging noise of functional magnetic resonance imaging [fMRI]) was added throughout Experiment 2. The background noise was adopted to reduce the signal/noise ratio of the auditory processing by enhancing the noise level and further impair cognition and behavior,[3435] so that the auditory task became more difficult than the visual task in Experiment 2. Apparatus and stimuli The visual stimuli consisted of twenty numbers: from 35 to 44 and from 46 to 55; the auditory stimuli consisted of twenty pure tones with the frequency of 178, 193, 208, 225, 243, 262, 283, 305, 330, 356, 3072, 3317, 3582, 3868, 4176, 4509, 4868, 5257, 5677, and 6130 Hz. The visual target was a black number of 1.64° visual angle shown on a white background via a 17” Acer V173 monitor with a resolution of 1280 (horizontal) × 1024 (vertical) pixels. Participants sat 65 cm away from the screen. The default visual display was a gray cross (red–green–blue value: 128, 128, and 128) that measured 1° × 1° of visual angle on the center of the screen, and the participants were instructed to fixate at the central cross without moving their eyes throughout the experiment. Auditory stimuli were delivered via headphones. Participants were instructed to perform a visual and an auditory two-choice comparison task. In the visual task, a number was presented in the center of the screen for 33 ms, and the participants were instructed to judge whether the number was larger or smaller than 45. The range of the numbers was between 35 and 55, except for 45. In the auditory task, a pure tone was presented through headphones for 33 ms, and the participants were instructed to judge whether the current pure tone was higher or lower than the standard pure tone (1050 Hz). The frequency of the auditory pure tones exponentially ranged from 178 Hz to 6130 Hz, except for 1050 Hz. Participants were instructed to use the middle and index fingers of the one hand to give responses for the visual task (one for bigger and the other for smaller than 45), and the middle and index fingers of the other hand to give responses for the auditory task (one for higher and the other for lower than the standard pure tone). Both the mapping between the two hands and two tasks and the mapping between the two fingers of each hand and the two options of each task were counterbalanced across participants. Experimental procedures and design At the beginning of each trial, the central fixation flashed (500 ms off and 500 ms on) to alert the participants of the start of a trial. Subsequently, a dual-task trial or a single-task trial was presented [Figure 1]. In the dual-task condition, the order of the two tasks was either visual task first (Visual-Auditory) or auditory first (Auditory-Visual), and the SOA between the two tasks was either short (300 ms) or long (1000 ms). Previous evidence proved that when the delay in the onset of the two tasks is <300 ms, execution of both tasks overlaps in time, and thus multisensory information competes to pass through the central bottleneck; when the delay is about 1000 ms, the presentation of the second stimulus comes after the completion of the first task, and thus both tasks are processed without interferences with each other.[72936] The 300 ms and 1000 ms were thus selected as the short and long SOAs, respectively. Therefore, the experimental design for the dual-task condition was a 2 (order of the two tasks: Visual-Auditory vs. Auditory-Visual) × 2 (SOA: short vs. long) within-subject design, resulting in four experimental conditions: (1) the auditory task was first presented, and the SOA between the two tasks was 300 ms (AV300); (2) the visual task was first presented, and the SOA between the two tasks was 300 ms (VA300); (3) the auditory task was first presented, and the SOA between the two tasks was 1000 ms (AV1000); and (4) the visual task was first presented, and the SOA between the two tasks was 1000 ms (VA1000; Figure 1). Two target stimuli (one visual and one auditory) were subsequently presented for 33 ms either in short (300 ms) or long (1000 ms) succession, and the task order could be either visual task first or auditory task first. There were 96 trials in each of the four experimental conditions, and the duration of each trial was 2500 ms. In addition to the dual-task conditions, two single-task conditions were included as well, in which only a visual task (Single-Visual) or an auditory task (Single-Auditory) was presented. There were 48 trials in each of the two single-task conditions, and the duration of each single-task trial was 1500 ms. Therefore, there were six experimental conditions and 480 trials in total. Trials in the six experimental conditions were randomly mixed together. The 480 trials were randomly assigned to 6 blocks with 80 trials in each block. There was a short break after each block. Participants were instructed to perform each task as accurately and as fast as possible. Prior to the formal experiment, participants were adequately familiarized with the experimental tasks with a practice session. Participants could not enter the formal experiment until their accuracy reached above 90% after the practice. Figure 1 Procedures in exemplary dual-task trials of the present experiment. SOA: Stimulus-onset asynchrony; AV300: The auditory task was first presented and the SOA was 300 ms; AV1000: The auditory task was first presented and the SOA was 1000 ms; VA300: The visual task was first presented and the SOA was 300 ms; VA1000: The visual task was first presented and the SOA was 1000 ms. Statistical analysis For the single-task conditions, omissions (0.1% and 0.7% for Experiments 1 and 2, respectively), incorrect responses (3.9% and 3.4%), and outlier trials with RTs beyond mean RT ±3 standard deviations (SDs) (1.7% and 1.4%) were discarded. For the dual-task conditions, trials in which the response order was inconsistent with the task order (3.3% and 5.0%), either RT1 (6.5% and 6.9%) or RT2 (8.9% and 10.2%) was omitted or incorrect, or RT1 (1.6% and 1.5%) or RT2 (1.6% and 1.5%) exceeded ±3 SDs were excluded from further analysis. Accuracies were calculated as the proportion of correct trials versus the total number of trials in each experimental condition. Three participants were removed due to abnormally low accuracy in either the single-task condition or the dual-task condition in Experiment 1. For the single-task conditions, the behavioral data were submitted to a 2 (between-subjects factor: Experiments 1 vs. 2) × 2 (modality: visual vs. auditory) repeated-measures analysis of variance (ANOVA), and then planned paired t-tests were used to test the simple effects in each experiment. For the dual-task conditions, to avoid complicated high-level (four-way) interactions, data were separately analyzed for the short and the long SOAs. RT and accuracy data were submitted to a 2 (between-subjects factor: Experiments 1 vs. 2) × 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (response order: RT1 vs. RT2) repeated-measures ANOVA for the short (300 ms) and long SOAs (1000 ms) conditions, respectively. To further explain the potential three-way interaction, a 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (response order: RT1 vs. RT2) repeated-measures ANOVA was performed for the two experiments, respectively. In addition, the ratio of trials with reversed response orders was submitted to a 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (SOA: 300 ms vs. 1000 ms) repeated-measures ANOVA for each experiment. Statistical analysis was performed with SPSS version 21.0 (IBM, New York, USA). Statistically significant difference was defined as P < 0.05. RESULTS Single-task condition For RTs, the main effect of modality was significant (F(1,53) = 8.73, P = 0.005, η2 = 0.14), indicating that RTs were significantly slower in the single auditory task (674 ± 15 ms) than in the single visual task (642 ± 11 ms). The two-way interaction was significant as well (F(1,53) = 4.68, P = 0.035, η2 = 0.08). Further, planned t-tests on simple effects showed that RTs in the single auditory task (703 ± 19 ms) were significantly slower than in the single visual task (647 ± 14 ms) only in Experiment 2 (t(32) = 3.67, P = 0.001, d = 0.64), but not in Experiment 1 (the single auditory task: 646 ± 23 ms; the single visual task: 637 ± 18 ms; t(21) = 0.62, P = 0.54, d = 0.13; Figure 2a). The main effect of experiment was not significant (F(1, 53) = 1.85, P = 0.180, η2 = 0.08). Figure 2 Mean reaction times (a) and accuracies (b) in the single-task conditions for Experiment 1 and Experiment 2, respectively. *P < 0.05; †Nonsignificant difference. The error bars represent standard error for the sample mean. For the accuracy, neither the main effect of modality (F(1, 53) = 0.44, P = 0.511, η2 = 0.01), nor the main effect of experiment (F(1,53) = 1.23, P = 0.272, η2 = 0.02), nor the two-way interaction (F(1, 53) = 0.00, P = 0.993, η2 = 0), was significant [Figure 2b]. Taken together, the results of the single-modality tasks confirmed our experimental manipulations: the single-modality auditory processing became more difficult than the single-modality visual processing under the noisy background in Experiment 2 [Figure 2a], while they were equally hard in Experiment 1 [Figure 2a]. Dual-task condition Short stimulus-onset asynchrony (300 ms) For the RTs [Figures 3a and 4a, left], the three-way repeated-measures ANOVA showed that the main effect of response order was significant (F(1,53) = 187.94, P < 0.001, η2 = 0.78), indicating that RT2s (796 ± 17 ms) were significantly slower than RT1s (684 ± 16 ms). The two-way interaction between task order and response order was significant as well (F(1,53) = 21.79, P < 0.001, η2 = 0.29). Further planned t-tests on simple effects showed that the IRI effect (RT2 − RT1) was significantly smaller in the Auditory-Visual condition (88 ± 9 ms) than the Visual-Auditory condition (135 ± 10 ms; t(54) = 4.97, P < 0.001, d = 0.67). No other effect was significant (all P > 0.10). Figure 3 Mean reaction times (a) and accuracies (b), shown as a function of task order and response order for the short (the left panel) and long SOAs (the right panel) condition in Experiment 1, respectively. The error bars represent standard error for the sample mean. RT1/RT2: Reaction time of the first/second task in the dual-task conditions; SOA: Stimulus-onset asynchrony. Figure 4 Mean reaction times (a) and accuracies (b), shown as a function of task order and response order for the short (the left panel) and long SOA (the right panel) condition in Experiment 2, respectively. The error bars represent standard error for the sample mean. RT1/RT2: Reaction time of the first/second task in the dual-task conditions; SOA: Stimulus-onset asynchrony. For accuracy [Figures 3b and 4b, left], the main effect of response order was significant (F(1,53) = 16.73, P < 0.001, η2 = 0.24), indicating lower accuracy for the second (86%) than the first (89%) responses. The two-way interaction between task order and response order was significant as well (F(1,53) = 4.59, P = 0.037, η2 = 0.08). Further planned t-tests on simple effects showed that accuracy of the first responses was comparable between the Auditory-Visual (89%) and the Visual-Auditory (91%) condition (t(54) = 1.99, P = 0.052, d = 0.27). For the second responses, however, accuracy was significantly lower in the Visual-Auditory (84%) than the Auditory-Visual (88%) condition (t(54) = 2.12, P = 0.039, d = 0.29). No other effect was significant (all P > 0.19). Taken together, at the short SOA, the RT and accuracy data consistently showed that visual precedence in the Visual-Auditory condition caused more impairment to the subsequent behavioral performance in the auditory task than vice versa in the Auditory-Visual condition. In terms of the RTs, a larger delay of the second response (i.e., a larger IRI) was revealed in the Visual-Auditory than the Auditory-Visual conditions [Figures 3a and 4a]. In terms of response accuracy, more errors were made in responding to the second task of the Visual-Auditory than the Auditory-Visual conditions [Figures 3b and 4b, left]. These results are thus consistent with our predictions that it is more difficult for the second auditory task to overcome the interferences caused by the first visual task than vice versa. Moreover, this visual dominance effect was independent of the difficulty of processing in different sensory modalities: it occurred both when the single modality visual and auditory processing was equally difficult in Experiment 1 [Figure 2a] and when the single modality processing was more demanding in the auditory than visual modality in Experiment 2 [Figure 2a]. Long stimulus-onset asynchrony (1000 ms) For the RTs [Figures 3a and 4a, right], the three-way repeated-measures ANOVA showed that the main effect of response order was significant (F(1,53) = 66.52, P < 0.001, η2 = 0.56), indicating that RT1s (651 ± 12 ms) was significantly slower than RT2s (584 ± 10 ms). The two-way interaction between task order and response order was significant (F(1,53) = 6.12, P = 0.017, η2 = 0.10). The three-way interaction was significant as well (F(1,53) = 4.41, P = 0.041, η2 = 0.08). No other effect was significant (all P > 0.09). To further explain the specific pattern of the three-way interaction, a 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (response order: RT1 vs. RT2) repeated-measures ANOVA was performed for Experiments 1 and 2, respectively. In Experiment 1, the main effect of response order was significant (F(1,21) = 35.99, P < 0.001, η2 = 0.63), indicating that the RT1s (636 ± 19 ms) were significantly slower than the RT2s (568 ± 15 ms; Figure 3a). However, neither the main effect of task order (F(1,21) = 1.22, P = 0.282, η2 = 0.06), nor the two-way interaction (F(1,21) = 0.09, P = 0.772, η2 = 0.00), was significant. These results suggested that at the long SOA of Experiment 1, RT2s were significantly faster than RT1s, irrespective of the task order. In Experiment 2, the main effect of response order was significant (F(1,32) = 35.52, P < 0.001, η2 = 0.53), again indicating that RT1s (666 ± 14 ms) were significantly slower than RT2s (601 ± 13 ms). The main effect of task order was not significant (F(1,32) = 1.81, P = 0.188, η2 = 0.05). In contrast to Experiment 1, the two-way interaction between task order and response order was significant (F(1,32) = 10.78, P = 0.002, η2 = 0.25). Further planned t-tests on simple effects revealed that RT1s (690 ± 17 ms) were significantly slower than RT2s (584 ± 12 ms) in the Auditory-Visual condition (t(32) = 6.25, P < 0.001, d = 1.09), which was consistent with the experimental manipulations in Experiment 2. However, RT1s (643 ± 14 ms) and RT2s (617 ± 16 ms) were comparable in the Visual-Auditory condition (t(32) = 1.60, P = 0.120, d = 0.28; Figure 4a). These results thus suggested that the RT difference between the auditory and visual processing was eliminated in the Visual-Auditory condition at the long SOA. For accuracy [Figures 3b and 4b, right], the three-way repeated-measures ANOVA showed that the main effect of response order was significant (F(1,53) = 20.29, P < 0.001, η2 = 0.28), indicating lower accuracy for the second (92%) than the first (94%) responses. The two-way interaction between task order and experiment was significant (F(1,53) = 9.62, P = 0.003, η2 = 0.15). The three-way interaction was significant as well (F(1,53) = 28.03, P < 0.001, η2 = 0.35). No other effect was significant (all P > 0.39). To further interpret the specific pattern of the three-way interaction, a 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (response order: RT1 vs. RT2) repeated-measures ANOVA was performed for the two experiments, respectively. In Experiment 1, the main effect of task order was significant (F(1,21) = 5.94, P = 0.024, η2 = 0.22), indicating lower accuracy in the Auditory-Visual (92%) than the Visual-Auditory (94%) condition. The main effect of the response order was significant as well (F(1,21) = 7.56, P = 0.012, η2 = 0.27), indicating lower accuracy for the second (92%) than the first (94%) responses. The two-way interaction between task order and response order was significant (F(1,21) = 23.52, P < 0.001, η2 = 0.53). Further planned t-tests on simple effects showed that accuracy was comparable for the first (94%) and the second (95%) responses in the Visual-Auditory condition (t(21) = 1.23, P = 0.233, d = 0.26). However, accuracy was significantly lower for the second (90%) than the first (95%) responses in the Auditory-Visual condition (t(21) = 4.70, P < 0.001, d = 1.00; Figure 3b). In Experiment 2, the main effect of task order was significant (F(1,32) = 4.59, P = 0.040, η2 = 0.13), indicating lower accuracy in the Visual-Auditory (92%) than the Auditory-Visual (94%) condition. The main effect of response order was significant as well (F(1,32) = 15.30, P < 0.001, η2 = 0.32), indicating lower accuracy for the second (91%) than the first (94%) responses. The two-way interaction between task order and response order was significant (F(1,32) = 11.95, P = 0.002, η2 = 0.27). Further planned t-tests on simple effects showed that accuracy was comparable for the first (94%) and the second (94%) responses in the Auditory-Visual condition (t(32) = 0.39, P = 0.698, d = 0.07). In the Visual-Auditory condition, however, accuracy was significantly lower for the second (89%) than the first (94%) responses (t(32) = 4.16, P < 0.001, d = 0.73; Figure 4b), which replicated the difficulty manipulations in Experiment 2. Taken together, at the long SOA when performance of the two tasks did not overlap in time, the RT2s became even faster than the RT1s [Figures 3a and 4a], except for the Visual-Auditory condition in Experiment 2. The faster RT2s than RT1s at the long SOA were consistent with previous results.[233637] Interestingly, in Experiment 2 when single auditory processing was more difficult than single visual processing, the second auditory processing in the Visual-Auditory condition (617 ± 16 ms) became faster than the first auditory processing in the Auditory-Visual condition (690 ± 17 ms, t(32) = 5.36, P < 0.001, d = 0.93), and the first visual processing in the Visual-Auditory condition (643 ± 14 ms) became slower than the second visual processing in the Auditory-Visual condition (584 ± 12 ms, t(32) = 5.51, P < 0.001, d = 0.96), resulting in eliminated differences between RT1s and RT2s in the Visual-Auditory condition [Figure 4a]. Accuracy data in the Visual-Auditory condition at the long SOA of Experiment 2, however, was consistent with the experimental manipulations by showing higher accuracy in the first visual task than in the second auditory task [Figure 4b]. In addition, accuracy data at the long SOA of Experiment 1 showed a trend of RT-accuracy tradeoff in the Auditory-Visual condition: although the first auditory responses were slower than the second visual responses, accuracy was higher in the former than in the latter case [Figure 3a and 3b]. Ratio of reversed response order In Experiment 1, the main effect of the SOA was significant (F(1,21) = 19.93, P < 0.001, η2 = 0.49), indicating more response reversals at the short (5%) than the long SOA (1%). Neither the main effect of task order (F(1,21) = 0.28, P = 0.605, η2 = 0.01), nor the two-way interaction (F(1,21) = 0.29, P = 0.598, η2 = 0.01), was significant [Figure 5a]. In Experiment 2, the main effect of the SOA was significant (F(1,32) = 24.92, P < 0.001, η2 = 0.44), again indicating more response reversals at the short (7%) than the long SOAs (2%). The main effect of task order was marginally significant (F(1,32) = 4.01, P = 0.054, η2 = 0.11), indicating that response reversals tended to be higher in the Auditory-Visual (5%) than the Visual-Auditory (3%) condition. The interaction between task order and SOA was marginally significance as well (F(1,32) = 3.07, P = 0.090, η2 = 0.09). Further planned t-tests on simple effects showed that, at the long SOA, the ratio of response reversals was comparable between the Auditory-Visual (2%) and the Visual-Auditory (1%) condition (t(32) = 0.58, P = 0.564, d = 0.10), while at the short SOA, participants tend to make more response reversals in the Auditory-Visual (9%) than Visual-Auditory (5%) condition (t(32) = 1.92, P = 0.063, d = 0.33; Figure 5b). Figure 5 Ratios of trials with reversed response order, shown as a function of task order and SOA for Experiment 1 (a) and Experiment 2 (b), respectively. The error bars represent standard error for the sample mean. SOA: Stimulus-onset asynchrony. To summarize, the more response reversals at the short SOA than the long SOA again suggested higher PRP interferences with the decreasing SOA [Figure 5a and 5b].[38] Moreover, when the auditory processing became more difficult than the visual processing in Experiment 2, a higher rate of response reversals was observed in the Auditory-Visual than the Visual-Auditory condition, especially at the short SOA [Figure 5b]. The results suggested that visual responses could more frequently precede auditory responses than vice versus even when the visual task was presented later than the auditory task. Previous evidences showed that the executive order control mechanism could be influenced by the task difficulty, leading to a tendency for participants to finish the easy task before the hard task when both tasks overlap in time.[28394041] Consistent with the previous evidence, the present findings thus suggested that participants tended to perform the easier visual task first even when the auditory stimuli were presented first. Gender effect In order to investigate whether there was an effect of gender between different modality and the two experiments, the data in the single-task conditions were further submitted to a 2 (between-subjects factor: Experiment 1 vs. 2) × 2 (gender: female vs. male) × 2 (modality: visual vs. auditory) repeated-measures ANOVA. In the dual-task conditions, the RT and accuracy data were further submitted to a 2 (between-subjects factor: Experiment 1 vs. 2) × 2 (gender: female vs. male) × 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (response order: RT1 vs. RT2) repeated-measures ANOVA for the short (300 ms) and long SOA (1000 ms) conditions, respectively. In addition, the ratio of trials with reversed response orders was further submitted to a 2 (gender: female vs. male) × 2 (task order: Auditory-Visual vs. Visual-Auditory) × 2 (SOA: 300 ms vs. 1000 ms) repeated-measures ANOVA for each experiment. In the single-task conditions, for the RTs, the main effect of modality was significant (F(1,51) = 7.54, P = 0.008, η2 = 0.13), indicating that RTs were significantly slower in the single auditory task (674 ± 15 ms) than in the single visual task (642 ± 11 ms). The two-way interaction between experiment and modality was significant as well (F(1,51) = 4.68, P = 0.045, η2 = 0.08). No other effect was significant (all P > 0.94). For the accuracy, no effect was significant (all P > 0.30). In the dual-task conditions, at the short SOA, for the RTs, the main effect of response order was significant (F(1,51) = 164.64, P < 0.001, η2 = 0.76), indicating that RT2s (796 ± 17 ms) were significantly slower than RT1s (684 ± 16 ms). The two-way interaction between task order and response order was significant as well (F(1,51) = 21.57, P < 0.001, η2 = 0.30). No other effect was significant (all P > 0.06). For the accuracy, the main effect of response order was significant (F(1,51) = 13.14, P < 0.001, η2 = 0.20), indicating lower accuracy for the second (86%) than the first (89%) responses. The two-way interaction between task order and response order was significant as well (F(1,51) = 4.11, P = 0.048, η2 = 0.07). No other effect was significant (all P > 0.14). At the long SOA, for the RTs, the main effect of response order was significant (F(1,51) = 63.96, P < 0.001, η2 = 0.56), indicating that RT1s (651 ± 12 ms) was significantly slower than RT2s (584 ± 10 ms). The main effect of task order was significant (F(1,51) = 4.58, P = 0.04, η2 = 0.08). The two-way interaction between task order and response order was significant (F(1,51) = 5.95, P = 0.018, η2 = 0.11). No other effect was significant, all P > 0.06. For the accuracy, the main effect of response order was significant (F(1,51) = 21.90, P < 0.001, η2 = 0.29), indicating lower accuracy for the second (92%) than the first (94%) responses. The two-way interaction between task order and experiment was significant (F(1,51) = 9.19, P = 0.004, η2 = 0.15). The three-way interaction between task order, response order, and experiment was significant as well (F(1,51) = 26.07, P < 0.001, η2 = 0.33). No other effect was significant (all P > 0.07). For the ratio of reversed response order, in Experiment 1, the main effect of the SOA was significant (F(1,20) = 14.28, P < 0.001, η2 = 0.52), indicating more response reversals at the short (5%) than the long (1%) SOA. No other effect was significant (all P > 0.43). In Experiment 2, the main effect of the SOA was significant (F(1,31) = 24.08, P < 0.001, η2 = 0.55), again indicating more response reversals at the short (7%) than the long SOAs (2%). No other effect was significant (all P > 0.07). To summarize, no significant effect of gender was found either between different sensory modalities or between the two experiments in the present study. More importantly, all the interactions between experiment and gender did not reach statistical significance, indicating that the gender effect was matched between the two experiments. DISCUSSION In the present study, we aimed to investigate whether a similar visual dominance effect exists in the cross-modal PRP tasks. We adapted the classic PRP paradigm by manipulating the task order of a visual and an auditory task. At the short SOA, a larger delay and lower accuracy of the second responses in the Visual-Auditory than the Auditory-Visual conditions were revealed, independent of the difficulty of processing in different sensory modalities [Figures 3a, 3b and 4a, 4b]. The results thus confirmed our hypothesis that it was easier for the second visual response to overcome the interference caused by the first auditory response than vice versa, indicating the dominance of vision over audition when competing the limited capacity of the central bottleneck. In addition, another indicator of visual dominance was found in Experiment 2: participants made more errors with reversed response orders in the Auditory-Visual than the Visual-Auditory conditions, indicating that visual responses tend to precede auditory responses even if the order of the visual task is secondary [Figure 5b]. There has been considerable evidence that the response-selection bottleneck is a principal cause of PRP interference.[1453637] It assumes that dual-task interference in the PRP paradigm occurs after the early sensory stage and before the motor execution stage, and the delay in RT2 that occurs in close temporal succession is primarily due to our inability to choose different responses at the same time. Depending on this account, we propose that the dominance of vision over audition in the cross-modal PRP paradigm is largely attributed to the preferential processing of visual information during the response-selection stage before the motor execution stage. Consistent with this proposal, previous evidence showed that visual dominance during the audiovisual competition was caused by prioritizing visual modality in activating response preparation before response execution in the motor system, compared to auditory modality.[38] For example, a recent fMRI study suggested that dorsal visual stream showed both increased neural activity and enhanced functional connectivity between sensorimotor cortex during visual dominance,[20] which might be the neural causes of prioritizing the visual information into the corresponding sensorimotor representations before motor execution. Accordingly, we suggest that the visual precedence before the motor execution processing makes it easier for the visual task to overcome the PRP interference induced by the previous auditory task than vice versa. In the present study, two qualitatively different tasks (e.g., the frequently adopted visual numerical judgment task and auditory frequency judgment task) were used in the PRP paradigm for Task 1 and Task 2, and the participants had to switch between the two tasks in each and every trial. Thus, one might argue that the current asymmetric modality effect should be attributed to the visual dominance effect manifested in the task switch cost. It has been found that the modality switching costs were higher for auditory than visual stimuli, that is, switching from vision into audition (VA trials) causes more costs than switching from audition into vision (AV trials).[394041] First, although some researchers attributed the PRP effect to the executive processing which consists of switching costs in task set,[42] most previous empirical evidence indicated that the central bottleneck in the PRP paradigm is not caused by the need to switch tasks.[434445] For example, several studies found that the costs of task switching were additive to the PRP effect, that is, the size of the PRP effect was approximately the same in the task repeat and switch conditions, indicating that the switch to Task 2 occurred later than the central bottleneck process of the PRP paradigm.[4445] Second, with regard to the visual dominance effect manifested in the task switch cost, recent evidence suggested that it was more likely to be caused by very specific experimental manipulations, rather than a general effect.[46] Specifically, after controlling for factors of modality appropriateness, contextual effects, and speeds of processing, similar tasks as those in the previous studies[3940] with comparable attentional demands and processing speed failed to show evidence for a visual dominance effect.[46] In the present study, since the general attentional demands, as indexed by the processing speed, were controlled to be equivalent between the visual and the auditory tasks, we tend to believe that the observed visual dominance effect is not attributed to the task switch effect. Moreover, the intertrial intervals between consecutive trials are relatively long in the classical task switch paradigm (usually more than 1500 ms),[39474849] which is more similar to the long than the short SOA trials in the current PRP paradigm. If the observed visual dominance effect was indeed a task switch effect, it should have been observed in the long SOA trials as well. However, our results showed that the first visual task caused larger response impairment to the second auditory task (i.e., the visual dominance effect) only at the short, but not at the long, SOA [Figures 3a, 3b and 4a, 4b], which is against the predictions based on the task switch account [Figures 3a, 3b and 4a, 4b]. Taken together, based on the current experimental manipulations and the visual dominance effect specifically at the short rather than long SOA, we propose that the current asymmetric modality effect is not attributed to the visual dominance effect manifested in the task switch cost. One important future direction in the field is to orthogonally manipulate the cross-modal PRP paradigm and the task switch effect in the same experimental design and to investigate whether the same visual dominance effect occurs in both the task repeat and the task switch conditions. To summarize, using visual and auditory tasks and manipulating the order of the two tasks in the cross-modal PRP paradigm, we found the larger impairment of the second response in the Visual-Auditory than the Auditory-Visual conditions at the short SOAs when multisensory information competes to pass through the central bottleneck, independent of the difficulty of processing in different sensory modalities. The results thus indicated that it was more difficult for the auditory information to recover from the PRP interference induced by the previous visual processing than vice versa. Taken together, our study, for the first time, revealed that the preferential processing of visual information manifested in the cross-modal PRP tasks. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest. Edited by: Xin Chen ==== Refs REFERENCES 1 Pashler H Dual-task interference in simple tasks: Data and theory Psychol Bull 1994 116 220 44 doi: 10.1037/0033-2909.116.2.220 7972591 2 Telford CW The refractory phase of voluntary and associative responses J Exp Psychol 1931 14 1 36 doi: 10.1037/h0073262 3 Pashler H Dissociations and dependencies between speed and accuracy: Evidence for a two-component theory of divided attention in simple tasks Cogn Psychol 1989 21 469 514 doi: 10.1016/0010-0285(89)90016-9 4 Welford AT The psychological refractory period’ and the timing of high-speed performance – A review and a theory Br J Psychol Gen 1952 43 2 19 doi: 10.1111/j.2044-8295.1952.tb00322.x 5 Pashler H Johnston JC Chronometric evidence for central postponement in temporally overlapping tasks Q J Exp Psychol 1989 41 19 45 doi: 10.1080/14640748908402351 6 Dell’acqua R Jolicoeur P Vespignani F Toffanin P Central processing overlap modulates P3 latency Exp Brain Res 2005 165 54 68 doi: 10.1007/s00221-005-2281-2 15827737 7 Sigman M Dehaene S Brain mechanisms of serial and parallel processing during dual-task performance J Neurosci 2008 28 7585 98 doi: 10.1523/JNEUROSCI.0948-08.2008 18650336 8 Donchin E Coles MG Is the P300 component a manifestation of context updating? Behav Brain Sci 1988 11 357 74 doi: 10.1017/S0140525X00058027 9 Verleger R Jaśkowski P Wascher E Evidence for an integrative role of P3b in linking reaction to perception J Psychophysiol 2005 19 165 81 doi: 10.1027/0269-8803.19.3.165 10 Marois R Ivanoff J Capacity limits of information processing in the brain Trends Cogn Sci 2005 9 296 305 doi: 10.1016/j.tics.2005.04.010 15925809 11 Dux PE Ivanoff J Asplund CL Marois R Isolation of a central bottleneck of information processing with time-resolved FMRI Neuron 2006 52 1109 20 doi: 10.1016/j.neuron.2006.11.009 17178412 12 Hesselmann G Flandin G Dehaene S Probing the cortical network underlying the psychological refractory period: A combined EEG-fMRI study Neuroimage 2011 56 1608 21 doi: 10.1016/j.neuroimage.2011.03.017 21397701 13 Colavita FB Human sensory dominance Atten Percept Psychophys 1974 16 409 12 doi: 10.3758/BF03203962 14 Colavita FB Tomko R Weisberg D Visual prepotency and eye orientation Bull Psychon Soc 1976 8 25 6 doi: 10.3758/BF03337062 15 Colavita FB Weisberg D A further investigation of visual dominance Percept Psychophys 1979 25 345 7 doi: 10.3758/BF03198814 461094 16 Egeth HE Sager LC On the locus of visual dominance Attent Percept Psychophys 1977 22 77 86 doi: 10.3758/BF03206083 17 Koppen C Spence C Seeing the light: Exploring the colavita visual dominance effect Exp Brain Res 2007 180 737 54 doi: 10.1007/s00221-007-0894-3 17333012 18 Koppen C Levitan CA Spence C A signal detection study of the Colavita visual dominance effect Exp Brain Res 2009 196 353 60 doi: 10.1007/s00221-009-1853 19488743 19 Hartcher-O’Brien J Alais D Temporal ventriloquism in a purely temporal context J Exp Psychol Hum Percept Perform 2011 37 1383 95 doi: 10.1037/a0024234 21728465 20 Huang S Li Y Zhang W Zhang B Liu X Mo L Multisensory competition is modulated by sensory pathway interactions with fronto-sensorimotor and default-mode network regions J Neurosci 2015 35 9064 77 doi: 10.1523/JNEUROSCI.3760-14.2015 26085631 21 Creamer LR Event uncertainty, psychological refractory period, and human data processing J Exp Psychol 1963 66 187 94 doi: 10.1037/h0041443 14023745 22 Osman A Moore CM The locus of dual-task interference: Psychological refractory effects on movement-related brain potentials J Exp Psychol Hum Percept Perform 1993 19 1292 312 doi: 10.1037/0096-1523.19.6.1292 8294893 23 Schumacher EH Seymour TL Glass JM Fencsik DE Lauber EJ Kieras DE Virtually perfect time sharing in dual-task performance: Uncorking the central cognitive bottleneck Psychol Sci 2001 12 101 8 doi: 10.1111/1467-9280.00318 11340917 24 Szameitat AJ Schubert T Müller K Von Cramon DY Localization of executive functions in dual-task performance with fMRI J Cogn Neurosci 2002 14 1184 99 doi: 10.1162/089892902760807195 12495525 25 Schubert T Szameitat AJ Functional neuroanatomy of interference in overlapping dual tasks: An fMRI study Brain Res Cogn Brain Res 2003 17 733 46 doi: 10.1016/S0926-6410(03)00198-8 14561459 26 Marti S King JR Dehaene S Time-resolved decoding of two processing chains during dual-task interference Neuron 2015 88 1297 307 doi: 10.1016/j.neuron.2015.10.040 26627309 27 Töllner T Strobach T Schubert T Müller HJ The effect of task order predictability in audio-visual dual task performance: Just a central capacity limitation? Front Integr Neurosci 2012 6 75 doi: 10.3389/fnint.2012.000 22973208 28 Hendrich E Determinants of Task Order in Dual-Task Situations. Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftliche Fakultät II 2014 1 42 doi: 10.18452/17088 29 Sigman M Dehaene S Dynamics of the central bottleneck: Dual-task and task uncertainty PLoS Biol 2006 4 1227 38 doi: 10.1371/journal.pbio.0040220 30 Töllner T Gramann K Müller HJ Eimer M The anterior N1 component as an index of modality shifting J Cogn Neurosci 2009 21 1653 69 doi: 10.1162/jocn.2009.21108 18767917 31 Pashler H Graded capacity-sharing in dual-task interference? J Exp Psychol Hum Percept Perform 1994 20 330 42 doi: 10.1037//0096-1523.20.2.330 8189196 32 Ruthruff E Pashler HE Klaassen A Processing bottlenecks in dual-task performance: Structural limitation or strategic postponement? Psychon Bull Rev 2001 8 73 80 doi: 10.3758/BF03196141 11340869 33 Ruthruff E Pashler HE Hazeltine E Dual-task interference with equal task emphasis: Graded capacity sharing or central postponement? Percept Psychophys 2003 65 801 16 doi: 10.3758/BF03194816 12956587 34 Hockey GR Hamilton P Arousal and information selection in short-term memory Nature 1970 226 866 7 doi: 10.1038/226866a0 5444634 35 Arnsten AF Goldman-Rakic PS Noise stress impairs prefrontal cortical cognitive function in monkeys Arch Gen Psychiatry 2000 55 1 7 doi: 10.1111/j.1365-2966.2007.12579.x 36 Sigman M Dehaene S Parsing a cognitive task: A characterization of the mind's bottleneck PLoS Biol 2005 3 0334 49 doi: 10.1371/journal.pbio.0030037 37 McCann RS Johnston JC Locus of the single-channel bottleneck in dual-task interference J Exp Psychol Hum Percept Perform 1992 18 471 84 doi: 10.1037/0096-1523.18.2.471 38 Li Y Liu M Zhang W Huang S Zhang B Liu XZ Neurophysiological correlates of visual dominance: A lateralized readiness potential investigation Front Psychol 2017 8 1 8 doi: 10.3389/fpsyg.2017.00303 28197108 39 Lukas S Philipp AM Koch I Switching attention between modalities: Further evidence for visual dominance Psychol Res 2010 74 255 67 doi: 10.1007/s00426-009-0246-y 19517132 40 Lukas S Philipp AM Koch I The role of preparation and cue-modality in crossmodal task switching Acta Psychol (Amst) 2010 134 318 22 doi: 10.1016/j.actpsy.2010.03.004 20398881 41 Cohen R Rist F The modality shift effect. Further explorations at the crossroads Ann N Y Acad Sci 1992 658 163 81 doi: 10.1111/j.1749-6632.1992.tb22844.x 1497257 42 Logan GD Gordon RD Executive control of visual attention in dual-task situations Psychol Rev 2001 108 393 434 doi: 10.1037//0033-295X.108.2.393 11381835 43 Pashler H Overlapping mental operations in serial performance with preview: Typing. A reply to pashler Q J Exp Psychol Sect A 1994 47 193 99 doi: 10.1080/14640749408401149 44 Band G van Nes FT Reconfiguration and the bottleneck: Does task switching affect the refractory period effect? Eur J Cogn Psychol 2006 16 593 623 doi: 10.1080/09541440500423244 45 Lien MC Schweickert R Proctor RW Task switching and response correspondence in the psychological refractory period paradigm J Exp Psychol Hum Percept Perform 2003 29 692 712 doi: 10.1037/0096-1523.29.3.692 12848334 46 Sandhu R Dyson BJ Re-evaluating visual and auditory dominance through modality switching costs and congruency analyses Acta Psychol (Amst) 2012 140 111 8 doi: 10.1016/j.actpsy.2012.04.003 22622231 47 Monsell S Sumner P Waters H Task-set reconfiguration with predictable and unpredictable task switches Mem Cognit 2003 31 327 42 doi: 10.3758/BF03194391 48 Dreher JC Koechlin E Ali SO Grafman J The roles of timing and task order during task switching Neuroimage 2002 17 95 109 doi: 10.1006/nimg.2002.1169 12482070 49 Sandhu R Dyson BJ Modality and task switching interactions using bi-modal and bivalent stimuli Brain Cogn 2013 82 90 9 doi: 10.1016/j.bandc.2013.02.011 23524240