
==== Front
Curr Neuropharmacol
Curr Neuropharmacol
CN
Current Neuropharmacology
1570-159X
1875-6190
Bentham Science Publishers

37533245
CN-22-1513
10.2174/1570159X21666230801110359
Medicine, Neurology, Pharmacology, Neuroscience
Sensory Reinforced Corticostriatal Plasticity
Vautrelle Nicolas 12#
Coizet Véronique 23#
Leriche Mariana 12
Dahan Lionel 24
Schulz Jan M. 15
Zhang Yan-Feng 16
Zeghbib Abdelhafid 2
Overton Paul G. 2
Bracci Enrico 2
Redgrave Peter 2
Reynolds John N.J. 1*
1 Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand;
2 Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK;
3 Institut des Neurosciences de Grenoble, Université Joseph Fourier, Inserm, U1216, 38706 La Tronche Cedex, France;
4 Centre de Recherches sur la Cognition Animale, Université de Toulouse, UPS, 118 Route de Narbonne, F-31062 Toulouse Cedex 9, France;
5 Department of Biomedicine, University of Basel, CH - 4056 Basel, Switzerland;
6 Department of Clinical and Biomedical Sciences, University of Exeter Medical School, Hatherly Laboratories, Exeter EX4 4PS, United Kingdom
* Address correspondence to this author at the Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand; Tel: +64 3 4795781; E-mail: john.reynolds@otago.ac.nz
# These authors contributed equally to this work.
29 8 2023
2024
22 9 15131527
29 10 2022
04 2 2023
10 2 2023
© 2024 The Author(s). Published by Bentham Science Publishers
2024
The Author(s)
https://creativecommons.org/licenses/by/4.0/ © 2024 The Author(s). Published by Bentham Science Publishers. This is an open access article published under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/legalcode.
Background

Regional changes in corticostriatal transmission induced by phasic dopaminergic signals are an essential feature of the neural network responsible for instrumental reinforcement during discovery of an action. However, the timing of signals that are thought to contribute to the induction of corticostriatal plasticity is difficult to reconcile within the framework of behavioural reinforcement learning, because the reinforcer is normally delayed relative to the selection and execution of causally-related actions.

Objective

While recent studies have started to address the relevance of delayed reinforcement signals and their impact on corticostriatal processing, our objective was to establish a model in which a sensory reinforcer triggers appropriately delayed reinforcement signals relayed to the striatum via intact neuronal pathways and to investigate the effects on corticostriatal plasticity.

Methods

We measured corticostriatal plasticity with electrophysiological recordings using a light flash as a natural sensory reinforcer, and pharmacological manipulations were applied in an in vivo anesthetized rat model preparation.

Results

We demonstrate that the spiking of striatal neurons evoked by single-pulse stimulation of the motor cortex can be potentiated by a natural sensory reinforcer, operating through intact afferent pathways, with signal timing approximating that required for behavioural reinforcement. The pharmacological blockade of dopamine receptors attenuated the observed potentiation of corticostriatal neurotransmission.

Conclusion

This novel in vivo model of corticostriatal plasticity offers a behaviourally relevant framework to address the physiological, anatomical, cellular, and molecular bases of instrumental reinforcement learning.

Keywords

Corticostriatal
plasticity
timing
dopamine
sensory
reinforcement
==== Body
pmc1 INTRODUCTION

A century ago, Thorndike’s cat was confined in a cage until, unwittingly, it pressed against a pedal which opened the cage door, giving the animal access to a piece of fish [1]. With repeated trials, the animal gradually learned what it had to do, so when placed in the cage again, it was able to select the newly acquired action of pedal pressing and gain immediate access to the fish. This first formal demonstration of instrumental conditioning exemplifies reinforcement-driven action acquisition where an unexpected sensory reinforcer (the cage-door opening) enables relevant neural systems to converge onto the causal aspects of the cat’s behaviour, the pedal press. Accumulating empirical evidence points to the basal ganglia, specifically the dorsal striatum, playing a critical role in such reinforcement-driven action acquisition [2-6]. In most models of this process [7-10], signals assumed to represent behavioural options originating from the cerebral cortex induce patterns of activity in the striatum, which are differentially reinforced by consequent sensory events that evoke phasic signals from midbrain dopaminergic neurons. Phasic dopamine (DA) activity is evoked by unexpected, non-habituated sensory events [11-14], including those associated with reward [14-17]. Historically, two main experimental protocols have been used to investigate the biological mechanisms of corticostriatal plasticity: (i) high-frequency stimulation of afferent corticostriatal fibres in association with postsynaptic neuron firing [18]; and (ii) spike-timing-dependent-plasticity (STDP) protocols in which pre- and post-synaptic activity in striatal neurons is manipulated to demonstrate long-term changes in corticostriatal transmission [19-22]. These paradigms have shown that the timing of activation of the pre-and post-synaptic elements and the presence/absence of DA is critical for certain forms of corticostriatal plasticity [17, 18, 21, 22]. It has, however, been difficult to reconcile the timing aspects of early experimental protocols with behavioural reinforcement in which delayed reinforcing sensory signals (the cage door opening in the case of Thorndike’s cat) typically occur hundreds of milliseconds, sometimes seconds, after the relevant causal behaviour (the cat pushing the pedal) [23-26]. Many previous studies have investigated the impact of phasic dopaminergic signals on action selection during the execution of well-learned tasks [27-32]. However, studies investigating the relative timing of afferent cortical and dopaminergic signals on corticostriatal plasticity that may underlie action discovery are limited. For example, the timing of dopaminergic signals seems to be crucial for (i) modulating the structural plasticity of dendritic spines of medium spiny neurons (MSN) [33], (ii) STDP of corticostriatal synapses on D1 and D2-type receptor-expressing MSN [34-36], and (iii) the interaction with cholinergic signalling in the induction of short-term corticostriatal potentiation [37]. However, a model of lasting corticostriatal plasticity in which the temporal dynamic of signals likely to converge within the striatum can be systematically manipulated at timescales consistent with action discovery remains to be interrogated. To address this issue, our strategy was to develop an in vivo preparation that permitted precise control over afferent signalling within the relevant neural network.

Based on our analysis of basal ganglia functional anatomy [5, 6] and following the neoHebbian three-factor learning rules [24, 25, 38], we sought to model three principal sources of input likely to be engaged during instrumental conditioning by a visual reinforcer (Fig. 1A): (i) the afferent collateral fibres, branching from cortical motor projections to the brainstem, which ensure that the striatum receives a running copy of the motor commands directing behavioural output [39, 40]; (ii) the short-latency information signalling the occurrence of an unexpected, salient visual event, relayed via ascending glutamatergic thalamostriatal projections [40-43]; and (iii) the short-latency, visually-evoked, phasic DA input from substantia nigra [44], widely considered to act as a critical reinforcement signal for corticostriatal plasticity [15, 17]. Following the onset of a potentially reinforcing salient visual event, an important source of short-latency input to both nigral DA neurons and thalamic regions that project to the striatum is from branching tecto-nigral/tecto-thalamic fibres that originate from deep-layer neurons of the midbrain superior colliculus (Fig. 1A) [45-47]. Earlier studies by our group have demonstrated that these bifurcating projections will ensure that a single reinforcing visual event can evoke near-simultaneous and potentially converging phasic inputs of DA and glutamate (GLU) into the striatum [43, 44]. Coincident DA and GLU input to the striatum has been shown to be essential for activating the plasticity marker ERK and expressing drug-induced locomotor sensitization [48] (Fig. 1). With this point in mind, we exploited our knowledge of how to use a neutral stimulus (a light flash) repetitively to produce combined short-latency release of DA [44] and GLU [43] into the striatum via intact pathways in anaesthetized rats (Fig. 1A). These procedures rely on the important discovery of Katsuta and Isa [49] who showed that a local injection of the GABAA antagonist bicuculline into the superior colliculus could restore visual responsiveness to deep layer neurons, previously rendered insensitive by anaesthesia. Therefore, the present study was designed to i) test whether sensory-reinforced corticostriatal plasticity could be demonstrated by pairing electrical stimulation of the motor cortex with simultaneous and appropriately timed sensory-evoked inputs from the thalamus (GLU) [43] and substantia nigra (DA) [44]; ii) test whether the temporal dynamics of the observed sensory-reinforced plasticity conformed to the timing of behavioural reinforcement learning; and iii) determine the extent to which intact dopaminergic neurotransmission is essential for this form of corticostriatal plasticity. In our model, the precisely controlled electrically-evoked input from the motor cortex takes the place of a motor command (e.g., a pedal press), which could be causally related to a consequent light flash (in the case of Thorndike’s cat, the door opening). Our prediction was that appropriate timing of the cortical-motor, and visually-evoked sensory inputs should induce prolonged reinforcement of the corticostriatal response in this potentially causal association [5, 10]. The demonstration of a novel, behaviourally relevant, in vivo model of sensory-reinforced corticostriatal plasticity, confirmed this prediction. Subsequent experiments showed that a pharmacological blockade of dopamine receptors partially suppressed the observed potentiation of corticostriatal transmission.

2 MATERIALS AND METHODS

2.1 Care of Animals

All animal husbandry and experimental procedures were performed in the UK with Govt. Home Office approval under section 5(4) of the Animals (Scientific Procedures) Acts 1986. In New Zealand, experiments were conducted in compliance with the Animal Welfare Act 1999. Experimental protocols also received prior approval from the relevant Institutional Ethics Committees.

2.2 Surgical Techniques

Seventy-six Hooded Lister and 9 Long Evans male rats (250-450 g) were prepared for electrophysiological recording under urethane anesthesia (1.25-2.0 g/kg). A concentric bipolar stimulating electrode (NEX-100, Rhodes Medical Instruments, Inc.) was introduced in the primary motor cortex (AP +3.7 to +2.2 mm, bregma; ML +2.0 to +3.0 mm, midline; DV -1.3 to -2.0 mm, dura). A tungsten microelectrode (A-M Systems, Inc., 2 MΩ) glued to a 30-gauge metallic injector needle filled with bicuculline methiodide (Sigma Aldrich, 100 ng/µl 0.9% saline) was placed vertically into the intermediate layers of the ipsilateral lateral superior colliculus (AP -6.3 to -7.3 mm, bregma; ML + 1.5 to 2.5 mm, midline; DV -4.5 to -5.3 mm, dura). An ipsilateral approach (angled 15° in the mediolateral plane; AP +0.2 to -0.8 mm, bregma; ML +2.0 to +3.5 mm, midline; DV -5.0 to -6.0 mm, dura) was used to position a multi-unit (2 MΩ tungsten or NeuroNexus, 16 channels) or single-unit microelectrode (6-13 MΩ glass pipette, internal solution: 0.5 M potassium acetate) into the striatal receptive field responsive to stimulation of the motor cortex ([40, 50] and Fig. S1). In the experiments where striatal microinjections of lidocaine (20-40 nl, 40 µg/µl, Sigma Aldrich) were made, a 30 µm diameter glass injection pipette was glued to the striatal single channel tungsten microelectrode.

2.3 Recording Techniques

A Micro 1401 hardware acquisition system connected to a standard PC running Spike 2 software (Cambridge Electronic Design) was used to sample striatal and collicular local field potential (filter setting: DC-50 Hz) and multi- or single-unit activity (filter setting: 0.2-15 kHz, sampling rate: 15 kHz). A System 3 modular rack-mount workstation (Tucker-Davis Technology) connected via a F15 Gigabit interface to a standard PC running a custom MatlabTM script was used to sample striatal 16 channels multiunit activity (unfiltered signal, sampling rate: 25 kHz).

In the first series of experiments (Figs. 1B and 2), multi-unit responses to ipsilateral motor cortex stimulation (single 100 µs duration pulse, 0.2-1.0 mA intensity, single pulse recurrence 0.5 Hz, 30% jittered) were recorded in the striatum and the superior colliculus. After recording 6 blocks of cortical stimulation-evoked responses (120 stimulations/block), each motor cortex stimulation was paired with a whole-field light flash (10 ms duration) delayed by +250 ms. The flash was delivered from a green LED (570 nm, 60 LUX) positioned 5 mm from the eye contralateral to the stimulation and recording electrodes. After recording 6 more stimulation blocks (120 stimulations/block), bicuculline methiodide was injected into the lateral part of the deep layers of the superior colliculus (0.5 µl, 1 µl/min). Disinhibition of the superior colliculus, assessed by online observation of a clear multi-unit response evoked by the light flash, typically lasted 10-20 min. When the disinhibitory effect of bicuculline had worn off, the light flash was discontinued. Recording of striatal and collicular responses to motor cortex stimulation continued for up to 3 h.

In the second set of experiments (Figs. 1C and 3), the ipsilateral single pulse cortical stimulation was delivered with a 0.2 Hz recurrence to accommodate our longer reinforcement delay of +2 sec. After recording 3 blocks of cortical stimulation-evoked responses (120 stimulations/block), each motor cortex stimulation was paired with a light flash presented either before (-250 ms, N = 7) or after (+250 ms, N = 4; +1000 ms, N = 4; or +2000 ms, N = 4) the cortical stimulation pulse. After recording 3 more stimulation blocks, bicuculline methiodide was injected into the lateral part of the deep layers of the superior colliculus (0.5 µl, 1 µl/min). Following the disinhibitory effect of bicuculline, the light was switched off, and striatal and collicular responses to motor cortex stimulation were recorded for up to 3 h.

In the third series of experiments (Figs. 1D and 6), multi-unit responses to ipsilateral motor cortex stimulation (0.33 Hz recurrence) were recorded in the striatum over 16 channels (Fig. S1B). After recording 4 blocks of cortical stimulation-evoked responses (120 stimulations/block), the animals received an i.p. injection of either saline (0.9%), D1-type dopamine receptor antagonist SCH 23390 hydrochloride (0.2 mg/kg, Sigma), D2-type dopamine receptor antagonist Sulpiride (30 mg/kg, Sigma) or both D1 and D2-type dopamine receptor antagonists. After recording 4 more stimulation blocks (24 mins), each motor cortex stimulation was paired with a light flash presented 250 ms after the cortical stimulation pulse. After recording 4 more blocks, bicuculline methiodide was injected into the lateral part of the deep layers of the superior colliculus (0.5 µl, 1 µl/min). Following the disinhibitory effect of bicuculline, the light was switched off, and striatal and collicular responses to motor cortex stimulation were recorded for up to 3 h.

During single-unit recording experiments (Fig. 4), single pulse cortical stimulation of the motor cortex was delivered with a 0.2 Hz recurrence (0.5-1 mA, 0.1 to 0.25 ms) and paired with a light flash delayed by +250 ms. After recording one block of cortical stimulation-evoked responses (60 stimulations/block) and one block of cortical stimuli paired with the light flash, bicuculline methiodide (0.2-0.3 µl, 0.4 µl/ min) was injected in the lateral superior colliculus. Visual stimulation continued until the collicular disinhibition was no longer present. Recording of the response of the striatal neuron to motor cortex stimulation was maintained until the cell was lost (30-90 min). In some single-unit experiments, the stimulating electrode was placed in the contralateral motor cortex (AP 2.0 mm bregma; ML -1.6 mm midline; DV-2.3 mm, dura). The pattern of response plasticity was similar to that obtained using ipsilateral electrode placements; hence these experiments were considered together.

2.4 Histology

Following the experiment, animals were perfused intracardially with saline (0.9%), followed by paraformaldehyde (4%), and their brains were taken for histological analysis. Using standard immunohistochemical procedures, sections of cortical, striatal, and collicular tissues were reacted to reveal Fos-like immunoreactivity (rabbit polyclonal antibody, 1:20,000 dilution) evoked by electrical, sensory, and chemical stimulation. Fos-like immunoreactivity was only detected in the superior colliculus of animals that had received a bicuculline injection. The distribution of Fos-positive neurons was subjectively analysed to determine the extent of the collicular area activated by each bicuculline injection (Fig. S2C). Other sections were stained with cresyl-violet to verify the locations of the recording and stimulation sites (Figs. S2, S3, and S4).

2.5 Data Analysis

Data were processed offline using CED Spike 2, MatlabTM software, and custom scripts. Multi-unit activity was extracted from high-pass filtered waveforms by applying a threshold determined over the baseline recordings for each experiment to include a wide range of striatal neurons responsive to motor cortex stimulation (Fig. 5). For both multi- and single-unit recordings, spike-count rasters and peri-stimulus time histograms were aligned on cortical stimulation onset (Figs. 5B and S7A). For the first 2 series of experiments, a threshold value (mean frequency + three times the standard deviation of the mean frequency) was calculated over 500 ms of baseline spontaneous activity preceding the cortical stimulation (Fig. 5C). The peak of the cortically-evoked response was then detected during the 50 ms following stimulation. The evoked-response onset and offset were defined as the time of the first bin to exceed or fall below the threshold before and after the peak, respectively. Response magnitude was defined as the number of spike counts during the evoked response minus the mean baseline count for the same period (Fig. 5C green). The value for each block (120 cortical stimulations) was normalized for each subject as a percentage change relative to the mean value of the blocks obtained prior to the injection of bicuculline. For the third set of experiments, striatal responses to cortical stimulation were obtained by subtracting from each peri-stimulus time histogram its own mean spontaneous firing calculated over the 500 ms of spontaneous activity preceding the cortical stimulation (Fig. S7A – green line). An average baseline response to cortical stimulation was then calculated for each channel over the 8 blocks preceding the bicuculline injection (4 post-drug blocks of stimulation + 4 post-drug blocks of stimulation paired with light flash; Fig. S7C blue period and Fig. S7B blue traces). Over all channels, peaks of potentiation were then detected for each block. A peak of potentiation was detected (Fig. S7B red dots) if a bin value in the block response (Fig. S7B green trace) was greater than the sum of the same bin value in the average baseline response (Fig. S7B blue trace) plus two standard deviations (Fig. S7B blue shading). Potentiation peaks were then plotted against time over the experiment (Fig. S7C). A channel was considered potentiated if it met one of the following requirements: i. following bicuculline injection, potentiation peaks with similar latencies were detected over a minimum of 5 consecutive blocks, and such peaks were absent during the pre-drug period; or ii. following bicuculline injection, potentiation peaks with similar latencies were detected over seven or more consecutive blocks, and such peaks were absent during the pre-drug period.

The latency of the potentiated response was defined as the time between the electrical stimulation and the first bin of the potentiated response. The duration of the potentiated response was defined as the number of bins over which a potentiated response was observed. To determine the magnitude of potentiation, the spike-count values for each peak of potentiation were calculated as the difference between the bin value of the block response (green trace) and the bin value of the average baseline response (blue trace). The total magnitude of the potentiation of the response was then calculated by summing the spike counts of all peaks of potentiation. The average magnitude of the potentiation was calculated by dividing that sum by the duration of the potentiated response.

2.6 Statistical Analysis

Group comparison of cortical stimulation-induced striatal responses elicited over the full-time period were made using repeated measures ANOVA to separate group and time effects. A Mann-Whitney U test was used to compare over all experimental conditions and the non-normally distributed mean change (%) in striatal response magnitude data at 44-56 min after collicular disinhibition. Changes from baseline were assessed using a Wilcoxon matched-pairs-signed-rank and Kruskal-Wallis tests. Within-group effects were analysed using paired t-tests.

To assess the effect of the dopamine antagonist(s) on striatal responses to cortical stimulation (Pre-drug baseline period (purple) vs. Post-drug baseline period (blue) in Fig. (S7C), an ANOVA-like table with tests of random-effect terms (RANOVA) was used [51]. This statistic is employed as a measure of the size of the difference between the conditions. A Chi-Square test was used to determine the effect of the drug treatments on the proportion of electrode channels on which pairing induced significant potentiation. Significance was considered for two-tailed p values < 0.05.

3 RESULTS

To simulate motor-copy input to the striatum in a controlled manner, single electrical pulses (0.1 ms; 0.2-1.0 mA; 0.5 Hz) were delivered to the ipsilateral motor cortex (Figs. S2A, S3A, and S4A) and recordings made from neurons in the dorsal striatum (Figs. S2B, S3B, and S4B). A contralateral whole-field light flash provided sensory reinforcement in the presence of a disinhibitory injection of bicuculline (50 ng/500 nl) into the deep layers of the superior colliculus [49] (Figs. S2C, S3C, and S4C). We have shown this treatment ensures that each light flash can effectively activate nigral and thalamic input to the striatum over an extended period [43, 44]. Thus, each cortical pulse was followed by a reinforcing light flash with a delay of 250 ms (Fig. 1B). This value was chosen based on behavioural delayed reinforcement data [23]. At the outset, we were unsure which, if any, striatal neurons would be affected by this paradigm. We, therefore, thought it prudent to record a multi-unit response (Fig. 2) to the cortical electrical stimulus within the motor territories of the striatum (Figs. S1A, S2B, S3B, and S4B).

3.1 Converging Afferent Signals are Required for Corticostriatal Potentiation

As predicted from previous work [43, 44, 52], the suppressive effects of urethane anaesthesia on visual sensory responding in the collicular deep layers also blocked all sensory reinforcement of cortically-evoked striatal activity (all visually-reinforced trials preceding time-0 in Fig. 2A). However, following disinhibitory injections of bicuculline into the superior colliculus, local collicular neurons became visually responsive (Fig. 2C: top), facilitating the relay of sensory signals to the striatum via the tecto-nigro-striatal and tecto-thalamo-striatal projections [43, 44]. Although collicular disinhibition enabled the light flashes to induce reliable visually-evoked local field potentials in the striatal territory receiving input from the motor cortex (Fig. 2C: middle), flash-induced spiking in this part of the striatum was rarely observed (Fig. 2C: bottom). In contrast, the visual reinforcer progressively enhanced multi-unit responses in the striatum evoked by continuing motor cortex single pulse stimulation (Fig. 2A blue line and Fig. 2B; repeated-measures ANOVA of group data, condition x time interaction, F12,84 = 3.6; P = 0.0002). This potentiation of corticostriatal transmission lasted for at least 1 h after the local disinhibitory effect of bicuculline had worn off – indicated by collicular neurons becoming unresponsive again to the visual stimulus. Representative examples of the facilitation of striatal multi-unit spiking activity caused by visual reinforcement are illustrated in Figs. (2B and S5A-E). Comparable potentiation of corticostriatal transmission was not observed when either the light flashes (Fig. 2A: green line) or the disinhibitory injections of bicuculline (Fig. 2A: red line) were omitted from the protocol. These control conditions confirmed first that visually-triggered reinforcing inputs to the striatum could not occur in the absence of signalling from the deep layer of the superior colliculus; and second, that the potentiation observed depends on the precisely timed visual stimulation as any non-specific activation caused by the general disinhibitory effects of intracollicular bicuculline were ineffective (cortical stimulation + collicular bicuculline – green line in Fig 2A).

Further, to test the possibility that bicuculline-gated sensory reinforcement was having a general sensitizing effect in the striatum, unrelated to the electrically-evoked corticostriatal input, the electrical stimulation of the motor cortex was turned off during the period of sensory reinforcement. The cortical stimulation was reinstated when the SC stopped responding to the light flash. Potentiation of the striatal response was then significantly attenuated (Fig. 2D: blue vs. yellow bars; Mann Whitney, U = 3, p < 0.02). Thus, a timed co-activation of cortical and sensory inputs was necessary to express sensory-reinforced potentiation of corticostriatal transmission fully.

However, due to the re-entrant looped architecture of the cortico-basal ganglia projections [53, 54], it is still difficult to ascertain the locus of plasticity in vivo. To exclude the possibility that sensory reinforcement was acting independently of transmission through the striatum, a further control experiment was conducted in which tissue surrounding the striatal recording electrode was temporarily inactivated by a local injection of the topical anaesthetic lidocaine during the period of sensory reinforcement. When cortically-evoked spiking in the striatum recovered from the local anaesthetic, the striatal response to cortical input was significantly depressed (Fig. 2D: blue vs. purple bars; Mann Whitney, U = 0, p < 0.005). This attenuation was not due to a lack of recovery or to possible mechanical damage induced by the local injection of lidocaine as striatal spontaneous spiking after dissipation of the lidocaine effect was similar to that observed before injection (average baseline frequency count before lidocaine 31.7 ± 2.9 Hz vs. 29.1 ± 2.8 Hz after lidocaine; paired t-test, p > 0.1), while cortically-evoked response magnitude was reduced (before lidocaine 1.97 ± 0.13 vs. 1.28 ± 0.19 after lidocaine; paired t-test, p < 0.002). Subsequent analyses were conducted on data from each condition. The mean post-treatment magnitude of the striatal response (+44 to +56 min – the grey shaded area in Fig. 2A) was compared with relevant data from the baseline period preceding treatment (-48 to 0 min). A reliable change from baseline was observed only when cortical stimuli were reinforced with light flashes presented during collicular disinhibition (Wilcoxon matched-pairs-signed-rank test Z = -2.366; p = 0.018). This increase in the amplitude of the cortically-evoked response was accompanied by a significant increase in its duration (Fig. S6B, Kruskal-Wallis, H = 26, d.f. = 4, p < 0.0001). although its latency was unchanged (Fig. S6A, Kruskal-Wallis, H = 4.5, d.f. = 4, p = 0.35). Together, the control experiments showed that the convergence within the striatum of cortical and sensory-evoked reinforcing inputs was necessary for corticostriatal potentiation to be observed.

3.2 Appropriate Signal Timing Required

A critical feature of behavioural reinforcement is that when a reinforcer precedes or is delayed too long after a causal action, its reinforcing effect is greatly diminished [23, 26, 38]. Consequently, to see if these principles also apply in the current model of corticostriatal plasticity, sensory reinforcement was presented at different times relative to the input to the striatum from the motor cortex. To accommodate an increased delay of the sensory reinforcement in this part of the study, the frequency of the cortical stimulation was reduced to 0.2 Hz (Fig. 1C). Under these conditions and consistent with behavioural studies, significant potentiation was observed only when sensory reinforcement occurred within a limited temporal window (+250 and +1000 ms) following the signal from the motor cortex (Figs. 3 and S5G and H). Sensory stimuli presented before (-250 ms) or too long (2000 ms) after cortical stimulation were comparatively ineffective. Moreover, following the reduced number of reinforcement pairings presented during the period of collicular disinhibition in this protocol (recurrence of pairing 0.2 vs. 0.5 Hz), the magnitude of the potentiation effect was also significantly reduced (c.f. Figs. 2D and 3, for the +250 ms condition only; Mann Whitney, U = 10, p < 0.04).

3.3 Potentiation of Single-unit Activity

Next, we sought to explore ways in which the observed enhancement of the multi-unit response may be understood in terms of the effect of sensory reinforcement on the responses of single striatal units. When single pulse cortical stimulation (0.2 Hz) was coupled to sensory reinforcement (light flashes delivered +250 ms after the cortical stimulus) during collicular disinhibition, potentiation was observed in 8/11 recordings from single striatal neurons. From these data, the gradual increase in potentiation seen in the multi-unit response (Fig. 2A) could be understood, in part, by the variable delays in the onset of the potentiation expressed by individual neurons (Fig. 4A). Secondly, the potentiation of multi-unit spiking (Fig. 2) was likely to reflect some neurons increasing their probability of firing at the same specific latencies at which they fired before potentiation (e.g., green neuron in Fig. 4). Alternatively, other neurons would start responding to the cortical stimulation at new latencies, while at the same time maintaining similar spiking probabilities at pre-potentiation latencies (blue neuron in Fig. 4A). Presumably, this variable pattern of firing latencies expressed by individual striatal neurons (Figs. 4B & C and S5F) reflects a combination of distinct afferent corticostriatal and intrastriatal contacts. The short latency evoked striatal responses (< 12 ms) are most likely to be driven by monosynaptic cortical inputs [55], while the longer latency components (> 12 ms) probably reflect multisynaptic contacts. Interestingly, sensory reinforcement seems capable of modulating both mono- and multisynaptic inputs [35]. This, in part, would explain the overall pattern of potentiation we observed in our multi-unit recordings.

3.4 Multiple Sources of Plasticity

Appropriately timed phasic dopaminergic neurotrans-mission is considered an essential factor for the induction of corticostriatal plasticity [18, 21, 34]. To test this, we conducted our plasticity protocol in the presence of systemically administered D1-type (SCH23390) and D2-type (sulpiride) dopamine receptor antagonists. In preparation for interpreting the effects of dopamine antagonists before and after plasticity induction, we used vertically aligned 16-channel electrodes to record cortically evoked multi-unit activity within a larger area of striatal tissue (Figs. S1B and S4B). Because the channels extend 1.5 mm above the tip at a 10° angle, the recording sites of these 16 channel electrodes are more ventrolateral than suggested by the tip location and are likely sampling from a similar area to the other two experiments. After recording a pre-drug baseline control period (Fig. S7C), each subject was injected IP with either 1ml/kg of saline (0.9%; N = 4), the D1 dopamine receptor antagonist SCH23390 (0.2 mg/kg; N = 5), the D2 dopamine receptor antagonist sulpiride (30 mg/kg; N = 5), or an injection that contained both dopamine receptor blockers at the same respective concentrations (N = 5). A post-drug baseline period was then recorded, which included light reinforcement in the absence of collicular disinhibition (Fig. S7C).

To determine the effects of DA antagonists on baseline striatal responding [56] and to detect the subsequent presence of a potentiated response on single recording channels, we constructed post-stimulus time histograms for successive blocks of 120 cortical stimulations (Fig. S7A). When comparing the initial and drug baseline periods (blocks 1-4 vs. blocks 5-12 in Fig. S7C) we confirmed that the D1-type receptor antagonist reliably suppressed the striatal response to cortical stimulation (F = 28.55, p = 0.0001, using a randomisation test based on the F statistic [51]), while the striatal response was enhanced by the D2-type receptor blocker (F = 4.81, p = 0.0321 [51]; Fig. S8). When the DA antagonists were administered in combination, there was a small but reliable increase in baseline striatal responses (F = 9.71, p = 0.0015 [51]; Fig. S8). Finally, there were no reliable differences between the effects of dopamine antagonists on the baseline responses recorded on the electrode channels that would later potentiate, compared with those that did not (Fig. S8).

Since we were now sampling from multiple sites in the striatum, the next step was to determine for each animal how many of the multielectrode’s 16 channels could detect the cortically-evoked neural response. Typically, several adjacent channels were responsive, confirming the restricted striatal responsiveness patterns observed when moving a single electrode (c.f. Figs. S4A and S4B). Consistent with previous experiments, evoked responses comprised time-locked spiking increases that resolved into peak activity at fixed latencies (Figs. S7A and S7B).

We then analysed the results from animals in which the two DA receptor blockers were administered separately by comparing the blocks' histograms following sensory reinforcement with the average histogram from a post-drug-baseline period (Fig. S7B and S7C). The main finding was that, compared with the saline control group, either DA receptor blocker significantly reduced the proportion of electrode channels on which potentiation was recorded (Fig. 6; D1-type antagonist – Chi-Square 11.5, df = 1, p < 0.001; D2-type antagonist – Chi-Square = 15.9, df = 1, p < 0.001). However, on channels where it remained, the observed potentiation was largely unaffected by the DA antagonists; i.e., the mean duration, latency, and magnitude of the potentiation were not statistically different from the values obtained from the saline control group. Lastly, we determined the effects of a combined blockade of D1-type and D2-type dopamine receptors on the corticostriatal plasticity induced by sensory reinforcement. Compared with the saline control group, response potentiation was again observed on significantly fewer electrode channels (Chi-Square 11.6; df = 1; p = 0.001; Fig. 6). However, the overall duration, magnitude, and latencies of positive instances of potentiation were again not reliably different from the saline control condition. We conclude that blocking D1-type and D2-type receptors effectively reduced but did not abolish the number of spatially distributed channels in the striatum on which sensory-reinforced potentiation could be observed.

4 DISCUSSION

The present study established an in vivo model of corticostriatal plasticity to explore the effects of delayed reinforcement signals generated by a natural sensory stimulus [13] and relayed into the striatum via intact afferent projections [43, 44, 46]. Validation of this protocol as an in vivo model of corticostriatal plasticity was strengthened after plasticity on behavioural time scales was observed. The study's main result was that a delayed light flash potentiated multi-unit striatal responses evoked by electrical stimulation of the motor cortex under experimental conditions known to promote visual sensory input to the striatum [43, 44]. The magnitude of the observed potentiation was quantitatively related to the number of stimulation-reinforcement pairings. Importantly, the observed potentiation of corticostriatal transmission was maximised when a behaviourally relevant time delay was imposed between input from the motor cortex and the sensory reinforcement. Reinforcement administered prior to or too long after the cortical input was ineffective. Therefore, this model of sensory-induced corticostriatal plasticity shares important aspects with the reinforcement that happens during behavioural conditioning [1, 23]. In both cases, an unexpected sensory event that occurs before an individual behavioural output cannot have been caused by the latter; therefore, the reinforcement process should not operate.

Similarly, an excessive delay between an action and a consequent reinforcing event invokes an increasingly difficult credit assignment problem, especially if irrelevant actions are expressed during the delay period. Thus, in our model and behavioural conditioning, effective reinforcement only occurs if a potentially contingent sensory reinforcer arrives hundreds of milliseconds after the neural representation of the causal motor output. This result, therefore, supports neoHebbian three-factor learning rules. It corroborates that motor-related input to the striatum generates a decaying synaptic eligibility trace that establishes a critical time window for reinforcement to induce potentiation [24-26, 38]. A mechanistic instantiation of this idea is provided by recent studies that have investigated the impact of delayed dopamine release on Hebbian plasticity at the corticostriatal synapse [33, 34, 36]. For example, in D1-type receptor-expressing medium spiny neurons, Yagishita et al. [33] showed that the structural plasticity of dendritic spines depended on the NMDA-receptor's sequential activation and dopamine D1-type receptor signalling pathways within a similarly restricted time window. Likewise, the potentiation of positive corticostriatal STDP by a delayed reinforcer in D1-type and D2-type receptors expressing striatal neurons was not observed if the activation of the dopamine inputs to the striatum [34] or the uncaging of dopamine [36] occurred with delays greater than ~2s after the corticostriatal pairing. The current protocol, therefore, offers a novel in vivo paradigm to evaluate the physiological, cellular, and molecular mechanisms underlying the concept of reinforcement eligibility [17].

The results show that the reinforcing effect of visual stimuli in the present study, under conditions where phasic DA is known to be released [44], occurred at subthreshold levels and in the absence of any changes in striatal spiking activity (Fig. 2C). This could provide important insights into the mechanisms of sensory reinforcement during behavioural instrumental conditioning [57, 58]. However, to understand how this might be the case, it is necessary to appreciate that instrumental reinforcement operates to bias the selection of future actions (i.e., modulates the frequency with which reinforced actions are selected). Therefore, the mechanism(s) underlying behavioural reinforcement would be expected to be present within the neural systems responsible for action selection [5, 8, 57-59]. A recurring theme within basal ganglia research is that they constitute a mechanism within the vertebrate brain for selecting between competing behavioural motivations and actions [60-62]. The proposed selection mechanism is by selective disinhibition [63] within the parallel loop architecture of the basal ganglia [64, 65]. Instrumental reinforcement is thought to potentiate transmission in recently eligible (selected) channels, thereby increasing their probability of future re-selection [5, 57, 58, 66]. As ‘recently active channels’ cannot be predicted, reinforcement signals must be broadcast widely across the competing channels. It is, therefore, relevant that afferent projections likely to carry short-latency signals reporting the occurrence of an unpredicted sensory reinforcer (both nigro-striatal DA and thalamo-striatal GLU), project widely throughout the striatum [40, 67-69]. Within such an architecture, it is interesting to note in the current model of corticostriatal plasticity that sub-threshold reinforcer-driven depolarization [35, 43, 70] (Fig. 2C), rather than an induction of all-out spiking, is preferred to adjust the sensitivity of recently active channels [66].

How sensory reinforcement might operate on the multiple cell types, and synaptic connections within the striatal microarchitecture will inevitably be complicated. The current in vivo model of cortico-striatal plasticity has revealed a complexity and diversity of potential synaptic changes. From our single-unit recordings of putative medium spiny neurons (MSNs), the observation that sensory reinforcement can potentiate existing responses (Fig. 4C green trace) and induce spiking at previously unresponsive latencies (Fig 4C blue trace) suggests the reinforcement process can operate at multiple synaptic locations and possibly across multi-synaptic pathways. This idea is reinforced by the finding that potentiated responses to cortical stimulation can occur at short latency (< 12 ms) but also at much longer latencies (up to 20-25 ms, Figs. 4C, S5, and S7C). Potentiation observed in our multi-unit responses could result from changes in the intrinsic excitability of MSNs, dependent on D1-type receptors and A2a-receptor signalling [22, 34], but also from changes in synaptic transmission at glutamatergic synapses formed on MSNs [71, 72] and striatal interneurons [73, 74]. While the identification of the different striatal cell types was not the remit of the current test of whether any plasticity was detectable, a principle has been established where future studies using spike sorting from multichannel electrode arrays [75-77] can interrogate how sensory reinforcement can independently modulate components of intrinsic striatal microcircuitry.

Further, our results show that the point at which cortico-striatal potentiation can be observed following a period of sensory reinforcement is highly variable. Thus, some of the observed potentiations occurred soon after the reinforcement period had commenced, yet in other cases, it became evident only 40 to 60 min after its cessation (Fig. 4). This is further evidence of a likely multi-dimensional response in mechanisms intrinsic to the striatum and possibly within other elements of the re-entrant looped architecture that connects the basal ganglia with the cerebral cortex. The current highly constrained model offers the opportunity to investigate independently how the different elements that contribute to the overall multiunit response are modulated by precisely timed sensory reinforcement [78].

Finally, our study confirms that plasticity induced in the striatum by delayed sensory reinforcement is partly dependent on intact DA transmission. Thus, some of the observed plasticity was blocked by systemic injection of a dopaminergic D1/D5-receptor antagonist [21, 22, 34, 79-81]. Some potentiations were also blocked by the systemic injection of a dopaminergic D2/D3-receptor antagonist. This latter effect could, in part, be attributed to the blockade of a form of long-term potentiation dependent on the activation of D2-type receptors and endocannabinoid-receptor signalling reported at the glutamatergic synapses formed on MSNs [71, 72]. However, in the condition where both D1-type and D2-type antagonists were administered, there was clear evidence that corticostriatal transmission could, in some cases, still be modulated by sensory reinforcement. The observed DA-independent plasticity might reflect spike-timing-dependent plasticity occurring at glutamatergic synapses formed on MSNs of the indirect pathway (t-LTP dependent on the activation of A2a adenosine receptors combined to the blockage of t-LTD dependent on D2R dopaminergic transmission) [9, 22, 34, 82]. To a lesser extent, the potentiation of cortical synapses formed on striatal GABAergic interneurons could be involved [73, 74]. In addition, thalamic gating by the light flash could have also contributed to the observed plasticity. The activated thalamic input may modulate the activity of fast-spiking interneurons [83] or cholinergic interneurons [41], with flow-on effects to corticostriatal inputs. In future studies, optogenetic methodo-logies will allow temporal control over the independent activation or silencing of the afferent pathways carrying sensory information to the striatum from the substantia nigra and/or the thalamus [59, 84-86]. Such studies will determine the relative importance of dopaminergic and glutamatergic transmission in sensory-reinforced plasticity within the striatal micro-circuit.

CONCLUSION

The current in vivo model of sensory reinforcement offers a novel paradigm to address the physiological, cellular, and molecular bases of diverse forms of corticostriatal plasticity. An important feature of the paradigm is that it parallels significant aspects of instrumental conditioning in behaving animals [1, 17]. While caution must be exercised over the extent to which our results may have been influenced by the animals being anaesthetised, what we have been able to show is that when precisely controlled motor and sensory inputs converge on striatal units in a temporally relevant manner, the response to the motor input was potentiated. The extent to which this finding generalises to awake behaving preparations is a question for the future. That said, the fact that in our in vivo model, behaviourally relevant afferent projections [40, 42, 45-47, 59] can be appropriately activated by a natural sensory stimulus in a reduced anaesthetised preparation [43, 44] offers a degree of experimental control that would be more difficult to achieve in awake behaving animals. While the current study was always intended as a principal demonstration of sensory-reinforced striatal plasticity, having shown that it can occur with precise experimental control, numerous additional features of this novel paradigm could be investigated. For example, will the current sensory-reinforced plasticity operate in the associative and limbic territories of the striatum? To support the observed plasticity, what cellular and molecular processes occur in different striatal cell types? Can sensory reinforcement potentiate striatal activity generated in functional territories coding for sensory information [40]? Such territories could reinforce contextual information in which an action leading to an unexpected outcome occurs [10]. A different line of future research would test whether variables that influence plasticity in the current model have comparable effects on behavioural reinforcement learning, conversely, whether variables known to influence the acquisition of novel actions have similar effects in the present plasticity model. A better appreciation of the neural processes underlying reinforcement can only assist our interpretation of instances when it fails or becomes pathologically modified, as in aspects of Parkinson’s disease [87], schizophrenia [88], dystonia [89], and addictions [90]. Understanding may also be a prerequisite for discovering rational therapies for these debilitating conditions.

ACKNOWLEDGEMENTS

We acknowledge the grants received from the Wellcome Trust (080943 and 091409) and the Marsden Fund of the Royal Society of New Zealand (UOO0513, UOO0904, and UOO1802). Véronique Coizet and Nicolas Vautrelle contributed equally to this work. We would also like to thank Manfred Oswald and Natalie Kennerley for technical assistance and the following for their constructive comments on an early draft of the manuscript – Jim Surmeier, Jeffrey Wickens, Atsushi Nambu, Joshua Berke, Yael Niv, and Nathaniel Daw.

AUTHORS’ CONTRIBUTIONS

N.V. and V.C. jointly performed and analyzed the multi-unit electrophysiological experiments and immunohistochemistry and co-drafted the manuscript. N.V. performed the antagonist studies. M.L. performed the majority of histological experiments. Y.F.Z., V.C., and J.N.J.R. performed single-unit recording experiments and J.M.S. and L.D. undertook additional supporting in vivo electrophysiological experiments. A.Z. assisted with data analysis. P.O. and E.B. contributed to experimental design and data interpretation. The experiments were designed by J.N.J.R. and P.R., who also drafted the manuscript and supervised the study. All authors critically reviewed the manuscript and gave consent for publication.

LIST OF ABBREVIATIONS

DA Dopamine

GLU Glutamate

MSN Medium Spiny Neuron

RANOVA Random effects ANOVA

STDP Spike-timing-dependent Plasticity

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

All experimental protocols received prior approval from the relevant Institutional Ethics Committees, University of Sheffield Animal Welfare & Ethical Review Body, PPL 40/3703 and University of Otago Animal Ethics Committee, AEC 24/08 and 113/09.

HUMAN AND ANIMAL RIGHTS

All animal experimental procedures were performed in the UK with Govt. Home Office approval under section 5(4) of the Animals (Scientific Procedures) Acts 1986. In New Zealand, experiments were conducted in compliance with the Animal Welfare Act 1999. All methods are reported in accordance with ARRIVE guidelines.

CONSENT FOR PUBLICATION

Not applicable.

AVAILABILITY OF DATA AND MATERIALS

The data that support the findings of this study are available from the corresponding author upon request.

FUNDING

This work was supported by grants from the Wellcome Trust (080943 and 091409) and the Marsden Fund of the Royal Society of New Zealand (UOO0513, UOO0904, and UOO1802).

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

SUPPLEMENTARY MATERIAL

Supplementary material is available on the publisher’s website along with the published article.

Fig. (1) Experimental paradigm and protocols. (A) An experimental paradigm to demonstrate sensory-reinforced corticostriatal plasticity. (i) Single electrical pulses were delivered to ipsilateral motor cortex (0.1 ms; 0.2-1.0 mA; 0.2-0.5 Hz). Sensory reinforcement relayed via the thalamostriatal (ii) and nigrostriatal (iii) projections was provided by a contralateral light flash delayed by 250 ms in the presence of a disinhibitory injection of the GABAA antagonist (bicuculline, 50 ng/ 500 nl) into the superior colliculus. (B, C & D) The timing of stimuli in the three experimental protocols. (B) In the first set of experiments, motor cortex stimulation (black bars) was delivered with an average frequency of 0.5 Hz (ISI 30% jittered - range 1.4 to 2.6 sec). A reinforcing whole-field light flash (yellow bars) delayed by + 250 ms was paired with each cortical stimulation. (C) In the second set of experiments, cortical stimulation (black bar) was applied at 0.2 Hz on average (ISI 30% jittered). The timing of the reinforcing light flash (yellow bars) was systematically varied to occur either before (-250 ms) or after (+250, +1000, +2000 ms) each cortical stimulation. (D) In a third set of experiments, cortical stimulation (black bars) was delivered at an average frequency of 0.33 Hz (ISI 30% jittered). A reinforcing whole-field light flash (yellow bars) delayed by +250 ms was paired with each cortical stimulation.

Fig. (2) Sensory-reinforced corticostriatal plasticity. (A) Single cortical pulses (0.5 Hz) were presented throughout (black bar). Presentation of the reinforcing light flash (+250 ms) is indicated by the red bar. Dishinibition of the superior colliculus is indicated by the blue shading. Each point represents the mean change (%) in the magnitude of striatal multi-unit responses. (B) A single case example of striatal multi-unit potentiation (raster plots and associated peri-stimulus histograms) (C) Visual reinforcement failed to evoke spiking responses in the striatum. Bicuculline-induced restoration of visual responses to deep layer collicular neurons (top graphs); visually-evoked striatal local field potential (middle graphs); striatal multi-unit spiking (bottom graphs). (D) For all experimental conditions, mean change (%) in the magnitude of the cortical stimulus-evoked striatal responses 44-56 min after collicular disinhibition (grey shaded area in A). Experimental conditions are below figure.

Fig. (3) Sensory reinforcement within a behaviourally relevant time window. Only when light reinforcement was delivered +250 and +1000 ms after the cortical stimulus was significant potentiation of the evoked striatal response observed (Two-way ANOVA: F3,15 = 3.6; p < 0.04, Fisher’s PLSD test: *p < 0.05, **p < 0.01).

Fig. (4) Changes in neuronal activity underlying sensory-reinforced corticostriatal plasticity. (A) Four examples of varied responses of individual neurons (different colored lines). For two neurons (dark blue and green lines), the squares mark the trials of cortical stimulation from which raster and histogram figures were calculated in C. (B) Examples of pre- and post-reinforcement activity of the two single units whose data are plotted in C. (C) In one case (green) spiking occurred more frequently at the same latencies after sensory reinforcement, while in the other case (blue) responses at some latencies remained unaltered while spiking at new shorter latencies appeared following potentiation.

Fig. (5) Analysis of multi-unit recording. (A) Multi-unit spike activity evoked by the cortical stimulus and the light flash were recorded locally in the striatum. (B) Data were processed in the form of spike-count rasters and peri-stimulus histograms. (C) Multi-unit striatal responses were recorded in successive blocks of 120 cortical stimulations. For each block multi-unit response characteristics were determined from the peri-stimulus histograms (bin width 1 ms). Response duration was determined by considering the consecutive bins when the firing rate exceeded 3SD (red dotted lines) over the mean base-line firing rate (blue dotted line). Response magnitude (green) for each block of 120 stimulations was recorded as the number of counts during the response minus the mean baseline count for the same period (blue).

Fig. (6) Effect of blocking dopamine neurotransmission on sensory-reinforced corticostriatal plasticity. Separate and combined blockade of D1-type and D2-type dopamine receptors reduced the proportion (%) of recorded channels showing potentiation (*** Chi-square = p < 0.001 compared with Saline group).
==== Refs
REFERENCES

1 Thorndike E.L. Animal intelligence. New York Macmillan 1911
2 Bar-Gad I. Havazelet-Heimer G. Goldberg J.A. Ruppin E. Bergman H. Reinforcement-driven dimensionality reduction-a model for information processing in the basal ganglia. J. Basic Clin. Physiol. Pharmacol. 2000 11 4 305 320 10.1515/JBCPP.2000.11.4.305 11248944
3 Cataldi S. Stanley A.T. Miniaci M.C. Sulzer D. Interpreting the role of the striatum during multiple phases of motor learning. FEBS J. 2022 289 8 2263 2281 33977645
4 Hart G. Leung B.K. Balleine B.W. Dorsal and ventral streams: The distinct role of striatal subregions in the acquisition and performance of goal-directed actions. Neurobiol. Learn. Mem. 2014 108 104 118 10.1016/j.nlm.2013.11.003 24231424
5 Redgrave P. Gurney K. The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci. 2006 7 12 967 975 10.1038/nrn2022 17115078
6 Redgrave P. Vautrelle N. Reynolds J.N.J. Functional properties of the basal ganglia’s re-entrant loop architecture: Selection and reinforcement. Neuroscience 2011 198 138 151 10.1016/j.neuroscience.2011.07.060 21821101
7 Balleine B.W. Delgado M.R. Hikosaka O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 2007 27 31 8161 8165 10.1523/JNEUROSCI.1554-07.2007 17670959
8 Graybiel A.M. The basal ganglia: Learning new tricks and loving it. Curr. Opin. Neurobiol. 2005 15 6 638 644 10.1016/j.conb.2005.10.006 16271465
9 Gurney K.N. Humphries M.D. Redgrave P. A new framework for cortico-striatal plasticity: Behavioural theory meets in vitro data at the reinforcement-action interface. PLoS Biol. 2015 13 1 e1002034 10.1371/journal.pbio.1002034 25562526
10 Redgrave P. Gurney K. Reynolds J. What is reinforced by phasic dopamine signals? Brain Res. Brain Res. Rev. 2008 58 2 322 339 10.1016/j.brainresrev.2007.10.007 18055018
11 Barto A. Mirolli M. Baldassarre G. Novelty or surprise? Front. Psychol. 2013 4 907 10.3389/fpsyg.2013.00907 24376428
12 Bromberg-Martin E.S. Matsumoto M. Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 2010 68 5 815 834 10.1016/j.neuron.2010.11.022 21144997
13 Lloyd D.R. Gancarz A.M. Ashrafioun L. Kausch M.A. Richards J.B. Habituation and the reinforcing effectiveness of visual stimuli. Behav. Processes 2012 91 2 184 191 10.1016/j.beproc.2012.07.007 22868172
14 Menegas W. Babayan B.M. Uchida N. Watabe-Uchida M. Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. eLife 2017 6 e21886 10.7554/eLife.21886 28054919
15 Reynolds J.N.J. Hyland B.I. Wickens J.R. A cellular mechanism of reward-related learning. Nature 2001 413 6851 67 70 10.1038/35092560 11544526
16 Schultz W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 2006 57 1 87 115 10.1146/annurev.psych.56.091103.070229 16318590
17 Wickens J.R. Synaptic plasticity in the basal ganglia. Behav. Brain Res. 2009 199 1 119 128 10.1016/j.bbr.2008.10.030 19026691
18 Calabresi P. Picconi B. Tozzi A. Di Filippo M. Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci. 2007 30 5 211 219 10.1016/j.tins.2007.03.001 17367873
19 Fino E. Glowinski J. Venance L. Bidirectional activity-dependent plasticity at corticostriatal synapses. J. Neurosci. 2005 25 49 11279 11287 10.1523/JNEUROSCI.4476-05.2005 16339023
20 Fino E. Venance L. Spike-timing dependent plasticity in the striatum. Front. Synaptic Neurosci. 2010 2 6 21423492
21 Pawlak V. Kerr J.N.D. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J. Neurosci. 2008 28 10 2435 2446 10.1523/JNEUROSCI.4402-07.2008 18322089
22 Shen W. Flajolet M. Greengard P. Surmeier D.J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 2008 321 5890 848 851 10.1126/science.1160575 18687967
23 Dickinson A. The 28th Bartlett Memorial Lecture Causal learning: An associative analysis. Q. J. Exp. Psychol. B 2001 54 1 3 25 10.1080/02724990042000010 11216300
24 Foncelle A. Mendes A. Jędrzejewska-Szmek, J.; Valtcheva, S.; Berry, H.; Blackwell, K.T.; Venance, L. Modulation of spike-timing dependent plasticity: Towards the inclusion of a third factor in computational models. Front. Comput. Neurosci. 2018 12 49 10.3389/fncom.2018.00049 30018546
25 Frémaux N. Gerstner W. Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Front. Neural Circuits 2016 9 85 10.3389/fncir.2015.00085 26834568
26 Izhikevich E.M. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 2007 17 10 2443 2452 10.1093/cercor/bhl152 17220510
27 Cacciapaglia F. Saddoris M.P. Wightman R.M. Carelli R.M. Differential dopamine release dynamics in the nucleus accumbens core and shell track distinct aspects of goal-directed behavior for sucrose. Neuropharmacology 2012 62 5-6 2050 2056 10.1016/j.neuropharm.2011.12.027 22261383
28 Howard C.D. Li H. Geddes C.E. Jin X. Dynamic nigrostriatal dopamine biases action selection. Neuron 2017 93 6 1436 1450.e8 10.1016/j.neuron.2017.02.029 28285820
29 Phillips P.E.M. Stuber G.D. Heien M.L.A.V. Wightman R.M. Carelli R.M. Subsecond dopamine release promotes cocaine seeking. Nature 2003 422 6932 614 618 10.1038/nature01476 12687000
30 Roitman M.F. Stuber G.D. Phillips P.E. Wightman R.M. Carelli R.M. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 2004 24 6 1265 1271 10.1523/JNEUROSCI.3823-03.2004 14960596
31 Stopper C.M. Tse M.T.L. Montes D.R. Wiedman C.R. Floresco S.B. Overriding phasic dopamine signals redirects action selection during risk/reward decision making. Neuron 2014 84 1 177 189 10.1016/j.neuron.2014.08.033 25220811
32 Stuber G.D. Roitman M.F. Phillips P.E.M. Carelli R.M. Wightman R.M. Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology 2005 30 5 853 863 10.1038/sj.npp.1300619 15549053
33 Yagishita S. Hayashi-Takagi A. Ellis-Davies G.C.R. Urakubo H. Ishii S. Kasai H. A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science 2014 345 6204 1616 1620 10.1126/science.1255514 25258080
34 Fisher S.D. Robertson P.B. Black M.J. Redgrave P. Sagar M.A. Abraham W.C. Reynolds J.N.J. Reinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivo. Nat. Commun. 2017 8 1 334 10.1038/s41467-017-00394-x 28839128
35 Schulz J.M. Redgrave P. Reynolds J.N. Cortico-striatal spike-timing dependent plasticity after activation of subcortical pathways. Front. Synaptic Neurosci. 2010 2 23 10.3389/fnsyn.2010.00023 21423509
36 Shindou T. Shindou M. Watanabe S. Wickens J. A silent eligibility trace enables dopamine‐dependent synaptic plasticity for reinforcement learning in the mouse striatum. Eur. J. Neurosci. 2019 49 5 726 736 10.1111/ejn.13921 29603470
37 Reynolds J.N.J. Avvisati R. Dodson P.D. Fisher S.D. Oswald M.J. Wickens J.R. Zhang Y.F. Coincidence of cholinergic pauses, dopaminergic activation and depolarisation of spiny projection neurons drives synaptic plasticity in the striatum. Nat. Commun. 2022 13 1 1296 10.1038/s41467-022-28950-0 35277506
38 Gerstner W. Lehmann M. Liakoni V. Corneil D. Brea J. Eligibility traces and plasticity on behavioral time scales: Experimental support of neoHebbian three-Factor learning rules. Front. Neural Circuits 2018 12 53 10.3389/fncir.2018.00053 30108488
39 Reiner A. Jiao Y. Del Mar N. Laverghetta A.V. Lei W.L. Differential morphology of pyramidal tract-type and intratelencephalically projecting-type corticostriatal neurons and their intrastriatal terminals in rats. J. Comp. Neurol. 2003 457 4 420 440 10.1002/cne.10541 12561080
40 Hunnicutt B.J. Jongbloets B.C. Birdsong W.T. Gertz K.J. Zhong H. Mao T. A comprehensive excitatory input map of the striatum reveals novel functional organization. eLife 2016 5 e19103 10.7554/eLife.19103 27892854
41 Ding J.B. Guzman J.N. Peterson J.D. Goldberg J.A. Surmeier D.J. Thalamic gating of corticostriatal signaling by cholinergic interneurons. Neuron 2010 67 2 294 307 10.1016/j.neuron.2010.06.017 20670836
42 Matsumoto N. Minamimoto T. Graybiel A.M. Kimura M. Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J. Neurophysiol. 2001 85 2 960 976 10.1152/jn.2001.85.2.960 11160526
43 Schulz J.M. Redgrave P. Mehring C. Aertsen A. Clements K.M. Wickens J.R. Reynolds J.N.J. Short-latency activation of striatal spiny neurons via subcortical visual pathways. J. Neurosci. 2009 29 19 6336 6347 10.1523/JNEUROSCI.4815-08.2009 19439610
44 Dommett E. Coizet V. Blaha C.D. Martindale J. Lefebvre V. Walton N. Mayhew J.E.W. Overton P.G. Redgrave P. How visual stimuli activate dopaminergic neurons at short latency. Science 2005 307 5714 1476 1479 10.1126/science.1107026 15746431
45 Benavidez N.L. Bienkowski M.S. Zhu M. Garcia L.H. Fayzullina M. Gao L. Bowman I. Gou L. Khanjani N. Cotter K.R. Korobkova L. Becerra M. Cao C. Song M.Y. Zhang B. Yamashita S. Tugangui A.J. Zingg B. Rose K. Lo D. Foster N.N. Boesen T. Mun H.S. Aquino S. Wickersham I.R. Ascoli G.A. Hintiryan H. Dong H.W. Organization of the inputs and outputs of the mouse superior colliculus. Nat. Commun. 2021 12 1 4004 10.1038/s41467-021-24241-2 34183678
46 Coizet V. Overton P.G. Redgrave P. Collateralization of the tectonigral projection with other major output pathways of superior colliculus in the rat. J. Comp. Neurol. 2007 500 6 1034 1049 10.1002/cne.21202 17183537
47 Comoli E. Coizet V. Boyes J. Bolam J.P. Canteras N.S. Quirk R.H. Overton P.G. Redgrave P. A direct projection from superior colliculus to substantia nigra for detecting salient visual events. Nat. Neurosci. 2003 6 9 974 980 10.1038/nn1113 12925855
48 Valjent E. Pascoli V. Svenningsson P. Paul S. Enslen H. Corvol J.C. Stipanovich A. Caboche J. Lombroso P.J. Nairn A.C. Greengard P. Hervé D. Girault J.A. Regulation of a protein phosphatase cascade allows convergent dopamine and glutamate signals to activate ERK in the striatum. Proc. Natl. Acad. Sci. USA 2005 102 2 491 496 10.1073/pnas.0408305102 15608059
49 Katsuta H. Isa T. Release from GABAA receptor-mediated inhibition unmasks interlaminar connection within superior colliculus in anesthetized adult rats. Neurosci. Res. 2003 46 1 73 83 10.1016/S0168-0102(03)00029-4 12725914
50 Sgambato V. Abo V. Rogard M. Besson M.J. Deniau J.M. Effect of electrical stimulation of the cerebral cortex on the expression of the fos protein in the basal ganglia. Neuroscience 1997 81 1 93 112 10.1016/S0306-4522(97)00179-6 9300404
51 Manly B.F.J. Randomization and Monte Carlo methods in biology. London Chapman and Hall 1991 281 10.1007/978-1-4899-2995-2
52 Coizet V. Graham J.H. Moss J. Bolam J.P. Savasta M. McHaffie J.G. Redgrave P. Overton P.G. Short-latency visual input to the subthalamic nucleus is provided by the midbrain superior colliculus. J. Neurosci. 2009 29 17 5701 5709 10.1523/JNEUROSCI.0247-09.2009 19403836
53 Alexander G.E. DeLong M.R. Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 1986 9 1 357 381 10.1146/annurev.ne.09.030186.002041 3085570
54 McHaffie J. Stanford T. Stein B. Coizet V. Redgrave P. Subcortical loops through the basal ganglia. Trends Neurosci. 2005 28 8 401 407 10.1016/j.tins.2005.06.006 15982753
55 Wilson C.J. Postsynaptic potentials evoked in spiny neostriatal projection neurons by stimulation of ipsilateral and contralateral neocortex. Brain Res. 1986 367 1-2 201 213 10.1016/0006-8993(86)91593-3 3008920
56 Brown J.R. Arbuthnott G.W. The electrophysiology of dopamine (D2) receptors: A study of the actions of dopamine on corticostriatal transmission. Neuroscience 1983 10 2 349 355 10.1016/0306-4522(83)90138-0 6138732
57 Balleine B.W. Liljeholm M. Ostlund S.B. The integrative function of the basal ganglia in instrumental conditioning. Behav. Brain Res. 2009 199 1 43 52 10.1016/j.bbr.2008.10.034 19027797
58 Peak J. Hart G. Balleine B.W. From learning to action: The integration of dorsal striatal input and output pathways in instrumental conditioning. Eur. J. Neurosci. 2019 49 5 658 671 10.1111/ejn.13964 29791051
59 Kato S. Fukabori R. Nishizawa K. Okada K. Yoshioka N. Sugawara M. Maejima Y. Shimomura K. Okamoto M. Eifuku S. Kobayashi K. Action selection and flexible switching controlled by the intralaminar thalamic neurons. Cell Rep. 2018 22 9 2370 2382 10.1016/j.celrep.2018.02.016 29490273
60 Klaus A. Alves da Silva J. Costa R.M. What, if, and when to move: Basal ganglia circuits and self-paced action initiation. Annu. Rev. Neurosci. 2019 42 1 459 483 10.1146/annurev-neuro-072116-031033 31018098
61 Mink J.W. The basal ganglia: Focused selection and inhibition of competing motor programs. Prog. Neurobiol. 1996 50 4 381 425 10.1016/S0301-0082(96)00042-1 9004351
62 Redgrave P. Prescott T.J. Gurney K. The basal ganglia: A vertebrate solution to the selection problem? Neuroscience 1999 89 4 1009 1023 10.1016/S0306-4522(98)00319-4 10362291
63 Chevalier G. Deniau J.M. Disinhibition as a basic process in the expression of striatal functions. Trends Neurosci. 1990 13 7 277 280 10.1016/0166-2236(90)90109-N 1695403
64 Humphries M.D. Stewart R.D. Gurney K.N. A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J. Neurosci. 2006 26 50 12921 12942 10.1523/JNEUROSCI.3486-06.2006 17167083
65 Prescott T.J. Montes González F.M. Gurney K. Humphries M.D. Redgrave P. A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Netw. 2006 19 1 31 61 10.1016/j.neunet.2005.06.049 16153803
66 Arbuthnott G.W. Wickens J. Space, time and dopamine. Trends Neurosci. 2007 30 2 62 69 10.1016/j.tins.2006.12.003 17173981
67 Menegas W. Bergan J.F. Ogawa S.K. Isogai Y. Umadevi Venkataraju K. Osten P. Uchida N. Watabe-Uchida M. Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. eLife 2015 4 e10032 10.7554/eLife.10032 26322384
68 Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 1998 80 1 1 27 10.1152/jn.1998.80.1.1 9658025
69 Van der Werf Y.D. Witter M.P. Groenewegen H.J. The intralaminar and midline nuclei of the thalamus. Anatomical and functional evidence for participation in processes of arousal and awareness. Brain Res. Brain Res. Rev. 2002 39 2-3 107 140 10.1016/S0165-0173(02)00181-9 12423763
70 Fino E. Deniau J.M. Venance L. Brief subthreshold events can act as Hebbian signals for long-term plasticity. PLoS One 2009 4 8 e6557 10.1371/journal.pone.0006557 19675683
71 Cui Y. Paillé V. Xu H. Genet S. Delord B. Fino E. Berry H. Venance L. Endocannabinoids mediate bidirectional striatal spike-timing-dependent plasticity. J. Physiol. 2015 593 13 2833 2849 10.1113/JP270324 25873197
72 Cui Y. Prokin I. Xu H. Delord B. Genet S. Venance L. Berry H. Endocannabinoid dynamics gate spike-timing dependent depression and potentiation. eLife 2016 5 e13185 10.7554/eLife.13185 26920222
73 Fino E. Deniau J.M. Venance L. Cell-specific spike-timing-dependent plasticity in GABAergic and cholinergic interneurons in corticostriatal rat brain slices. J. Physiol. 2008 586 1 265 282 10.1113/jphysiol.2007.144501 17974593
74 Fino E. Paille V. Deniau J.M. Venance L. Asymmetric spike-timing dependent plasticity of striatal nitric oxide-synthase interneurons. Neuroscience 2009 160 4 744 754 10.1016/j.neuroscience.2009.03.015 19303912
75 Peters A.J. Fabre J.M.J. Steinmetz N.A. Harris K.D. Carandini M. Striatal activity topographically reflects cortical activity. Nature 2021 591 7850 420 425 10.1038/s41586-020-03166-8 33473213
76 Sharott A. Doig N.M. Mallet N. Magill P.J. Relationships between the firing of identified striatal interneurons and spontaneous and driven cortical activities in vivo. J. Neurosci. 2012 32 38 13221 13236 10.1523/JNEUROSCI.2440-12.2012 22993438
77 Sharott A. Moll C.K.E. Engler G. Denker M. Grün S. Engel A.K. Different subtypes of striatal neurons are selectively modulated by cortical oscillations. J. Neurosci. 2009 29 14 4571 4585 10.1523/JNEUROSCI.5097-08.2009 19357282
78 Martiros N. Burgess A.A. Graybiel A.M. Inversely active striatal projection neurons and interneurons selectively delimit useful behavioral sequences. Curr. Biol. 2018 28 4 560 573.e5 10.1016/j.cub.2018.01.031 29429614
79 Centonze D. Grande C. Saulle E. Martín A.B. Gubellini P. Pavón N. Pisani A. Bernardi G. Moratalla R. Calabresi P. Distinct roles of D1 and D5 dopamine receptors in motor activity and striatal synaptic plasticity. J. Neurosci. 2003 23 24 8506 8512 10.1523/JNEUROSCI.23-24-08506.2003 13679419
80 Kerr J.N.D. Wickens J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 2001 85 1 117 124 10.1152/jn.2001.85.1.117 11152712
81 Suzuki T. Miura M. Nishimura K. Aosaki T. Dopamine-dependent synaptic plasticity in the striatal cholinergic interneurons. J. Neurosci. 2001 21 17 6492 6501 10.1523/JNEUROSCI.21-17-06492.2001 11517238
82 Flajolet M. Wang Z. Futter M. Shen W. Nuangchamnong N. Bendor J. Wallach I. Nairn A.C. Surmeier D.J. Greengard P. FGF acts as a co-transmitter through adenosine A2A receptor to regulate synaptic plasticity. Nat. Neurosci. 2008 11 12 1402 1409 10.1038/nn.2216 18953346
83 Sciamanna G. Ponterio G. Mandolesi G. Bonsi P. Pisani A. Optogenetic stimulation reveals distinct modulatory properties of thalamostriatal vs corticostriatal glutamatergic inputs to fast-spiking interneurons. Sci. Rep. 2015 5 1 16742 10.1038/srep16742 26572101
84 Saunders B.T. Richard J.M. Margolis E.B. Janak P.H. Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nat. Neurosci. 2018 21 8 1072 1083 10.1038/s41593-018-0191-4 30038277
85 Saunders BT Richard JM Janak PH Contemporary approaches to neural circuit manipulation and mapping: Focus on reward and addiction. Philos Trans. R Soc. Lond B Biol. Sci. 2015 370 1677 20140210 10.1098/rstb.2014.0210 26240425
86 Huang M. Li D. Cheng X. Pei Q. Xie Z. Gu H. Zhang X. Chen Z. Liu A. Wang Y. Sun F. Li Y. Zhang J. He M. Xie Y. Zhang F. Qi X. Shang C. Cao P. The tectonigral pathway regulates appetitive locomotion in predatory hunting in mice. Nat. Commun. 2021 12 1 4409 10.1038/s41467-021-24696-3 34285209
87 Voon V. Fernagut P.O. Wickens J. Baunez C. Rodriguez M. Pavon N. Juncos J.L. Obeso J.A. Bezard E. Chronic dopaminergic stimulation in Parkinson’s disease: From dyskinesias to impulse control disorders. Lancet Neurol. 2009 8 12 1140 1149 10.1016/S1474-4422(09)70287-X 19909912
88 Ziauddeen H. Murray G.K. The relevance of reward pathways for schizophrenia. Curr. Opin. Psychiatry 2010 23 2 91 96 10.1097/YCO.0b013e328336661b 20051858
89 Sciamanna G. Tassone A. Mandolesi G. Puglisi F. Ponterio G. Martella G. Madeo G. Bernardi G. Standaert D.G. Bonsi P. Pisani A. Cholinergic dysfunction alters synaptic integration between thalamostriatal and corticostriatal inputs in DYT1 dystonia. J. Neurosci. 2012 32 35 11991 12004 10.1523/JNEUROSCI.0041-12.2012 22933784
90 Russo S.J. Dietz D.M. Dumitriu D. Morrison J.H. Malenka R.C. Nestler E.J. The addicted synapse: mechanisms of synaptic and structural plasticity in nucleus accumbens. Trends Neurosci. 2010 33 6 267 276 10.1016/j.tins.2010.02.002 20207024
