Introduction

Dopamine participates in a wide range of behavioral and cognitive functions in the brain, including movement, reward processing, and creativity (Flaherty, 2005). Of the central monoamine neurotransmitters, dopamine presents the greatest challenge in terms of deducing its main physiological role. Midbrain dopamine neurons of the ventral tegmental area (VTA) send projections to many regions including the striatum and prefrontal cortex and are together referred to as the mesocorticolimbic dopaminergic (DA) system. This system has traditionally been viewed as a single “neural currency” of reward and this view has numerous experimental confirmations (Berridge, 2007; Tsai et al., 2009; Wise, 2012; Rossi, Sukharnikova, Hayra- petyan, Yang, & Yin, 2013; Steinberg et al., 2013). How exactly the midbrain DA system is involved in the processes of learning, and what aspects or specific links of the physiological mechanisms of reinforcement it represents are an important focus of ongoing research (Berridge, 2007; Ivlieva, 2010; Bromberg-Martin, Matsumoto, & Hikosaka, 2010; Steinberg & Janak, 2013).

Current hypotheses suggest that (1) DA itself induces pleasure (Wise & Bozarth, 1985), (2) DA provides the universal teaching signal that induces learning (Schultz, 1998; Schultz, 2013), (3) DA mediates motivation or incentive salience (Berridge, 2007) and (4) DA energizes behavior (Salamone, Correa, Farrar, & Mingote, 2007).

Each viewpoint is based on extensive experimental data, and it is believed that they are not mutually exclusive. However, there are at least two main discrepancies within these explanations. First, it is paradoxical that the activation of the same brain substratum should be both reinforcing and drive-inducing, so hypotheses (1) and (3) are conflicting. Second, it is hard to explain the activation of the DA system, which is believed to be engaged in reward processes, in response to aversive stimuli. If midbrain dopamine neurons actually encode value-related signals, their activity should be inhibited by aversive stimuli because aversive stimuli have negative motivational values. However, the results are inconsistent, with many studies showing excitation by aversive stimuli (Anstrom & Woodward, 2005; Anstrom, Miczek, & Budygin, 2009; Brischoux et al., 2009, Matsumoto & Hikosaka, 2009; Wang & Tsien, 2011). At the intersection of such contradictory aspects of DA system functions is instrumental active avoidance, because in this paradigm aversive stimulation as well as motivational and reward processes (in the form of omission of a harmful event) take place.

Thus, the present study recorded VTA neurons during the acquisition of active avoidance. The study of neuronal activity in behavior is one of the most appropriate methods of functional specificity investigation because of the possibility of precise spatial localization of recorded changes with high temporal resolution and minimal invasiveness. Previous studies have shown that the phasic activation of midbrain dopamine neurons strongly resembles the prediction error; that is, dopamine neurons burst when an animal receives an unexpected reward (Schultz, 1998). However, if the animal has learned to associate a conditioned stimulus (CS) with a reward, dopamine neurons burst at the presentation of the CS but not the reward. Furthermore, if the CS is presented but then the predicted reward is omitted, dopamine neurons are inhibited at the approximate time at which the reward should have been delivered. It was assumed that all of these findings are consistent with the idea that dopamine neurons report reward prediction errors (RPE): a positive prediction error signal that the outcome was better than expected, or a negative prediction error signal that the outcome was worse than expected. It has been suggested that such activity can in turn serve as a teaching signal in the projection structures of the VTA (Schultz, 2013). This assumption underlies the most influential hypothesis on the specific role of mesencephalic dopamine in learning processes. In many neuronal studies which consider this idea, the obtained results were in good agreement with the viewpoints basic assumptions (Satoh, Nakai, Sato, & Kimura, 2003; Bromberg-Martin et al., 2010; Cohen, Haesler, Vong, Lowell, & Uchida, 2012; Schultz, 2013). In most of these studies, classical conditioning procedures were used.

Data about the relationship between DA neurons activity and instrumental movement are rather contradictory (Schultz, Ruffieux, & Aebischer, 1983; Schultz, 1986; Nishino, Ono, Muramoto, Fukuda, & Sasaki, 1987; Satoh et al., 2003). In electrochemical studies of food conditioning, it was shown that DA release precedes the onset of instrumental movement execution (Roitman, Stuber, Phillips, Wightman, & Carelli, 2004; Puryear, Kim, & Mizumori, 2010; Cacciapaglia, Wightman & Carelli, 2011; Oleson, Gentry, Chioma, & Cheer, 2012). In response to the unexpected reward omission, DA level in the projection structures increases, and conversely, when the rate of food supply suddenly increases, a sharp decrease in the DA concentration is observed (Richardson & Gratton, 1996, 1998, 2008). Together, these findings contradict the notion of the role of dopamine as a signal of RPE.

While it is widely assumed that DA neurons exhibit homogenous reward coding across the entire population (Schultz, 1998), studies of dopaminergic system activity under immediate aversive stimuli exposures are also a source of controversy. This is because a neurons response to aversive events provides a crucial test of its functions in motivational control. In these studies, conflicting results have been obtained in anesthetized (Ungless, Magill, & Bolam, 2004; Brischoux, Chakraborty, Brierley, & Ungless, 2009; Brown, Henny, Bolam, & Magill, 2009) as well as in conscious animals (Anstrom & Woodward, 2005; Anstrom, Miczek, & Budygin, 2009; Wang & Tsien, 2011). Rather contradictory are data on the location and properties of neurons somehow reacting to aversive stimulation in the VTA and the surrounding areas

(Brischoux et al., 2009; Matsumoto & Hikosaka, 2009; Lammel, Ion, Roeper, & Malenka, 2011; Valenti, Gill, & Grace, 2012). Most histochemically identified DA neurons are inhibited in response to fear of the CS (Mileykovskiy & Morales, 2011). In the context of classical conditioning based on unpleasant but not harmful stimuli, conflicting data about the sign of the reaction were obtained (Joshua, Adler, Mitelman, Vaadia, & Bergman, 2008; Matsumoto & Hikosaka, 2009; Cohen et al., 2012; Wang & Tseien, 2012). Studies of DA neurons responding during active avoidance when an animal learns to control aversive stimuli are rare (Mirenovizc & Schultz, 1996) and that study was carried out under mild defensive incentives.

Method

Animals. Four adult male rabbits weighing 2.5-3.5 kg were involved in an experiment which recorded neuronal activity in conditions of free behavior.

Surgeries. Surgical procedures were performed under Nembutal anesthesia (sodium ethaminal, 55 mg/kg) after a period of acclimating the animal to the experimental apparatus for 7 to 14 days. The animal’s head was fixed in a stereotaxic apparatus and two trepanned openings were made in the skull without damaging the dura mater. One opening was 2.5 mm in diameter and was either in the region of the projection of the VTA, with coordinates of AP = 6-7, L=l-2, H = -(4-6) (Fig. 1; Fifkova & Marsala, 1960); a micromanipulator holder was positioned over the first hole. The second hole was made in the occipital region to accommodate the indifferent electrode, which consisted of a steel screw contacting the dura mater.

Unit activity collection. During the experiment, a micromanipulator was used to insert a microelectrode (a tungsten wire 100 mm in diameter and coated with viniflex lacquer, tip diameter 5-7 mm, resistance 800-2500 kQ at 1 kHz) into the brain. A biopotential minitransmitter was also attached to the head, and transmitted neuron spikes to the amplification system, from which signals were transferred to an amplitude-time spike discriminator. Standard impulses from the discriminator output were passed to a computer.

Behavioral procedures. Experiments involving the recording of neuronal activity started 5 to 7 days after surgery. Animals were trained and experiments were performed in an electrically shielded sound-attenuated chamber with dimensions of 60 cm x 80 cm and fitted with an observation system for monitoring the animal’s behavior. The defensive conditioned reaction (CR) was developed by pairing the CS with electrocutaneous stimulation (ECS) applied to the ear as an unconditioned stimulus (US). The CS was switched off when the animal completed the speciesspecific ear-twitching response within seven seconds of the start of the CS during the initial period of acquisition of the CR, and within four seconds of the start of the CS once the reflex had formed. When this occurred, ECS was stopped. If the rabbit did not perform the conditioned reflex movement, the US was presented during the last second of the CS. The CS was a sound tone at 400 Hz; ECS consisted of square-wave impulses of 10 ms in duration, at frequency of 10 Hz and amplitude of 10-30 .generated using an ES-50-1 electrostimulator. ECS was delivered via electrodes attached to the animals ear; the duration of stimulation was one second. ECS parameters did not exceed threshold values inducing responses in the form of twitching movements of the ear and were selected individually in each experiment.

Statistics. Initial processing of neuronal activity consisted of averaging the discharge frequencies over specified periods of time; discharge frequencies were determined at the following intervals: 1) during the single second before presentation of the CS; 2) during the first 100 ms and one second of exposure to the CS; 3) during the period between the end of one second of CS exposure and the beginning of the period preceding the operant movement (only for latent periods of movement responses of greater than two seconds; 4) during the second immediately preceding the movement; 5) during the second immediately after completion of the movement; and 6) during the second after the CS was switched off. When there was no CR to the CS, the mean discharge frequency was determined during each second of exposure to the CS and the poststimulus period.

Statistical analysis of data consisted of 1) calculation and construction of peristimulus histograms of neuron spike activity in the presence and absence of positive behavioral responses; 2) the use of the two-tailed signs test to establish significant (p < 0.05) changes in discharge frequencies in response to the CS; and 3) the use of the two-tailed Mann-Whitney test to identify significant (p < 0.05) differences in discharge frequency and pattern in background conditions, in responses to the CS, and in the poststimulus period in the presence and absence of behavioral conditioned responses. The significance of differences in the percentage compositions of the neuron groups identified here was determined by analysis of 2 x 2 tables. Statistical analysis of results was performed using Statistica 6.0 for Windows.

Results

Since there is no generally accepted electrophysiological criteria that identify DA neurons and applicability of such criteria in different experimental conditions is under debate in the literature (Ungless & Grace, 2012), we hereby describe all the neurons recorded in the VTA. During instrumental conditioning, the activity of 60 VTA neurons was recorded. Statistically significant (p < 0.05) changes in the frequency and pattern of discharge in response to the CS were found for 29 (48 %) of the 60 VTA neurons. Evaluation of the sign of reaction to the CS revealed a statistically significant (p < 0.01) prevalence of activation responses. So, in 18 (62 %) of the 29 recorded cells with a firing rate modification, an increase in discharge frequency was observed while in five (17 %) of the neurons a decrease occurred.

The activity of the VTA neurons in response to CS when performing the instrumental movement. The most characteristic patterns of discharge during the period of CS were defined as follows: (1) the reaction at the CS onset, which manifests itself during the first 100 ms or the first 1000 ms of the CS; (2) a change in discharge frequency during the period of the CS immediately before and/or during instrumental movement; (3) an increase or decrease of the firing rate throughout the action of the CS, and excitatory-inhibitory or inhibitory-excitatory responses, which together constituted a minority of recorded neurons and were referred to as “others”.

The most numerous reaction was an excitatory response to the onset of the CS, which was observed in 13 eurons (45 % of the number of cells reactive to CS) (Fig. 2), as well as the activation associated with the instrumental movement, registered in 11 cells (42 % of the number of cells changing activity in the period of the CS and analyzed in relation to movement) (Fig. 3).

Three of eight cells with a reaction to the CS onset, as recorded in the presence and in the absence of instrumental responses to CS, showed significant differences in the severity of this reaction (see Fig. 2).

In the post-stimulus period (PSP) after the instrumental movement or lack thereof, the one-second time interval immediately following the turning off of the CS was analyzed. Significant differences in discharge rates compared to background activity during this period were found in 13 of the 54 neurons (p < 0.05).

Dynamics of neuronal activity at different stages of the conditioning. The activity of VTA neurons was recorded at different stages of conditioning. These stages differed in terms of the probability of instrumental responses to CS, and in the amount of interstimulus reactions, in latent periods of the motor responses. Two main stages were identified: generalization, and specialization of CR.

The generalization stage was characterized by: an increase in the number of positive behavioral responses to the CS from 0% to 60-80% in different animals; large numbers of intersignal reactions; a variable movement repertoire in response to the CS (startle responses to switching on of the CS, active movement in the arena, turning of the head towards the sound source, intense twitching movements of the head, and multiple twitching movements of the ear lasting into the poststimulus period); and an unstable latent period for the behavioral response. Since the instrumental movement was identical to those caused by unconditioned stimulus action, and strength of US was matched to the threshold level, the development of avoidance was quite fest, and the period before the first instrumental movement emergence was short The probability of performing operant movements in response to the CS reached an average of 70%, though increases in positive conditioned responses at this stage were accompanied by increases in the proportion of operant responses during the interstimulus intervals. At the specialization stage, the proportion of adequate responses was greater than 70% while the proportion of intersignal responses relative to the number of CS presentations decreased to 0-30% and the efferent generalization of responses to the CS was transformed into single twitching responses with a stable latent period.

This stage has been described in greater detail in Ivlieva and Timofeeva (2003).

A comparison of the dynamics of learning and of neuronal activity has revealed a small decline in the proportion of neurons reactive to CS at the stage of specialization compared with the stage of generalization of CR (see Table 1). The acquisition of CR was sufficiently rapid, therefore in the period before the first instrumental movements only two neurons reactive to CS were recorded, but it should be noted that the responses of the cells were inhibitory in nature.

At different stages of learning, there was a significant redistribution of ratio of neurons with certain types of reactions (2x2 table, x² = 9.12, p = .0025). From the initial to the final stage of learning the proportion of neurons reactive to the CS onset decreased significantly, while the proportion of cells that changed activity in connection with the instrumental movement significantly increased (see Table 1, Fig. 4).

In the post-stimulus period (PSP), the proportion ofneuronsmodifyingactivityascompared with the background level were equal in both stages of the conditioning. However, as in the case of the reaction to CS, the neurons redistributed in percentages of inhibitory and excitatory responses (see Table 1). The ratio of activity in the period of the CS and in the PSP also changed. In the generalization stage, no neuron was recorded with the response restricted by only the period of the CS, and at the stage of specialization of CR such neurons represent the majority of the reactive cells as shown in Table 1. That is, we can talk about the transition from the more generalized reactions to CS to the time-limited responses as the result of instrumental reflex consolidation.

Discussion and Conclusions

In the present study the activity of neurons in the ventral tegmental area (VTA) is investigated during the acquisition of avoidance conditioning in a freely moving rabbits. 48 % of the neurons investigated were shown to alter their activity in response to conditioned stimulus (CS) during the experiment.

Mirenowicz and Schultz, during avoidance conditioning in macaques (Mirenowicz & Schultz, 1996), showed that only about 10% of the presumably dopaminergic neurons recorded change their discharge rate. This discrepancy with our data has at least three possible explanations: firstly, these authors used tactile and gustatory (not painful) stimuli as unconditioned stimuli; secondly, they recorded neural activity at the stage of an automated instrumental movement; and thirdly, they used a short CS and the neural activity in connection with the movement was not considered by the authors. In the present study, we also found that at the stage of specialization, the proportion of neurons reactive to the onset of CS decreased, and the proportion of neurons activated before or during of the movement increased, and thus the percentage of neurons reactive to the CS onset at the stage of specialization in our study is actually similar to that identified by Mirenowicz and Schultz (1996).

As was mentioned in the introduction, current hypotheses suggest that DA neurons signaling is associated with (1) stimuli with hedonic value (Wise & Bozarth, 1985), (2) situations when positive outcomes (rewards) are better than expected thus signaling RPE that drives learning (Schultz, 2013), (3) motivation or incentive salience (Berridge, 2007) and (4) energization of behavior (Salamone, Correa, Farrar, & Mingote, 2007).

Our results are not consistent with the hedonic hypothesis (1), since this hypothesis predicts that DA neurons’ activation may be strongly associated with termination of the US as well as of CS, given that relief from shock or avoidance of shock is pleasurable. The activation of neurons in response to the CS onset, which is more pronounced at the stage of generalization when an aversiveness of stimulus is doubtless, also contradicts the hedonic hypothesis. Finally, the preferential activation of neurons before and during instrumental movement in the late stage of learning (stage of specialization) does not correspond with any pleasurable events.

According to RPE hypothesis (2), neuronal activation should be expected primarily in sessions when the first successful instrumental movements lead to the abolition of the US; that is, immediately after the turning off of the CS, which we have observed in the activity of only one neuron. This, of course, might be due to the fact that this stage is very short and a few specific cells recorded at this stage showed no expected response. However, the predominance of excitatory reactions of neurons in response to the onset of the aversive stimulus and in the period before movement execution is difficult to explain in terms of the RPE hypothesis, if we do not assume that the majority of neurons recorded are not dopaminergic.

To a greater extent, our data agree with the salience hypothesis (3): there is a preferential activation of cells in response to CS onset on the stage of generalization, as well as a significant decrease in the proportion of these cells during the transition to the stage of specialization, when, a) the skill becomes automated, and b) the number of electric shocks is reduced, which together reduces the significance of the stimulus. However, in light of this hypothesis, the meaning of neuronal activation before the instrumental movement at the stage of specialization is not entirely clear. Nonetheless, of all experimental events the movement becomes the most significant one.

Our data are well suited to the last hypothesis (4): in the stage of specialization, a majority of reactive cells significantly changed, with most increasing their discharge rate during and sometimes before the execution of the instrumental movement. Also, in the stage of generalization, the activation responses to the CS onset were predominant which is consistent with the traditional view that effective stimuli for discharge responses of DA neurons should have a trigger or releasing function for immediate behavioral events that are important to the animal (Schultz, 1986).

This interpretation allows us to compare our results with those obtained under other behavioral paradigms where rewards instead of punishments were used. Similar to the dynamics of VTA neuronal reactivity as the consolidation of a CR are the dynamics of discharge pattern changes of a single neuron during the acquisition of delayed instrumental food reactions described by Nishino et al. (1987). They show that at the first trials in a situation with delay, the neuron responded to the presentation of the CS and to the series of motor responses, but in subsequent trials only the response that was associated with the instrumental movement remained. Sato et al. (2003) did not find significant modulation of neuronal activity in connection with appetitive instrumental movement, but unlike the behavioral task in our study, movement itself was the simplest part of the behavioral task for the animal in that experiment. After analyzing responses to the onset of stimuli initiating instrumental movements the authors suggested that the magnitude of neuronal reponse reflects the motivational properties of the CS. Applying this interpretation to our data, we can assume that if the motivational significance of the CS as a precursor of pain decreased during the avoidance acquisition, the decrease in the proportion of neurons activated in response to the onset of CS should be expected. Thus, our data confirm the assumption of Sato and colleagues.

Also of interest is an association of the neuronal response amplitude to CS onset with probability and intensity of the instrumental response (Fig. 2), which confirms the VTA involvement in the processes of sensorimotor integration (Berrige, 2007). In support of such participation is the rearrangement of general reactivity patterns in the direction from the sensory to the motor aspect during the consolidation of instrumental skills: as the instrumental movement is becoming more important than the CS, the response of neurons to CS decreases.

Of considerable interest are the data on the relationship between neuronal activity in the VTA and reinforcement in conditions of avoidance when the removal of the pain stimulus is the reward. We found great heterogeneity of neuronal responses in the period of poststimulus activity which is, on the one hand, consistent with different patterns of DA release in projection structures in aversive situations (Budygin et al., 2012, Badrinarayan et al., 2012), and on the other hand, requires assumptions about the neurochemical nature of the neurons. We note here that most of the cells inhibiting in this period (6 of 7) did not meet the criteria of DA neurons.

Data on the change of the dopamine level in the basic projection structures of the VTA during instrumental conditioning also showed maximum activity of the DA system at the moment of the instrumental movement. Studies conducted in unrestrained animals engaged in more naturalistic motivated behaviors to obtain rewards demonstrate that dopaminergic system activation precedes and accompanies the instrumental movement (Richardson & Gratton, 1996, 1998, 2008; Roitman et al., 2004; Puryear et al., 2010; Cacciapaglia et al., 2011; Wassum, Ostlund, & Maidment, 2012).

While it is well accepted that rewards and their CS elicit excitation of DA neurons, it remains a topic of discussion whether DA release plays a similar role in different aversive situations. Our data, as well as the results of a voltammetric study of Oleson et al. (2012), extend such a pattern of DA system activation on learned behaviors aimed at the reduction of harm or at punishment avoidance. Moreover, the association of decreased DA activity in aversive situations with a behavioral arrest connected with a failure to use active behavioral strategies or with a lack of behavioral control (Mileykovsky & Morales, 2011; Tye et al., 2011; Oleson et al., 2012), as well as our demonstrated dynamics of DA system activation from response to cue onset to activation before movement, suggests a crucial role of the DA system in the formation of active behavioral strategies.

References

Anstrom, К. К., Miczek, К. A., & Budygin, E. A. (2009). Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats. Neuroscience, 161(1), 3-12. doi: 10.1016/i.neuroscience.2009.03.023

Anstrom, К. К. & Woodward, D. J. (2005). Restraint increases dopaminergic burst firing in awake rats. Neuropsychopharmacology, 30(10), 1832-1840. doi: 10.1038/sj.npp.l300730

Badrinarayan, A., Wescott, S. A., Vander Weele, С. M., Saunders, В. T., Couturier, В. E., Maren, S., & Aragona, B. J. (2012). Aversive stimuli differentially modulate real-time dopamine transmission dynamics within the nucleus accumbens core and shell. Journal of Neuroscience, 32(45), 15779-15790. doi: 10.1523/INEURQSCL355712.2012

Berridge, К. C. (2007). The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology (Bert), 191. 391-431. doi: 10.1007/s00213-006-0578-x

Brischoux, E, Chakraborty, S., Brierley, D. I., & Ungless, M. A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proceedings of the National Academy of Sciences USA, 106(12), 4894-4899. doi: 10.1073/ pnas.0811507106

Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive, and alerting. Neuron, 68(5), 815-834. doi:10.1016/j. neuron.2010.11.022

Brown, M. T, Henny, P., Bolarn, J. P., & Magill, P. J. (2009). Activity of neurochemically heterogeneous dopaminergic neurons in the substantia nigra during spontaneous and driven changes in brain state. Journal of Neuroscience, 29(9), 2915- 2925. doi: 10.1523/INEUROSCI.4423-08.2009

Budygin, E. A., Park, J., Bass, С. E., Grinevich, V. P., Bonin, K. D., & Wightman, R.M. (2012). Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience, 201, 331-337. doi: 10.1016/). neuroscience.2011,10.056

Cacciapaglia, E, Wightman, R. M., & Carelli, R. M. (2011). Rapid dopamine signaling differentially modulates distinct microcircuits within the nucleus accumbens during sucrose- directed behavior. Journal of Neuroscience, 31,13860-13869. doi: 10.1523/TNEUROSCI. 1340-11,2011

Cohen, J. Y., Haesler, S., Vong, L., Lowell, В. B., & Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature, 482(7383), 85-88. doi: 10.1038/naturel0754

Fifkova, E. & Marsala, J. (1960). Stereotaxic atlas for the cat, rabbit and rat. In: Electrophysiological methods in biological research. Prague. 426 p.

Flaherty, A.W (2005). Frontotemporal and dopaminergic control of idea generation and creative drive. Journal of Comparative Neurology, 493(1), 147-153. doi: 10.1002/cne.20768

Ivlieva, N.Y. (2010). [Involvement of the mesocorticolimbic dopaminergic system in adaptive behavior]. In Russian. Zhur- nal Vysshei Nervnoi Deyatel’nosti imeni I. P. Pavlova, 60(3), 259-278. Translation into English published as: Ivlieva, N.Yu. (2011). Involvement of the Mesocorticolimbic Dopaminergic System in Adaptive Behavior. Journal of Neuroscience and Behavioral Physiology, 41 (7), 715-729. doi: 10.1007/ si1055-011-9477-7

Ivlieva, N.Y. & Timofeeva, N. O. (2003). Neuron activity in the pedunculopontine nucleus during an operant conditioned defensive reflex. Neuroscience and Behavioral Physiology, 33(5), 499-506. doi: 10.1023/A: 1023419418869

Joshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. (2008). Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. Journal of Neuroscience, 28(45), 11673-11684. doi: 10.1523/TNEUROSCL3839-08.20Q8

Lammel, S., Ion, D. I., Roeper, J., & Malenka, R. C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron, 70(5), 855-862. doi: 10.1016/j.neuron.2011,03.025

Matsumoto, M. & Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature, 459(7248), 837-841. doi: 10.1038/ nature08028

Mileykovskiy, B. & Morales, M. (2011). Duration of inhibition of ventral tegmental area dopamine neurons encodes a level of conditioned fear. Journal of Neuroscience, 31(20), 7471- 7476. doi: 10.1523/1NEUROSCL5731-10.2011

Mirenowicz, J. & Schultz, W. (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature, 379,449-451. doi: 10.1038/379449a0

Nishino, H., Ono, T, Muramoto, K., Fukuda, M., & Sasaki, K. (1987). Neuronal activity in the ventral tegmental area (VTA) during motivated bar press feeding in the monkey. Brain Research, 413(2), 302-313. doi: 10.1016/ 0006-8993(87)91021-3

Oleson, E. B., Gentry, R. N., Chioma, V. C., & Cheer, J. F. (2012). Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. Journal of Neuroscience, 32,14804-14808. doi: 10.1523/ TNEUROSCI.3087-12.2012

Puryear, С. B., Kim, M. J., & Mizumori, S. J. (2010). Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent. Behavioral Neuroscience, 124, 234-247. doi: 10.1037/a0018865

Richardson, N. R. & Gratton, A. (1996). Behavior-relevant changes in nucleus accumbens dopamine transmission elicited by food reinforcement: an electrochemical study in rat. Journal of Neuroscience, 16, 8160-8169.

Richardson, N. R. & Gratton, A. (1998). Changes in medial prefrontal cortical dopamine levels associated with responsecontingent food-reward: an electrochemical study in rat. Journal of Neuroscience, 18, 9130-9138.

Richardson, N. R. & Gratton, A. (2008). Changes in nucleus accumbens dopamine transmission associated with fixed- and variable-time schedule-induced feeding. European Journal of Neuroscience, 27(10), 2714-2723. doi: 10,111 l/j.l460-9568.2008.06236.x

Roitman, M. E, Stuber, G. D., Phillips, P. E., Wightman, R. M., & Carelli, R. M. (2004). Dopamine operates as a subsecond modulator of food seeking. Journal of Neuroscience, 24, 1265-1271. doi: 10.1523/TNEUROSCI.3823-03.2004

Rossi, M. A., Sukharnikova, T, Hayrapetyan, V. Y, Yang, L., & Yin, H. H. (2013). Operant self-stimulation of dopamine neurons in the substantia nigra. PLoS One, 8(6):e65799. doi: 10.1371/ journal.pone.0065799

Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. M. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berk), 191,461-482. doi: 10.1007/s00213-006-0668-9

Satoh, T, Nakai, S., Sato, T, & Kimura, M. (2003). Correlated coding of motivation and outcome of decision by dopamine neurons. Journal of Neuroscience, 23(30), 9913-9923.

Schultz, W, Ruffieux, A., & Aebischer, P. (1983). The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Experimental Brain Research, 51, 377-387. doi: 10.1007/BF00237874

Schultz, W. (1986). Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. Journal of Neurophysiology, 56(5), 1439-1461.

Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80,1-27.

Schultz, W. (2013). Updating dopamine reward signals. Current Opinion in Neurobiology, 23(2), 229-238. doi: 10.1016/i. conb.2012.11.012

Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I.B., Deisseroth, K., & Janak, P. H. (2013). A causal link between prediction errors, dopamine neurons and learning. Nature Neuroscience, 16(7), 966-973. doi: 10.1038/nn,3413

Steinberg, E. E. & Janak, P. H. (2013). Establishing causality for dopamine in neural function and behavior with optogenetics. Brain Research, 1511, 46-64. doi: 10.1016/j. brainres.2012.09.036

Tsai, H. C., Zhang, E, Adamantidis, A., Stuber, G. D., Bond, A., de Lecea, L., Deisseroth, K. (2009). Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science, 324(5930), 1080-1084. doi: 10.1126/science.U68878

Ungless, M. A., Magill, P. J., & Bolarn, J. P. (2004). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science, 303(5666), 2040-2042. doi: 10.1126/science.l093360

Ungless, M. A., & Grace, A. A. (2012). Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends in Neuroscience, 35(7), 422-430. doi: 10.1016/i.tins.2012.02.003

Valenti, O., Gill, К. M., & Grace, A. A. (2012). Different stressors produce excitation or inhibition of mesolimbic dopamine neuron activity: response alteration by stress pre-expo- sure. European Journal of Neuroscience, 35(8), 1312-1321. doi: 10.1111/І.1460-9568.2012.08038.Х

Wang, D. V. & Tsien, J. Z. (2011). Convergent processing of both positive and negative motivational signals by the VTA dopamine neuronal populations. PLoS One, 6(2):el7047. doi: 10.1371/journal.pone.0017047

Wassum, К. M., Ostlund, S. B., & Maidment, N. T. (2012). Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task. Biological Psychiatry, 71, 846-854. doi: 10.1016/i.biopsych.20U.12.019 Wise, R. A. & Bozarth, M. A. (1985). Brain mechanisms of drug reward and euphoria. Psychiatric Medicine, 3, 445-460.

Wise, R. A. (2012). Dual roles of dopamine in food and drug seeking: The drive-reward paradox. Biological Psychiatry, 73(9), 819-826. doi: 10.1016/i.biopsych.2012.09.001