Neural systems of reinforcement for drug addiction - from actions to habits to compulsions
- Type:#article
- Year read:#read2017
- Subject: Habits
- Bibtex: @everitt2005
- Bibliography: Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience, 8(11), 1481–1489. http://doi.org/10.1038/nn1579
Why and when I was reading this
Key takeaways
- During initial drug use, a drug is voluntarily taken because it has reinforcing effects. This behavior becomes habitual over time and ultimately compulsive. At the neural level, this represents a shift from prefrontal to striatal control over drug seeking and drug taking behavior.
”Overall, we hypothesize that the transition from voluntary actions (governed mainly by their consequences) to more habitual modes of responding in drug seeking behavior represents a transition from prefrontal cortical to striatal control over responding, and from ventral to more dorsal striatal subregions.”
Learning mechanisms underlying drug-taking
- Instrumental learning, i.e. the consequences of drug-seeking and drug-taking (e.g. arousal, relaxation)
- Classical conditioning
Reinforcers have been conceptualized in three different ways
Reinforcers produce…
- Drive reduction according to a homeostatic regulatory model
- Memory consolidation
- Incentive-motivational effects (i.e. an expected reinforcer evokes approach behavior or physiological adjustment)
What is left out of all of these is subjective responses or feelings associated with drug effects, which will eventually be needed in a complete theoretical framework. These responses have not yet been linked to particular brain regions or networks.
Second-order reinforcement (Box 2)
- Drug-associated conditioned reinforcers (CR) maintain drug seeking behavior over relatively prolonged periods.
- This explains why relapse is so high, the ability of drug-associated CRs maintain or reinstate drug seeking responses may actually increase with the duration of withdrawal
p.1481: Drug addiction is increasingly viewed as the endpoint of a series of transitions from initial drug use—when a drug is voluntarily taken because it has reinforcing, often hedonic, effects—through loss of control over this behavior, such that it becomes habitual and ultimately compulsive. Here we discuss evidence that these transitions depend on interactions between pavlovian and instrumental learning processes. We hypothesize that the change from voluntary drug use to more habitual and compulsive drug use represents a transition at the neural level from prefrontal cortical to striatal control over drug seeking and drug taking behavior as well as a progression from ventral to more dorsal domains of the striatum, involving its dopaminergic innervation. These neural transitions may themselves depend on the neuroplasticity in both cortical and striatal structures that is induced by chronic self-administration of drugs. -- Highlighted 2 nov. 2017
p.1481: improved understanding of associative learning mechanisms that conceive of behavioral output as an interaction between pavlovian and instrumental learning processes -- Highlighted 2 nov. 2017
p.1481: In particular, we argue that it is the sense of expectancy, or perhaps even more importantly, the sense of ‘control’ over such interoceptive and exteroceptive states, including the overall level of arousal accompanying them, acquired through action-outcome learning (Box 1) that constitutes instrumental drug reinforcement. -- Highlighted 2 nov. 2017
p.1481: In particular, the description of two processes that seem to function partly in parallel, but with the second eventually dominating behavioral output, has led to the concepts of action-outcome and stimulus-response (‘habit’) learning. Here we elaborate the hypothesis that these behavioral processes can be mapped onto the parallel and serial, dynamic functioning of corticostriatal circuitry (Fig. 1) to mediate the ‘switches’4,5 between drug reinforcement, drug abuse and drug addiction. -- Highlighted 2 nov. 2017
p.1481: These motivational effects of CSs can be ascribed to a hypothetical process of pavlovian arousal, which serves to energize or activate responding, whether in terms of enhanced locomotor activity or increasing rates of instrumental (operant) behavior. -- Highlighted 2 nov. 2017
p.1482: either phenomenon (neither approach to a CS predictive of a drug, nor enhancement of drug seeking by the unexpected presentation of a drugassociated CS) has been clearly demonstrated in laboratory studies of drug seeking or relapse -- Highlighted 2 nov. 2017
p.1482: Therefore, it might logically be thought that pavlovian approach is involved in maladaptively attracting humans toward sources of addictive drug reinforcers, and that drug-associated CSs that occur unexpectedly invigorate their efforts to seek and take or ‘want’ drugs as emphasized in the incentive salience theory of addiction -- Highlighted 2 nov. 2017
p.1483: Burgeoning evidence links the orbitofrontal cortex to the sensory representation of reinforcers as well as their value and the relative utility of different courses of action producing them. However, the neural mediation of those aspects of the reinforcer conveying its hedonic properties remain elusive because the useof functional neuroimaging procedures thus far has confounded the sensory properties of a reinforcer with hedonic subjective responses associated with it. -- Highlighted 6 nov. 2017
p.1483: Although these subjectively loaded terms referto hypothetical processes of attribution that are associatedwith reinforcement, the processes themselves have never been identified or localized to particular brain regions or networks. -- Highlighted 6 nov. 2017
p.1483: This dopamine-dependent potentiation of conditioned reinforcement is a key component of the reinforcing effects of stimulant drugs such as cocaine, amphetamine and nicotine and likely other drugs as well. -- Highlighted 2 nov. 2017
p.1483: Responding in the interim is maintained by the presence of drug-associated CSs that are presented as a consequence of instrumental seeking responses (Box 2). The CSs must be presented as conditioned reinforcers (that is, their presentation must depend on the animal’s behavior); merely presenting them unexpectedly fails to increase drug seeking18. This seems to contradict the ‘incentive salience’ model of drug seeking behavior, which would predict enhancement from pavlovian, or unexpected, presentations of the CS. -- Highlighted 2 nov. 2017
p.1484: ‘Drug seeking’ can be studied in a number of ways. A ‘second-order schedule of drug reinforcement’ (in contrastto continuous reinforcement) emphasizes the role of drugassociated conditioned reinforcers in maintaining drug seeking behavior over relatively prolonged periods24. -- Highlighted 6 nov. 2017
p.1484: The ability of drug-associated conditioned reinforcers to maintain or reinstate drug seeking responses may actually increase with the duration of withdrawal -- Highlighted 6 nov. 2017
p.1484: It is easy to devalue ingestive reinforcers, but it is much more difficult to devalue intravenously selfadministered drugs such as cocaine. -- Highlighted 6 nov. 2017
p.1484: In habit learning, instrumental performance is acquired through the association of responses with stimuli present during training. It therefore reflects the formation of stimulus-response associations, and reinforcers primarily serve the function of strengthening the stimulus-response association but do not become encodedas a goal. Therefore, devaluing the reinforcer does not affect instrumental responding acquired by habit learning. -- Highlighted 6 nov. 2017
p.1485: presented as a consequence of instrumental responses—as conditioned reinforcers. Hypothetically, such stimulus-response associative (‘habit’) learning occurs in parallel with instrumental action-outcome learning but, with extended training, eventually dominates behavioral output. -- Highlighted 6 nov. 2017
p.1485: The experiments discussed above and further below show that the nucleus accumbens core, as distinct from the shell, is important in the maintenance of instrumental behavior involving delays in the provision of cocaine, in particular in the capacity of CSs to bridge that delay. -- Highlighted 6 nov. 2017
p.1485: Evidence for this concept of drug addiction as a maladaptive and persistent habit comes from several sources, which also increasingly point to the dorsal striatum as a major contributor to this form of learning. An operational definition of a habit is that the behavior continues even after the controlling influence of the goal is reduced by devaluation procedures, such as satiation or even poisoning in the case of a food goal -- Highlighted 6 nov. 2017
p.1485: Important issues to be resolved include how the contributory factors such as pavlovian arousal and instrumental reinforcement, including conditioned reinforcement, are integrated within the nucleus accumbens circuitry. Perhaps the most obvious mechanism could stem from the cascading loop circuitry by which output from the nucleus accumbens shell can influence the functioning of the ascending dopamine projections to the core, and similarly, from the output of the core via the substantia nigra to other domains of the dorsal striatum -- Highlighted 6 nov. 2017
p.1485: In theoretical terms, it seems reasonable to characterize such compulsive behavior as a maladaptive stimulus-response habit in which the ultimate goal of the behavior has been devalued so that the behavior is not directly under the control of the goal20,25. Rather, responding is governed by a succession of discriminative stimuli, which also function—when they are -- Highlighted 6 nov. 2017
p.1486: associative information in the basolateral amygdala is translated into goal-directed, drug seeking behavior via its interactions with the nucleus accumbens core -- Highlighted 6 nov. 2017
p.1486: From the neurocomputational perspective, the ventral and dorsal striatum could conceivably correspond to the ‘critic’ and ‘actor’ components, respectively, of contemporary models of reinforcement learning49. The critic learns to predict future rewards, and the actor maintains information about the rewarding outcome of actions; in other words, the interaction of pavlovian and instrumental learning through the intermediary of conditioned reinforcement. -- Highlighted 6 nov. 2017
p.1486: There is general consensus on the functions of the amygdala, nucleus accumbens core and OFC and their interactions in the control over goaldirected behavior by discrete CSs acting as conditioned reinforcers. -- Highlighted 6 nov. 2017
p.1486: Thus, the mechanisms underlying drug taking are dissociable from those underlying drug seeking. -- Highlighted 6 nov. 2017
p.1486: Hippocampal contextual information and amygdala-dependent discrete CSs may compete for control over goal-directed behavior3. -- Highlighted 6 nov. 2017
p.1487: Habitual responding by itself, however, does not capture the persistent, indeed, compulsive aspects of ‘out-of-control’ drug bingeing; some additional factor seems to be required. -- Highlighted 6 nov. 2017
p.1487: We hypothesize that, in psychological terms, hippocampal mechanisms provide the contextual background that defines the motivational arousal upon which goal-directed responding occurs. Inactivating this mechanism at the hippocampus or nucleus accumbens shell level reduces exploration, activity and contextual conditioning and also the potentiation of these responses by psychomotor stimulants—providing, therefore, an additional basis for understanding the reinforcing effects of drugs acting on the dopamine and other systems in the nucleus accumbens -- Highlighted 6 nov. 2017
p.1487: Overall, we hypothesize that the transition from voluntary actions (governed mainly by their consequences) to more habitual modes of responding in drug seeking behavior represents a transition from prefrontal cortical to striatal control over responding, and from ventral to more dorsal striatal subregions -- Highlighted 6 nov. 2017