Study sheds new light on dopamine’s contribution to reinforcement learning

9 Min Read

A physiologically relevant frequency of dopamine stimulation (20 Hz) does not function as a meaningful reward, but high-frequency dopamine stimulation (50 Hz) functions as a reward that is encoded as a specific sensory event. Top: Histological verification showing A) bilateral Cre-dependent ChR2 expression in TH-Cre rats, B) colocalization of TH and virus expression approximated ~90%, and C) diagram of minimum and maximum virus expression and fiber placement. Left column: Schematic representation of the task design using one counterbalanced example, which consisted of D) Pavlovian conditioning, E) Instrumental conditioning, and F) the PIT test. Rats first learned that two auditory cues (e.g., click and white noise) led to two outcomes (e.g., dopamine stimulation and pellets), then learned to press the lever twice that led to the two outcomes. Finally, rats were presented with the two auditory cues and given the opportunity to press either lever, without reward feedback. Middle column: G) Rats in the 20 Hz group (n = 6) showed an increase in food port entry during the pellet-paired stimulus, but not the dopamine-paired stimulus. These rats showed equivalent increases in locomotor activity when learning both stimuli. H) During instrumental conditioning, where rats learned to press the lever for the two outcomes, rats in the 20 Hz group showed robust lever press responses for the pellets, but not for the dopamine stimulation. I) In the final PIT test, when the pellet-paired cue was presented, these rats showed significant increases in the response to the pellet-paired lever, indicative of specific PIT. However, they showed no PIT for the dopamine-paired cue. Right column: J) Rats in the 50 Hz group (n = 5) showed an increase in food port entry during the pellet-paired stimulus, but not the dopamine-paired stimulus. The increase in locomotor activity during learning was similar for both the dopamine and pellet-paired stimuli. K) During the instrumental training, the 50 Hz group showed strong lever control for both the dopamine stimulation and the pellets. L) In the final PIT test, the dopamine and pellet-paired stimuli both produced robust specific PIT. Error bars =SEM. Credit: Millard et al.

The neurotransmitter dopamine is often associated with pleasure-seeking behavior and makes stimuli in combination with rewards (e.g. food and drink) valuable. Nevertheless, the processes by which this important chemical messenger contributes to learning have not yet been fully elucidated.

Researchers from the University of California at Los Angeles, the University of Sydney and the State University of New Jersey recently conducted a study to better understand how dopaminergic neurons (that is, brain cells that support the production of dopamine) support reward-based learning . Their findings, published in Nature Neurosciencesuggest that these neurons, rather than reflecting the value assigned to different stimuli, contribute to the formation of new mental associations between stimuli and reward (or other neutral stimuli), which help us form cognitive maps of our environment.

“Our recent research has shown that the firing of dopamine neurons acts as the brain’s learning signal,” Melissa Sharpe, co-author of the paper, told Medical Xpress. “This happens when something new or salient happens, which helps us associate events with each other to create a new memory. Crucially, we have shown that dopamine neurons do this without seeing things in themselves as ‘valuable’ or ‘good ‘ to make.”

This work is at odds with previous studies that have defined dopamine as the neurotransmitter that produces “happiness” or “pleasure.” However, if dopaminergic neurons do not convey value signals, they should not be able to attribute positive or pleasant qualities to specific experiences or actions.

“We wondered: if dopamine neurons do not carry a value signal, how do they support intracranial self-stimulation, suggesting that dopamine neurons carry a value signal?” Dr. Sharpe explained. “Our experiments were thus aimed at answering the question: if dopamine neurons indeed have value in the context of intracranial self-stimulation, what is the cognitive representation that makes this possible?” [them] to do that?”

To answer this research question, Dr. Sharpe and her colleagues conducted a series of experiments on rats. During these experiments, they used a Pavlovian-to-instrumental transfer procedure, a well-known experimental test designed to elucidate the cognitive representations that drive animal or human behavior.

“We teach rats that a signal (for example, a tone or click) leads to a certain outcome (for example, dopamine stimulation or a food pellet),” said Dr. Sharpe. “So when the tone or click is played, one of these outcomes occurs (e.g. tone -> dopamine stimulation). We then teach them that they can earn these outcomes by pressing one of two levers. If the tone gets them thinking of the ‘specific’ outcome it was associated with (e.g., dopamine stimulation), they will selectively increase lever pressing associated with the dopamine stimulation (and not the food).”

Dr.’s experiments Sharpe and her colleagues produced several interesting findings. First, the researchers found that a physiological firing rate of dopamine neurons did not support intracranial self-stimulation in a way that would suggest that dopamine neurons carry a value signal.

However, they noted that if they allowed dopaminergic neurons to fire above this physiological rate, the firing of these neurons could function as a sensory-specific target toward which the animals would exhibit behavior. That is, a high frequency of firing in dopamine neurons could function as a reward that would ultimately prompt the rats to exhibit pleasure-seeking behaviors associated with the so-called Pavlovian-to-instrumental transfer effect.

“This suggests that when dopamine neurons fire in everyday life, they don’t make things valuable,” explained Dr. Sharpe out. “Instead, they function to help us form new memories or how things in our environment relate to each other. In a case where dopamine neurons fire more than they should (for example, when taking drugs), This is encoded in the brain as a rewarding event that makes us more likely to seek drugs in the future.”

Overall, this recent study by Dr. Sharpe and her colleagues can greatly contribute to the understanding of dopamine and its role in reward-based (i.e. reinforcement) learning. In particular, their findings suggest that dopamine neurons do not convey value signals that attach pleasure or happiness to stimuli in the environment. In the future, they could pave the way for additional experiments aimed at further validating the team’s findings or exploring the unique contribution of specific dopamine-producing neural circuits.

“Our team is now interested in how different dopamine circuits contribute to different types of learning and how this helps us create a complex but unified representation of our environment,” added Dr. Sharpe added.

More information:
Samuel J. Millard et al., Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency, Nature Neuroscience (2024). DOI: 10.1038/s41593-024-01643-1

© 2024 Science X Network

Quote: Study sheds new light on dopamine’s contribution to reinforcement learning (2024, June 2) retrieved June 2, 2024 from

This document is copyrighted. Except for fair dealing purposes for the purpose of private study or research, no part may be reproduced without written permission. The content is provided for informational purposes only.

See also  Scientists develop a new organoid model to study thymus function
Share This Article
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *