How the Orbitofrontal Cortex Learns How to Learn

Post by Rachel Sharp

The takeaway

The orbitofrontal cortex (OFC) plays a crucial role in the brain's ability to learn and adapt to new situations. By studying mice and computational models, researchers show how the OFC supports meta-reinforcement learning, a process where learning evolves to incorporate better strategies and allow for better decision-making.

What's the science?

Meta-learning, or 'learning to learn', has been a fundamental concept in psychology and artificial intelligence (AI), enabling humans and AI models to quickly acquire new skills based on generalized knowledge from past experiences. In the AI field, this concept has been used to enhance deep learning models, allowing them to refine their learning algorithms across multiple episodes. Similarly, in neuroscience, meta-reinforcement learning (meta-RL) involves the quick neural computation of multiple reinforcement learning (RL) processes at parallel, but different timescales. This week in Nature Neuroscience, Hattori and colleagues unravel how the brain, specifically the orbitofrontal cortex (OFC), manages this complex form of learning. The authors explore whether, and how, synaptic plasticity (a mechanism for adaptive learning at the cellular level) and neural activity-based mechanisms in the OFC collaborate to facilitate meta-RL, using a combination of mouse models and deep RL algorithms.

How did they do it?

The researchers trained mice on a probabilistic reversal learning task, where mice had to choose between two options with varying reward probabilities. This task was designed to mimic the complexities of decision-making in changing environments. Additionally, they focused on the OFC as a key area for mediating meta-RL. To test the role of synaptic plasticity in the OFC, they used paAIP2, a light-inducible inhibitor of CaMKII kinase activity, which blocks synaptic plasticity without affecting pre-existing neural connectivity. Put another way, they injected a virus containing an inhibitor of synaptic plasticity into neurons in the OFC. They were then able to control when this inhibitor was active or not by shining a light at these neurons. When they applied a light to the neurons, paAIP2 would become active, and inhibit the activity of CaMKII.

They then trained deep RL models on the same task, using a meta-RL framework. This meta-RL framework allowed the models to update their strategies across sessions, similar to how the mice learned. The researchers then examined the neural mechanisms behind this learning process, focusing on the OFC in mice, and comparing their findings with the computational models. 

What did they find? 

The study revealed that synaptic plasticity in the OFC is essential for efficient across-session meta-RL in mice. When synaptic plasticity was inhibited in the OFC, mice showed delayed learning and stabilization of reward-based choices. However, once the mice became task experts, blocking OFC plasticity did not affect their performance, indicating that OFC activity (as opposed to plasticity) is required for fast or immediate RL These results when taken together, suggest that CaMKII-mediated plasticity in the OFC is necessary for meta-learning of RL which supports long-term strategy development, while OFC activity supports more immediate decision-making. The findings also demonstrate that both mice and deep RL models improved their decision-making over time, adapting their choices based on reward history, indicating the OFC's vital role in conducting meta-RL. 

What's the impact?

This study sheds light on the dual role of the orbitofrontal cortex (OFC) in managing both slow and fast reinforcement learning (RL). The OFC uses CaMKII-dependent synaptic plasticity for slow, across-session meta-learning, in which generalized knowledge is accumulated and stored over long periods. For immediate, trial-by-trial decision-making, the OFC relies on its neural activity. The study's findings align closely with deep reinforcement learning models, demonstrating a remarkable parallel between artificial and biological learning systems. The insights from this study not only enhance our understanding of the brain's learning mechanisms but also pave the way for more sophisticated AI models.  

Access the original scientific publication here.

Transplanting Microbiota From Alzheimer’s Patients Leads to Changes in Brain Function

Post by Anastasia Sares 

The takeaway

The need to understand Alzheimer’s disease is becoming more urgent. This work establishes a causal role for changes to the gut microbiome in the development of Alzheimer’s.

What's the science?

The microbes living in our intestinal tract can produce compounds that either affect our body directly or are important precursors for the body’s functions. Studies have noted a correlation between Alzheimer’s disease and gut health, but this correlation is not enough to say with confidence that poor gut health actually contributes to Alzheimer’s disease. For that, a real experiment is needed. This week in Brain, Grabrucker, Marizzoni, Silajžić and colleagues transplanted fecal samples from people with and without Alzheimer’s into rats, which led to changes in their brain development and cognitive function.

How did they do it?

In order to show that gut bacterial composition caused changes in brain health, the authors took samples of human fecal matter from older adults with and without Alzheimer’s-type dementia and transplanted it into rats with a depleted microbiome (the depleted microbiome was achieved by giving the rats a cocktail of antibiotics before the fecal transplant). In essence, this procedure replaced a significant amount of the rats’ original gut bacteria with that of the human participants. In this way, rats were randomly subjected to “healthy” or “unhealthy” gut bacteria, and the authors could then measure the effects on the brain.

The rats were examined for changes in brain structure and function. Measures of brain structure included how many new neurons were generated in the hippocampus and the branching patterns of these new neurons. Measures of brain function included the ability to complete a maze and the ability to recognize and explore novel objects.

What did they find?

The human participants with Alzheimer’s disease had signs of inflammation in blood and fecal samples. Their microbiomes were also abnormal, with an increase in bacterial species that are thought to cause inflammation and pathology (Bacteriodetes and Desulfovibrio) and a decrease in species that are thought to produce beneficial compounds (Fimicutes, Verruocomicrobiota, Clostridium sensu stricto 1, and Coprococcus). Several of these microbial differences were observed in the rats who received the fecal transplants as well, along with alterations in the colon for the rats who received transplants from humans with Alzheimer’s (more fecal water content, fewer goblet cells, and a reduction in colon length). Not only that, but the rats with the Alzheimer’s microbiota performed worse on cognitive tests, like distinguishing between new and familiar objects or remembering where to go in a water maze. These rats also had fewer new neurons in their hippocampi at the end of the 50-day period, and these neurons had less complex branching structures.

What's the impact?

This work supports the idea of a causal role of gut health in the development of Alzheimer’s disease, which may lead to interventions that focus on gut health as a protective factor for the disease. This work also highlights how animal research can bring high-value insights, by uncovering new avenues for therapeutic approaches to devastating diseases like Alzheimer’s disease.

Access the original scientific publication here

Does the Gut-Brain Axis Play a Role In the Emergence of Psychiatric Disorders?

Post by Lani Cupo

The takeaway

The gut-brain axis undergoes many changes in adolescence, and these can, in turn, impact the nervous system. It is possible that environmental changes impacting the gut may influence the emergence of psychiatric disorders during adolescence, but more research is required on the topic.

What's the science?

Over the past two decades, the gut-brain axis has increasingly been investigated as important to mental, psychiatric, and neurological health. Most research, however, has focused on the first years of life or old age, with few studies investigating the period of adolescence where both the gut microbiome and brain undergo many changes and psychiatric disorders commonly emerge. This week in Biological Psychiatry, McVey and colleagues sought to synthesize the current knowledge base around the impact of the environment on the gut-brain axis and its role in the emergence of psychiatric disorders during adolescence.

What did they review?

The authors performed a review of the literature on adolescent development and plasticity — or changeability — of the gut microbiome and the brain in adolescence in both human and nonhuman animal studies. The gut-brain axis comprises several component parts investigated in this review. The gut microbiome includes the microorganisms that live in the intestines. The enteric nervous system (ENS) comprises neural and glial cells that directly control the digestive tract (from the esophagus to the anus). While it operates largely independently of the central nervous system, the vagal nerves carry signals to and from the brain to the ENS. Finally, the review examined brain development during adolescence in the context of psychiatric development and atypicality.

What did they find?

First, examining the gut microbiome during adolescence, the authors found that the diversity and stability of microorganisms in the gut were altered, with important sex differences. Specifically, they found that during puberty male gut microbiomes became less diverse, whereas female microbiomes remained stable. There was evidence of bidirectional influence of gut microbiome and sex hormones, meaning the microbiome influences sex hormones and sex hormones influence the microbiome. An altered diet during adolescence is known to impact the gut microbiome as well, drawing attention to the impact of the adolescent environment.

While the authors found there was a lack of studies examining the ENS during adolescence, there was some evidence pointing to changes to the neurons and glia of the ENS during adolescence. For example, in the duodenum, the ratio of neurons to glia was reduced, but in the colon, it was increased. Regarding the vagal nerve, there is some association between changes in vagal tone, measured via heart-rate variability (changes in the interval between heartbeats), and poorer psychological and physical outcomes. Specifically low (worse) vagal tone in adolescence has been associated with later life cardiovascular disease risk, adolescent asthma, emotional dysregulation in adolescents with autism spectrum disorders, and clinical depression.

Adolescence is a period defined by many brain changes, including in the frontal lobes, which continue to develop well after puberty. Two major areas of brain development include synaptic density (synapses are pruned through childhood and adolescence) and myelination (which increases during early life and adolescence). Both of these processes result in greater efficiency of brain networks from childhood to early adolescence. The authors draw attention to the psychiatric disorders that emerge in adolescence, including major depressive disorder, anxiety disorders, eating disorders, and alcohol use disorders, for all of which there is evidence pointing to the association of alterations in the gut microbiome.

The authors also summarize evidence from nonhuman animal studies linking alterations in communication between the microbiome, gut, and brain and clinical dysfunction. The work they highlight indicates the role of the gut microbiome in modulating the HPA axis activity in response to acute stress, as well as the possibility of moderating the gut microbiome, gene expression in the brain, and behaviors with diet.

What's the impact?

The results of this review first draw attention to the potential role the gut microbiome may play in moderating the emergence of psychiatric disorders in adolescence. Importantly, however, this review identifies a lack of studies that comprehensively investigate this topic, and therefore more research is needed to understand how the gut microbiome influences the development of psychiatric disorders.

Access the original scientific publication here