Coding Subjective Value in Preference-Based Decisions

Post by Elisa Guma

What's the science?

In our daily lives, we are constantly processing information in order to make decisions. Due to capacity limitations, our brain has had to adopt various strategies for information processing. One such strategy is known as efficient coding: the more we are exposed to a certain environmental stimulus, the more precise our neural representation of the stimulus is. Another model of how we process information is Bayesian decoding, which suggests that for optimal processing we combine our representation of environmental stimuli with our prior expectations. Recent work suggests that the brain can adapt to using different strategies based on the environment, however, it is currently unknown whether efficient coding and Bayesian decoding principles are used jointly to generate subjective value preferences, and whether these can explain variability, biases, and confidence in value-based decisions made by humans. This week in Nature Neuroscience, Polania and colleagues aimed to show that choice variability, biases, and confidence in human preference-based decisions can be explained by single value-inference process using modeling and behavioural experiments.

How did they do it?

The authors recruited healthy young volunteers (n=127) to participate in one of four different behavioral experiments in which they were tasked with assigning preference-based value to different food images. Experiments 1 and 2 each consisted of two different rating phases, followed by a decision-making task. In the first rating phase, participants were shown images of 64 food items and asked to provide a rating (on a continuous sliding scale) based on how much they would like to eat that food. During this phase, participants were unaware that there would be a second session, in order to prevent them from trying to memorize their ratings. Items were randomly presented for the second rating phase, in which participants followed the same instructions as the first. Experiment 2 was the same as Experiment 1, except that participants used a 20-point scale for ratings instead of a sliding scale to control for the possibility of anchoring biases. Immediately after the two rating sessions for Experiments 1 and 2, participants were put through a choice task, in which they had to make a preference-based choice between two food items (shown during the rating phases) whose ratings differed by either ~5%, ~10%, ~15%, or ~20%. In Experiment 3, participants were presented the same food items as in Experiment 1, except that half the images were randomly selected to be shown for 900ms, and the other half for 2,600ms to test whether longer exposure would improve accuracy of the encoding. Finally, Experiment 4 was the same was as Experiment 1, except participants also had to provide a confidence rating for each food preference rating. The authors used various modeling techniques to determine how the presentation of an object with a true stimulus value (the ground truth) elicits an internal noisy response (encoding) that is then used by the observer to generate a subjective value estimate (decoding). They used hierarchical logistic mixed-effects regression as well as Bayesian modeling to decipher how the preference-rating variability and value difference affected choice. Predictive accuracy of their models was tested using leave-one-out cross-validation (a machine learning technique).

What did they find?

In Experiment 1, the authors observed that in the choice phase, the greater the difference in value participants assigned to the two food items (during the initial two rating phases), the more consistent the choice between the rated items. Further, the greater the variability in the rating given to the item, the less consistent the choice. In Experiment 2, they observed a similar impact on choice behaviour as in Experiment 1, confirming that the rating procedure (continuous vs. fixed scale) didn’t influence rating variability. In Experiment 3, they found that a longer exposure reduced ‘internal noise’ of individuals’ subjective preferences, decreasing rating variability (these findings were also confirmed theoretically using a mathematical proof). Finally, in Experiment 4, the authors showed that the confidence in an individual's’ rating relates to the rating variability (independent from the actual rating). Qualitative predictions and leave-one-out cross-validation suggested that human preference-based decisions are inferred and employed using both efficient coding and Bayesian decoding, but that the efficient coding model predicted the data best.

elisa_week50 (1).png

What's the impact?

This study found that variability, biases, and confidence in preference-based choices can all be explained by one specific encoding-decoding strategy optimally suited for a limited-capacity system. This work provides a new framework for modeling and predicting preference-based decisions and furthers our understanding of how humans perceive and evaluate their environment in order to guide behaviour. This work can be expanded to other fields such as psychology and economics in order to further understand decision-making strategies accounting for environmental and biological constraints.

ruff_quote.png

Polania et al. Efficient coding of subjective value. Nature Neuroscience (2018). Access the original scientific publication here.

Reward Processing in the Brain, Reduced Sleep and Food Valuation

Post by Shireen Parimoo

What's the science?

Sleep deprivation is linked to weight gain, and this is largely attributed to hormones like ghrelin and leptin, which are involved in promoting and inhibiting hunger, respectively. Other studies suggest that this link is due to altered activity in brain regions associated with food reward processing following sleep deprivation, like the hypothalamus. It is currently unclear whether endocrine factors or reward-processing regions of the brain have a greater impact on food valuation following sleep deprivation, or whether they interact to affect food valuation. This week in the Journal of Neuroscience, Rihm and colleagues explored the role of endocrine function and reward processing in the brain to understand the relationship between sleep deprivation and food valuation.

How did they do it?

Thirty-two healthy young male participants completed the Becker-deGroot-Marschak (BDM) auction task in two sessions; one following habitual sleep and one following a night of sleep deprivation. In the sleep deprivation condition, participants remained awake all night and did not eat anything. The BDM task consists of three phases and provides a measure of participants’ valuation of food and non-food items. In the first bidding phase, participants were shown food and non-food items and asked how much they would spend on the items (subjective value), with a maximum limit of 3 €. For each participant, the median price across all items was used as a reference price for the choice phase. In the choice phase, participants were presented with the same items while they underwent functional magnetic resonance imaging (fMRI) scanning and asked if they would purchase each item for the reference price. Finally, in the post-scan auction, one food and one non-food item were selected at random from the first and the second phase, and the participants’ bid competed against a computer-generated price to determine if they won the item.

The authors assessed participants’ willingness to pay for the items and the probability that they would buy a given item during the choice phase of the BDM task in each session. They also collected blood samples in order to measure ghrelin (total ghrelin, acyl ghrelin and des-acyl ghrelin), leptin, cortisol, insulin, and glucose. Finally, they examined changes in the activation and functional connectivity of brain regions involved in food and reward processing across the habitual sleep and sleep deprivation conditions.

What did they find?

Sleep deprivation influenced participants’ valuation of food rewards. Participants were more willing to pay for food following sleep deprivation than after a night of sleep, but there was no difference in their valuation of non-food items. All participants in both conditions combined were more likely to buy food items than non-food items. The change in subjective value of items across the two sessions was positively correlated with the change in the probability that the participant would buy the food items. That is, if participants indicated that they would spend more money on food items in the bidding phase following sleep deprivation, they were also more likely to purchase food items during the corresponding choice phase. The concentration of des-acyl ghrelin was higher following sleep deprivation, but this did not correlate with food valuation. Brain activity on the other hand, was correlated with food valuation. Following sleep deprivation, there was greater activation of the hypothalamus for food rewards with higher subjective value. Similarly, the right amygdala was more active for food items than for non-food items after sleep deprivation. Functional connectivity analyses revealed that during the choice phase of the BDM task, activity in the right amygdala was coupled with activity in the bilateral hypothalami and the left orbitofrontal cortex, but activity in the hypothalamus was not coupled with other brain regions. This indicates that changes in food valuation following sleep deprivation are associated with activation of brain regions involved in food reward processing but not hormone concentrations, and that the altered neural activity was not driven by changes in endocrine function.

shireen_week50 (1).png

What's the impact?

This study is the first to show that reward processing in the brain, rather than endocrine function, might underlie the link between food valuation and sleep deprivation. Food valuation was correlated with the activation and functional connectivity of the hypothalamus and the amygdala following sleep deprivation, but not with changes in hormone concentrations. This work provides greater insight into the relationship between sleep deprivation and weight gain and has important implications for mitigating the adverse effects of reduced sleep on decision-making, particularly as it relates to the risk of obesity.

Rihm et al. Sleep deprivation selectively up-regulates an amygdala-hypothalamic circuit involved in food reward. The Journal of Neuroscience (2018). Access the original scientific publication here.

Neurofeedback Training for Stress Resilience

Post by Stephanie Williams

What's the science?

Adaptive coping strategies in stressful situations have been linked to the downregulation of activity in the amygdala. Several studies have confirmed that amygdala-targeted neurofeedback can improve emotion regulation and decrease depressive symptoms. Neurofeedback is when information about an individual’s brain activity is reported to that individual in near real time, with the goal of having the person learn to alter or control their own brain activity. Despite evidence in favor of targeted neurofeedback as a method for improving strategies to cope with stress, neurofeedback using functional magnetic resonance imaging (fMRI) is not widespread due to high cost and lack of accessibility of the MRI scanner. This week in Nature Human Behavior, Jakob Keynan from Talma Hendler's lab at Tel Aviv University and Medical Center and colleagues administered and demonstrated the efficacy of a highly scalable fMRI-informed electroencephalography (EEG) neurofeedback short training protocol (coined ‘Amygdala-Electrical FingerPrint-NF’ or ‘amyg-EFP-NF’) for individuals undergoing stressful military training.

How did they do it?

The authors administered a neurofeedback training program to a large group of young men who were beginning combat training. Participants were randomly assigned (administration was double-blinded) to one of three groups: 1) neurofeedback based on an amygdala specific signal (amyg-EFP-NF), 2) control neurofeedback based on the alpha/theta ratio signal (control-NF), or 3) no neurofeedback. The authors assessed several emotion-related traits in participants (e.g. state anxiety levels, cognitive processing and emotional expression (alexithymia), which is heightened following traumatic stressors). The authors used an emotional Stroop task (eStroop) to assess emotion regulation. Participants saw a series of faces express either a fearful or happy emotion paired with a word (eg.’happy‘ or ‘fearful’). The participant’s goal was to ignore the words and correctly report the emotion they observed on the face.

Participants in the two neurofeedback groups completed six neurofeedback training sessions over four weeks while undergoing demanding military combative training on their base. Sessions involved a feedback interface: a 3D animated scenario in which avatar figures in a hospital waiting room were either agitated, speaking in raised voices, or were resting in a chair quietly. Participants first passively watched a noisy scenario with agitated avatars (the “attend” section), and then actively tried to relax the scene (the “regulate” section). Participants were instructed to find a mental way to make the figures in the animation sit down and lower their voice.

The authors used an Amyg-EFP model that had been previously developed using simultaneous EEG and fMRI recordings to predict activity in the amygdala from EEG signals. During neurofeedback trials, the authors calculated either the participant’s alpha/theta power (control-NF group) or the participant’s Amyg-EFP online during the task (amyg-EFP-NF group), and used the results of the calculation to modify the audio-visual scenario in near real time. Participants in the control-NF group learned to down-regulate their alpha/theta ratio. Participants in the Amyg-EFP group learned to down-regulate activity that was localized to the amygdala.

stephanie_dec18 (1).png

The authors also added an additional interference component into the task where participants had to continue to down-regulate the targeted brain activity they had learned to modify, during a cognitively demanding task. The authors also tested the participant’s ability to regulate the target brain activity (alpha/theta or amyg-EFP) without feedback. They compared the Amyg-EFP signal in both neurofeedback groups during the no feedback trials to investigate whether participants who had trained to control the Amyg-EFP signal could still willfully control Amyg-EFP without feedback. The authors also performed a one-month fMRI follow-up on participants from theAmyg-EFP-NF and no-NF groups to investigate whether participants could down-regulate amygdala BOLD activity (i.e. signal detected using fMRI) during a new neurofeedback task. This task was selected to ensure that results were transferable to different contexts.  

What did they find?

The authors found that the participants who received the amyg-NF training showed lower alexithymia scores and better performance on the eStroop tasks than participants who received control-NF training, which suggests the Amyg-EFP-NF selectively modified participant’s emotion regulation. Participants in the Amyg-EFP-NF group showed significant improvement in down-regulating the Amyg-EFP signal by the fourth session and the improvement was correlated with lower alexithymia scores following NF training. As the authors hypothesized, participants in the control neurofeedback group did not show a similar down-regulation of the Amyg-EFP signal, but did show significant down-regulation of the alpha/theta ratio by the fifth session. Interestingly, each participant’s performance continued to improve with each neurofeedback session (no plateau in performance), with most participants hitting peak performance during the last session

Results from the no-feedback trials showed that the down-regulating skills participants had learned in previous sessions could be sustained in the absence of feedback. Results from the cognitive interference task showed that participants could willfully control the target signal even while carrying out a simultaneous cognitive task. The results of the one-month follow-up showed that participants from the Amyg-EFP-NF group were able to better down regulate amygdala BOLD activity than participants from the no-NF group.  Participants from the Amyg-EFP-NF group also exhibited higher amygdala to ventromedial prefrontal cortex functional connectivity during the regulation period than participants from the no-NF group.

What's the impact?

Previously, the cost and mobility of fMRI neurofeedback limited the availability of neurofeedback training programs. Kenyan and colleagues successfully implemented a highly mobile and cost-effective (EEG) neurofeedback paradigm that improved emotion regulation in individuals experiencing chronic stress. The authors’ results confirm the beneficial impact of neurofeedback on emotion regulation, and show that individuals can successfully learn to volitionally regulate limbic activity and enhance their stress resilience.

feedback_quote_new+%281%29.jpg

Kenyan et al. Electrical Fingerprint of the Amygdala Guides Neurofeedback Training for Stress Resilience. Nature Human Behavior (2018). Access the original scientific publication here.