Predicting Preference for Art Through Low- and High-Level Features

Post by Leanna Kalinowski

What's the science?

We are surrounded by visual art, from classic paintings in a museum to photographs on social media. While navigating through this art-filled world, we constantly make judgements about whether we like or dislike a particular piece. However, the process by which we perceive art is unclear. Do prior experiences with certain features of the piece of art shape our preferences, or are the visual properties of an image more important? The answer is that both are likely important. Computational methods have previously been applied to tease apart how we develop different preferences. However, in the case of visual art, this process is much more challenging due to the visual complexity and variation of some art. This week in Nature Human Behavior, Iigaya and colleagues developed and tested a computational framework to investigate how preferences for visual art are formed.

How did they do it?

The authors first divided the properties of an image into two categories: ‘low-level’ and ‘high-level’. ‘Low-level’ (i.e., bottom-up) features included those derived from an image’s statistics and visual properties, such as hue and brightness, while ‘high-level’ (i.e., top-down) features included those that require human judgement, such as realism and emotion. Participants were asked to report how much they liked various paintings and photographs on a four-point scale, and the authors used these ratings to determine the extent to which they could predict art preferences. They also applied machine learning: a deep convolutional neural network (DCNN) that had been trained for object recognition to predict the pattern by which these visual features emerge when the brain processes visual images.

What did they find?

By engineering a linear feature summation (LFS) model, the authors first observed that visual preference for art can be predicted through a combination of low- and high-level features. This model predicted preferences for both paintings and photographs, suggesting that the features used for driving visual preferences may be universal across different mediums. They also found that their model may represent a biologically plausible computation, as their DCNN model mirrored the results from the LFS model above. Specifically, when the authors did not specify certain features for the DCNN as they did with the LFS model, they found that the DCNN model could learn to predict all of those features on its own.

What's the impact?

The findings here uncover a mechanism through which art preferences can be predicted, shedding light on how these preferences are formed in the brain. These tools have the potential to influence the arts and media industry by predicting which works of art may be more likely to be preferred, and could be extended to predict judgements and perceptions beyond art.   

Iigaya et al. Aesthetic preference for art can be predicted from a mixture of how- and high-level visual features. Nature Human Behaviour (2021) Access the original scientific publication here.

How Much Effort Does it Take to Just Listen?

Post by Anastasia Sares

What is listening effort?

We don’t often think about how much of our mental space we reserve for listening to speech—for many people, it feels effortless. However, there are often obstacles: background noises, distracting conversations, or age-related hearing loss, to name a few. According to some models, we have a limited amount of mental resources, and the more we spend trying to decipher speech, the less we have left over for critical thinking, memory, and other high-level processing. However, a nebulous concept such as “effort” isn’t easy to quantify, and scientists have tried a number of approaches, from self-report questionnaires to full-sized brain scanners. Here’s a run-down of all the techniques used to measure listening effort.

Using self-report measures

One way to measure effort on a task is to ask people about it directly. This is the simplest method, but it can get tricky because people may have different interpretations of what “effort” means. To be more precise, some recommend breaking down effort into sub-components, like mental effort, physical effort, time pressure, or frustration. The NASA task load index is one such breakdown. However, a recent study suggests that we should ask about tiredness, a question that is not present in the NASA task load index. In that study, people’s tiredness ratings during speech listening were shown to correlate with the next method of measuring effort: pupil size.

Using the size of the pupil (pupillometry)

Our pupils dilate under states of mental arousal or effort, and specifically when listening conditions are worsened. Using video recordings of the eye or special glasses with infra-red cameras, we can measure the size of a person’s pupil as they hear and respond to sounds—this is called pupillometry. It is less subjective than self-report, and we can evaluate the effort someone expends on a task without forcing them to stop and reflect. However, pupillometry currently needs specific lighting conditions, and sometimes the pupil response can plateau in complex tasks. So, there are still some challenges to using this method.

Using brain activity (EEG, fMRI, fNIRS)

Electroencephalography, or EEG, measures electrical activity in the brain and can be used as another way to tap into listening effort. Among other EEG measures, the N100 response to sound is one index of this effort. This automatic response happens 100 milliseconds after the onset of a sound, and it becomes bigger when the speech is made less intelligible. Another EEG measure of effort is alpha power. If we take the activity in the alpha range and sum up its power over the course of an experiment, we can see when more effort is being expended.

Blood flow to different brain areas has long been used as a proxy for brain activity in those areas. In particular, blood flow to the left inferior frontal region of the brain (close to the temples) and the superior temporal gyrus (just above the ears) can give us a hint about how much effort is being exerted. This can be done in a magnetic scanner that detects the magnetic properties of blood (fMRI) or using a cap with small infra-red lights pointed at the scalp (fNIRS). These blood flow methods are a little slower than EEG, but fMRI, in particular, can pinpoint the location of activity in the brain with better accuracy, and fNIRS is advantageous because it doesn’t interfere with hearing aids or other devices.

What’s the bottom line?

Hearing is a crucial aspect of health: hearing loss has a large societal burden and may contribute to the risk of dementia later in life. Armed with multiple tools to measure listening effort, we can study how it varies in different conditions and populations, and better understand the link between hearing and cognition.

References

Pichora-Fuller, M. K. et al. Hearing Impairment and Cognitive Energy. Ear Hear. 37, 5S-27S (2016).

Hart, S. G. & Staveland, L. E. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Adv. Psychol. 52, 139–183 (1988).

McGarrigle, R., Rakusen, L. & Mattys, S. Effortful listening under the microscope: Examining relations between pupillometric and subjective markers of effort and tiredness from listening. Psychophysiology 58, 1–22 (2021).

Zekveld, A. A., Kramer, S. E. & Festen, J. M. Pupil response as an indication of effortful listening: The influence of sentence intelligibility. Ear Hear. 31, 480–490 (2010).

Koo, M. et al. Effects of noise and serial position on free recall of spoken words and pupil dilation during encoding in normal-hearing adults. Brain Sci. 11, 1–14 (2021).

Zhang, Y., Lehmann, A. & Deroche, M. Disentangling listening effort and memory load beyond behavioural evidence: Pupillary response to listening effort during a concurrent memory task. (2020). doi:10.1101/2020.05.04.076588

Obleser, J. & Kotz, S. A. Multiple brain signatures of integration in the comprehension of degraded speech. Neuroimage 55, 713–723 (2011).

Obleser, J., Wöstmann, M., Hellbernd, N., Wilsch, A. & Maess, B. Adverse listening conditions and memory load drive a common alpha oscillatory network. J. Neurosci. 32, 12376–12383 (2012).

Wild, C. J. et al. Effortful listening: The processing of degraded speech depends critically on attention. J. Neurosci. 32, 14010–14021 (2012).

Kousaie, S. et al. Language learning experience and mastering the challenges of perceiving speech in noise. Brain Lang. 196, 104645 (2019).

Zekveld, A. A., Heslenfeld, D. J., Johnsrude, I. S., Versfeld, N. J. & Kramer, S. E. The eye as a window to the listening brain: Neural correlates of pupil size as a measure of cognitive listening load. Neuroimage 101, 76–86 (2014).

Rovetti, J., Goy, H., Pichora-Fuller, M. K. & Russo, F. A. Functional Near-Infrared Spectroscopy as a Measure of Listening Effort in Older Adults Who Use Hearing Aids. Trends Hear. 23, 233121651988672 (2019).

Ford, A. H. et al. Hearing loss and the risk of dementia in later life. Maturitas. 112, 1–11 (2018).

Neural Mechanisms Involved in the Extinction of Long-Term Trauma

Post by Lina Teichmann

What's the science?

Traumatic experiences often result in enduring memories of fear. Exposure therapy is a common treatment to overcome trauma by exposing patients to the context of fear-inducing memories in a safe environment. However, it is known that exposure therapy is less successful if the traumatic experience occurred a long time ago. This week in Nature Neuroscience, Silva and colleagues used a fear extinction paradigm with mice to test how neural mechanisms involved in overcoming fear differ depending on the age of the traumatic memory.

How did they do it?

To mimic the trauma, the authors placed mice in a conditioning chamber, where they received electric shocks to their paws. After either 1 day or 30 days, the mice were re-exposed to the same chamber without receiving shocks (recall) with subsequent re-exposing events over several days (fear extinction). When the animals were re-exposed to the traumatic context, the fear response was behaviorally quantified by examining freezing responses (prolonged absences of motion). Viral tracing, neuronal activity mapping, fiber photometry, and chemo- and optogenetics were used to examine the effect of long-term fear extinction on neural circuitry. In particular, the authors examined the functional responses to long- and short-term fear extinction in infralimbic cortex to basolateral amygdala and thalamic nucleus reuniens to basolateral amygdala pathways.

What did they find?

To overcome trauma, fear-evoking contexts have to be newly associated with safety. The results show that the neural mechanisms underlying this type of fear extinction depend on whether the fear-evoking experience occurred recently or a long time ago. While direct connections from the infralimbic cortex to the basolateral amygdala are critical for recent fear extinction, long-term fear extinction requires the recruitment of an additional pathway. In particular, when overcoming long-term fear, fear-related information is sent upstream from the infralimbic cortex via the thalamic nucleus reuniens to the basolateral amygdala. The behavioral expression of fear – freezing – was modulated by the activity in the thalamic nucleus reuniens. The activity in the nucleus reuniens peaked just before the freezing response ended and the freezing length could be manipulated by artificially increasing or decreasing activity in this area. This finding suggests that activity in the thalamic nucleus reuniens plays a role in learning to associate safety with a context that initially evoked fear.

What's the impact?

Traumatic memories are often long-lasting and can lead to mental disorders such as post-traumatic stress disorder. Silva and colleagues show that the time that has passed since a fear-evoking event modulates neural mechanisms involved in overcoming trauma. These findings improve our understanding of long-lasting traumatic memories and set the stage for future research into how we can weaken traumatic associations.   

Silva et al. A thalamo-amygdalar circuit underlying the extinction of remote fear memories. Nature Neuroscience (2021).Access the original scientific publication here.