Hedging your bets by learning reward correlations in the human brain
by Klaus Wunderlich
Risk minimization is a key concept in insurance and financial markets. A common risk management strategy in these contexts is hedging, the process of combining multiple positions in different assets to reduce the total risk in a portfolio. However, these choices involving multiple interdependent factors entail rather complex computational processes.
How we make sense of these factors has until now been unclear. We show here that our brains do this naturally by learning the correlation between events. This intricate capability has a defined benefit for our choices by allowing us to observe the outcome of just one action and then infer the outcomes of other actions without having to sample them individually. Our study also identifies the regions of the brain involved in tracking this correlation, which include the insula and the anterior cingulate cortex.
To get to the heart of the question we scanned the brains of 16 subjects using functional magnetic resonance imaging (fMRI), which measures activity in the brain, while the subjects played a game of resource management. The task was similar to a simple portfolio problem in finance: subjects were instructed to adjust the proportion of energy coming from two renewable power sources, a solar plant and a wind farm, in an effort to create the most stable energy output possible. Importantly, the probabilistic outcomes of the two resources co-varied with each other. For example, when they were positively correlated, the wind blew while the sun was shining and each source generated power. We changed the correlation between the two sources throughout the experiment, thus requiring subjects to continuously revise their predictions of how correlated the outcomes were in order to perform well.
We found that the subjects changed their behaviours to reflect new correlations far better than they could have had they been relying on simple trial and error. Instead, they were estimating the correlation between the sources, tracking mistakes in their estimations, and adjusting their estimates of the correlation on the fly. The neuroimaging measurements provide insight into how the brain might perform these computations, highlighting neural activity in two parts of the brain, both of which have previously been associated with decision-making and awareness. Insular cortex represented information about how much the two resources were correlated with each other and the anterior cingulate cortex tracked how accurate the prediction was in order to update this information each time new information becomes available.
We believe there is an evolutionary importance to being sensitive to correlations in the environment. Risks are ubiquitous in nature with predation, starvation, or adverse environmental change acting as constant background variables that shape our behaviour. Imagine our ancestors foraging for food in the woods. They could spend their time either collecting berries or hunting deer. Now imagine they have previously observed that deer eat berries. So, as they are foraging, if they notice a lack of fresh berries, they can infer that there are lots of deer around and instead focus on hunting.
The majority of the research into learning and decision-making over the past decade has focused on direct relationships between actions and rewards. The scientific understanding was hitherto that we separately learn about each of a number of action-reward associations and use that information to make choices.
In the above example, this would mean that the hunter could only predict the success of hunting deer after having tried it and observed the actual outcome. In an ever-changing natural environment it is likely the case that the key correlations are more stable than the relationship between individual actions and reward. In other words, the fact that deer eat berries is always true, but the success at hunting deer can vary from year to year. Learning about the correlations therefore has immediate benefits for efficient choices.
In the financial world, when investing in more than one asset, it is important that one does so with the right mix, which is determined by the correlation between the returns on the assets. In contrast, we are often presented with rough rules of thumb depending on our risk aversion – the more risk averse, the more bonds we should invest in. In fact, from the point of view of financial returns, the mix between stocks and bonds should not depend on risk aversion at all, but on how correlated the two are. And, as we show in this research, we do in fact naturally combine sources in an optimal way, taking into account explicitly how the outputs are correlated.
Importantly, subjects learned those correlations in our experiment through continuous outcome observations. An evolved mechanism in the brain might in fact work best if information is provided in such a sequence of observations, rather than in form of summary statistics from tables or charts (in which case humans often utterly fail at making financial decisions that are based on understanding correlations). These findings may from a basis to promote better decision-making strategies in everyday life and complex financial situations, whereby decision makers are encouraged to observe information sequentially in order to best utilize their inherent neural problem solving mechanism.
Read the paper in full