The Missing Link: How to Combine Neuroimaging Data with Computational Models of Behavior

This course will take place on 9 June 2019 from 13:00 to 16:30 in the OHBM meeting.

Cognitive computational models proved useful (and in some cases even groundbreaking) for the scientific community in a wide range of research areas, such as decision making, learning and memory. Modelling behaviour offers a way to see beyond raw behavioral measures, such as accuracies and reaction times, and estimate latent cognitive parameters which drive behaviour. These might include how fast information is sampled and integrated (evidence accumulation), how much we adjust our expectations based on new information (learning rates; prediction errors) or how much information do we need to make a choice (thresholds).

The key challenge of the field lies in integrating cognitive models of behaviour with the rich ensemble of neural data. Neuroimaging technologies offer a plethora of ways to sample neural data, each having its benefits and drawbacks. Though location and timing of the neural activity could be determined with neuroimaging tools, it is still not clear how neural mechanisms lead to changes in behavior. In this workshop, we aim to bridge these two well-developed and self-proved areas.

If you are a researcher working with a variety of neuroimaging methods, such as fMRI, EEG, MEG, DWI, TMS, and others, and incorporate behavioral measures (e.g. reaction time, accuracy, ratings) in your experiments, this course could be beneficiary for you.

We will be collecting questions during the course on If you have any question for the presenters, please use the code #OHBM on

Course material will be available here soon.


"Defining the missing link: introduction to combining neural data with cognitive models of behavior." by Wojciech Zajkowski

In recent years, the amount of neuroimaging data we collect has been growing in an exponential fashion. Due to this, the issue of developing methods that aid in drawing meaningful conclusions from large and rich (often multimodal) datasets has gained paramount importance. This talk will consist of three segments. I will start by providing an overview of the purpose of using cognitive models of behavior and how can they be utilized together with brain data, using models such as Drift-Diffusion Model or Linear Ballistic Accumulator as examples. I will then explain the nature of the linking problem. Using the Marr’s levels of analysis framework, I will go through the rationale behind why is it crucial for our understanding of brain and behavior. Finally, I will introduce different methods of dealing with the problem: from a simple correlation to recently developed frameworks, such as joint modelling, which utilize the covariability of neural and behavioural data to draw meaningful conclusions.

"Model-based cognitive neuroscience" by Birte Forstmann

Cognitive neuroscientists study how the brain implements particular cognitive processes such as perception, learning, and decision-making. Traditional approaches in which experiments are designed to target a specific cognitive process have been supplemented by two recent innovations. First, formal cognitive models can decompose observed behavioral data into multiple latent cognitive processes, allowing brain measurements to be associated with a particular cognitive process more precisely and more confidently. Second, cognitive neuroscience can provide additional data to inform the development of formal cognitive models, providing greater constraint than behavioral data alone. We argue that these fields are mutually dependent; not only can models guide neuroscientific endeavors, but understanding neural mechanisms can provide key insights into formal models of cognition.

"Reinforcement learning models and integration with fMRI" by Jian Li

Reinforcement learning (RL) has witnessed its wide application in cognitive sciences in the past decades. In this session I will briefly introduce different RL models used in cognitive neuroscience/psychology research and how they can be integrated with fMRI techniques to better understand the computations that brain carries out during learning and inference.

"Model-based assessment of brain-behavior relationships" by Jean Daunizeau

Functional outcomes (e.g., subjective percepts, emotions, memory retrievals, decisions, etc...) are partly determined by external stimuli and/or cues. But they may also be strongly influenced by (trial-by-trial) uncontrolled variations in brain responses to incoming information. In turn, this variability may provide critical information regarding how behaviourally relevant inputs are eventually transformed into functional outcomes. Assessing brain-behavior relationships thus requires considering the (possibly nonlinear and stochastic) impact of biological constraints of input-output transformations in the brain. In this talk, I will review the portfolio of existing approaches to decomposing the brain's transformation of stimuli into behavioural outcomes, in terms of the relative contribution of brain regions and their connections. In particular, I will highlight three novel techniques, namely: mass mediation analysis, artificial neural network modelling, and behavioral DCM (dynamic causal modelling). The aim here is for attendees to understand the strengths and weaknesses of each approach in turn, as well as gain practical know-how regarding how to perform such analyses.

"A tutorial on joint models of neural and behavioral data" by Brandon Turner

A growing synergy between the fields of cognitive neuroscience and mathematical psychology has sparked the development of several unique statistical approaches exploiting the benefits of both disciplines (Turner, Forstmann et al., 2017). One approach in particular, called joint modeling, attempts to model the covariation between the parameters of ‘‘submodels’’ intended to capture important patterns in each stream of data. Joint models present an interesting opportunity to transcend conventional levels of analyses (e.g., Marr’s hierarchy; Marr, 1982) by providing fully integrative models (Love, 2015). In this talk, we provide a tutorial of two flavors of joint models — the Directed and Covariance approaches. Computational procedures have been developed to apply these approaches to a number of cognitive tasks, yet neither have been made accessible to a wider audience. Here, we provide a step-by-step walkthrough on how to develop submodels of each stream of data, as well as how to link the important model parameters to form one cohesive model. For convenience, we provide code that uses the Just Another Gibbs Sampler (Plummer, 2003) software to perform estimation of the model parameters.

Panel discussion </summary>

Join the event on [] with the code **#OHBM** and ask your questions!

</details> ### Organizers [Lorena Santamaria](, [Esin Karahan](, [Wojciech Zajkowski](, [Jiaxiang Zhang](