Interpretable artificial neural networks for fMRI data classification

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
School of Science | Doctoral thesis (article-based) | Defence date: 2023-12-21
Degree programme
64 + app. 60
Aalto University publication series DOCTORAL THESES, 209/2023
Functional magnetic resonance imaging (fMRI) technology allows non-invasive measurement of neuronal activity in the human brain with a combination of reasonable temporal and fine spatial resolution. Recently, multivariate methods have attracted attention in fMRI data analysis to study task-related activation patterns. Concurrent research in the field of machine learning has led to the establishment of inherently multivariate computational graphs that facilitate efficient, robust and interpretable classification of fMRI data. Here we studied methods for classification of fMRI data based on neural networks. In particular, we focused on techniques that assess the contribution of different brain regions to the classification result, referred to as "importance maps" and proposed novel neuroscientifically motivated architectures. In the first study, we successfully classified basic emotions from fMRI data, elicited by short movies and mental imagery, generating whole brain importance maps indicating the contribution of individual voxels to the classification result. The second study provided a comparison of importance extraction methods and their reproducibility, applied to both simulated and real data sets, revealing patterns that do not convey significant univariate information. The third study examined the effect of distractors in visual imagery using classification methods and importance map extraction, identifying robust activation patterns related to shape imagery and a visual distractor in object-selective lateral extrastriate cortex at the junction of left occipital, temporal and parietal lobes. The fourth study examined the use of anatomically driven topologies based on spatial information. In particular, the addition of layers motivated by voxel proximity and brain atlases to the model, led to an increase in the classification accuracy and produced smoother and more interpretable importance maps. The purpose of this thesis is to showcase machine learning techniques specifically designed for analyzing neuroscience data. This work aims to motivate further research towards the use of machine learning as a means to gain a better understanding of the human brain.
Supervising professor
Lampinen, Jouko, Prof., Aalto University, Department of Computer Science, Finland
Thesis advisor
Sams, Mikko, Prof., Aalto University, Department of Neuroscience and Biomedical Engineering, Finland
brain, fMRI, machine learning, classification, neural networks, importance maps
Other note
  • [Publication 1]: Saarimäki, H., Gotsopoulos, A., Jääskeläinen, I. P., Lampinen, J., Vuilleumier, P., Hari, R., Sams, M., & Nummenmaa, L. (2015). Discrete Neural Signatures of Basic Emotions. Cerebral Cortex, 1–11.
    DOI: 10.1093/cercor/bhv086 View at publisher
  • [Publication 2]: Gotsopoulos, A., Saarimäki, H., Glerean, E., Jääskeläinen, I. P., Sams, M., Nummenmaa, L., & Lampinen, J. (2018). Reproducibility of importance extraction methods in neural network based fMRI classification. NeuroImage, 181, 44–54.
    DOI: 10.1016/j.neuroimage.2018.06.076 View at publisher
  • [Publication 3]: Alho, J., Gotsopoulos, A., & Silvanto, J. (2023). Where in the brain do internally generated and externally presented visual information interact? Brain Research, 148582.
    DOI: 10.1016/j.brainres.2023.148582 View at publisher
  • [Publication 4]: Gotsopoulos, A., Sams, M., & Lampinen, J. Adding anatomical information increases interpretability and performance of neural-network-based fMRI classification. (submitted to NeuroImage: Reports)