Using deep learning to predict continuous hand kinematics from magnetoencephalographic (MEG) measurements of electromagnetic brain activity.

Abstract

Brain imaging techniques aim to study and discover hidden patterns from the brain activity that can lead to a better understanding of brain dynamics as well as to better clinical diagnoses. However, the brain is a complex system. Therefore, the encoded information is tedious to extract and analyze. In recent years, deep learning has outperformed state-of-the-art statistical techniques in different fields, such as computer vision and speech recognition. The reason for this is mainly its capability to extract complex patterns throughout an automatic end-to-end learning process. Thus, the main goal of the thesis was to investigate the potential and limitations of deep learning (DL) techniques to decode continuous hand-kinematics parameters from electromagnetic brain activity measured with magnetoencephalography (MEG).

The primary thesis experiment consisted of decoding circular hand-movement captured using accelerometers placed on the back of both subject hands. Specifically, the principal analysis was a within-subject experiment. The primary baseline approach used as a comparison to the DL proposed solutions is a state-of-the-art algorithm called Source Power Comodulation (SPoC), previously used to model regression tasks. It performs spatial filtering of the MEG data estimating the source space that maximally correlates with the continuous target value, and, eventually, uses the estimated source space to predict the target.

The two proposed models are Convolutional Neural Network (CNN) architectures that aim to extract meaningful features from the measurement by applying specific transformations to the input data. The first proposed model (MNet) aims firstly to extract global features convolving simultaneously in the spatial and temporal domain of the recording. Secondly, it aims to extract local features. The second one is a Spatial CNN (SCNN) that aims to separately extract temporal and spatial features and, eventually, combine them to predict the final output. Moreover, to augment the input data to boost the performances, the Relative Power Spectrum (RPS) of some specific bands was integrated into the data inpu t.

The main results have shown that the proposed models outperformed the SPoC algorithm in a within-subject experiment. Specifically, the final performances were the following: the SPoC had an RMSE of 0.976, the RPS-SCNN got an RMSE of 0.841, and the RPS-MNet got an RMSE of 0.428.

As a result, DL techniques specifically designed to work with MEG data outperformed the SPoC algorithm in decoding the continuous target variable. Consequently, DL-based application can provide a valuable alternative to decode hand-movement parameters from MEG measurements in a within-subject experiment.