Efficient Transfer Learning with Sequential and Multi-Modal Approaches for Electronic Health Records

Abstract

The digital transformation in healthcare has dramatically increased data availability, yet the potential for data-driven insights is frequently constrained by the quality of data. Securing high-quality data is particularly challenging in fields like healthcare, where expert involvement is crucial for gathering, annotating, and ensuring data quality. This thesis applies deep learning to Electronic Health Records (EHR) to enhance predictive accuracy and operational efficiency. Deep learning models are particularly adept at capturing complex and non-linear relationships present in EHR data, but they require extensive training datasets to be effective. This study explores and develops ways to employ transfer learning to effectively mitigate these data constraints.

The thesis tackles four key research questions aimed at improving healthcare outcomes using EHR data. It begins by enhancing prediction accuracy for healthcare utilization through an RNN model with multi-headed attention, which significantly outperforms traditional count-based models and shows robust time generalization. The study then introduces SANSformer, a custom-built, attention-free sequential model optimized for EHR specifics. This model excels in predicting healthcare demand, particularly in diverse patient subgroups, while managing limited data scenarios via transfer learning. Thirdly, the thesis explores the enhancement of neural network similarity metrics in assessing functional similarities, particularly in the context of transfer learning and model performance. It introduces a covariate adjustment to correct traditional metrics, which are often misled by input data structures, ensuring they reflect true functional similarities. Lastly, it explores the integration of expert annotations into the medical CLIP model, eCLIP, which utilizes radiologist eye-gaze heatmaps to substantially improve the quality of embeddings and sample efficiency in multi-modal medical imaging.

The findings from this thesis highlight the significant potential of deep learning to enhance prediction of healthcare outcomes by addressing the unique challenges of EHR data. The research adapts sophisticated deep learning models to meet the complex demands of EHR data and introduces novel techniques like covariate adjustment for similarity metrics and integrating expert annotations to set a foundation for further advancements in healthcare analytics.