Learning Mental States from Biosignals
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Science |
Doctoral thesis (article-based)
| Defence date: 2013-05-04
Checking the digitized thesis and permission for publishing
Instructions for the author
Instructions for the author
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2013
Major/Subject
Mcode
Degree programme
Language
en
Pages
96 + app. 88
Series
Aalto University publication series DOCTORAL DISSERTATIONS, 61/2013
Abstract
As computing technology evolves, users perform more complex tasks with computers. Hence, users expect from user interfaces to be more proactive than reactive. A proactive interface should anticipate the user’s intentions and take the right action without requiring a user command. The crucial first step for such an interface is to infer the user’s mental state, which gives important cues about user intentions. This thesis consists of several case studies on inferring mental states of computer users. Biosensing technology provides a variety of hardware tools for measuring several aspects of human physiology, which is correlated with emotions and mental processes. However, signals gathered with biosensors are notoriously noisy. The mainstream approach to overcome this noise is either to increase the signal precision by expensive and stationary sensors or to control the experiment setups more heavily. Both of these solutions undermine the usability of the developed methods in real-life user interfaces. In this thesis, machine learning is used as an alternative strategy for handling the biosignal noise in mental state inference. Computer users have been monitored under loosely controlled experiment setups by cheap and inaccurate biosensors, and novel machine learning models that infer mental states such as affective state, mental workload, relevance of a real-world object, and auditory attention are built. The methodological contributions of the thesis are mainly on multi-view learning and multitask learning. Multi-view learning is used for integrating signals of multiple biosensors and the stimuli. Multitask learning is used for inferring multiple mental states at once, and for exploiting the inter-subject similarities for higher prediction accuracy. A novel multitask learning algorithm that transfers knowledge across multi-view learning tasks is introduced. Another novelty is a Bayesian factor analyzer with a time-dependent latent space that captures the dynamic nature of biosignals better than methods that assume independent samples. The overall outcome of the thesis is that it is feasible to predict mental states from unobtrusive biosensors with reasonable accuracy using state-of-the-art machine learning models.Description
Supervising professor
Kaski, Samuel, Prof., Aalto University, FinlandThesis advisor
Klami, Arto, Dr., Aalto University, FinlandKeywords
multitask learning, multiple kernel learning, probabilistic modeling, affective computing, intelligent user interfaces
Other note
Parts
- [Publication 1]: Melih Kandemir, Veli-Matti Saarinen, Samuel Kaski. Inferring Object Relevance from Gaze in Dynamic Scenes. In Eye Tracking Research and Applications, Austin TX, USA, pages 105–108, 2010.
- [Publication 2]: Antti Ajanki, Mark Billinghurst, Hannes Gamper, Toni Jarvenpaa, Melih Kandemir, Samuel Kaski, Markus Koskela, Mikko Kurimo, Jorma Laaksonen, Kai Puolamaki, Teemu Ruokolainen, Timo Tossavainen. An augmented reality interface to contextual information. Virtual Reality, 15(2):161–173, 2011.
- [Publication 3]: Mehmet Gonen, Melih Kandemir, Samuel Kaski. Multitask Learning Using Regularized Multiple Kernel Learning. In International Conference on Neural Information Processing, Shanghai, China, pages 500–509, 2011.
- [Publication 4]: Melih Kandemir, Samuel Kaski. Learning Relevance from Natural Eye Movements in Pervasive Interfaces. In International Conference on Multimodal Interaction, Santa Monica, CA, USA, pages 85–92, 2012.
- [Publication 5]: Melih Kandemir, Arto Klami, Akos Vetek, Samuel Kaski. Unsupervised inference of auditory attention from biosensors. In European Conference on Machine Learning and Practice of Knowledge Discovery in Databases, Bristol, UK, 2012, Lecture Notes in Computer Science, 7524:403–418, 2012.
- [Publication 6]: Melih Kandemir, Akos Vetek, Mehmet Gonen, Arto Klami, Samuel Kaski. Multi-task and multi-view learning of user state. Submitted to a journal, 24 pages, 2012.