VIMES: A Wearable Memory Assistance System for Automatic Information Retrieval

dc.contributorAalto Universityen
dc.contributor.authorBermejo, Carlosen_US
dc.contributor.authorBraud, Tristanen_US
dc.contributor.authorYang, Jien_US
dc.contributor.authorMirjafari, Shayanen_US
dc.contributor.authorShi, Bowenen_US
dc.contributor.authorXiao, Yuen_US
dc.contributor.authorHui, Panen_US
dc.contributor.departmentHong Kong University of Science and Technologyen_US
dc.contributor.departmentDartmouth Collegeen_US
dc.contributor.departmentMobile Cloud Computingen_US
dc.contributor.departmentUniversity of Helsinkien_US
dc.contributor.departmentDepartment of Communications and Networkingen
dc.description.abstractThe advancement of artificial intelligence and wearable computing triggers the radical innovation of cognitive applications. In this work, we propose VIMES, an augmented reality-based memory assistance system that helps recall declarative memory, such as whom the user meets and what they chat. Through a collaborative method with 20 participants, we design VIMES, a system that runs on smartglasses, takes the first-person audio and video as input, and extracts personal profiles and event information to display on the embedded display or a smartphone. We perform an extensive evaluation with 50 participants to show the effectiveness of VIMES for memory recall. VIMES outperforms (90% memory accuracy) other traditional methods such as self-recall (34%) while offering the best memory experience (Vividness, Coherence, and Visual Perspective all score over 4/5). The user study results show that most participants find VIMES useful (3.75/5) and easy to use (3.46/5).en
dc.description.versionPeer revieweden
dc.identifier.citationBermejo , C , Braud , T , Yang , J , Mirjafari , S , Shi , B , Xiao , Y & Hui , P 2020 , VIMES: A Wearable Memory Assistance System for Automatic Information Retrieval . in Proceedings of the 28th ACM International Conference on Multimedia . ACM , pp. 3191-3200 , ACM International Conference on Multimedia , Virtual, Online , 12/10/2020 .
dc.identifier.otherPURE UUID: d64adf1c-b3b3-444e-a96a-4865d12fabf4en_US
dc.identifier.otherPURE ITEMURL:
dc.identifier.otherPURE FILEURL:
dc.relation.ispartofACM International Conference on Multimediaen
dc.relation.ispartofseriesProceedings of the 28th ACM International Conference on Multimediaen
dc.subject.keywordinformation retrievalen_US
dc.subject.keywordwearable computingen_US
dc.subject.keywordmemory assistance systemen_US
dc.titleVIMES: A Wearable Memory Assistance System for Automatic Information Retrievalen
dc.typeConference article in proceedingsfi