Visual communication in ubiquitous computing: from smartphones to pervasive displays

Thumbnail Image

Files

isbn9789526403472.pdf (11.84 MB)
isbn9789526403472_errata.pdf (29.65 KB)

URL

Journal Title

Journal ISSN

Volume Title

School of Science | Doctoral thesis (article-based) | Defence date: 2021-06-04
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2021

Department

Tietotekniikan laitos
Department of Computer Science

Major/Subject

Mcode

Degree programme

Language

en

Pages

92 + app. 74

Series

Aalto University publication series DOCTORAL DISSERTATIONS, 54/2021

Abstract

Rapid technological advances have enabled the integration of heterogeneous devices into the physical world to provide different kinds of services to users. This led to the emergence of ubiquitous systems that are available at any time and place. A key aspect of such systems is communication, not only among devices but also towards end users. This dissertation investigates several methods to enable visual communication in ubiquitous systems. Specifically, this research addresses visual communication broadly defined in the context of both device-to-device and device-to-people scenarios. In the first case, the main goal is to achieve reliable and effective communication between different devices. To this end, barcodes are leveraged to establish a visual communication channel by using the display and the camera of a smartphone. Accordingly, a mobile application framework is proposed for adaptive and bi-directional commu- nication through barcodes. Moreover, a novel barcode design is devised to provide different data depending on the scanning distance by means of color blending. This novel approach enables new use cases in mobile computing, including casual interactions with public displays and augmented reality for retail. As for the device-to-people scenario, research is carried out to address two important aspects involving how users obtain information from pervasive displays. First, display solutions should be able to reach users and provide meaningful content to them. This can be achieved by extending the coverage of target audience through replication of display content. The same approach allows to monitor the displays to ensure that they are properly functioning and actually showing the intended content. Accordingly, several methods to access display content from different devices are studied and evaluated. Second, user engagement with pervasive displays is particularly challenging as they are easily ignored, especially when they do not convey information relevant to the users. In this context, a novel approach is taken through models suitable to characterize user behavior in observing the content shown on a screen. In particular, the information foraging theory is applied to build several models with the goal of optimizing content selection in tiled display layouts. Accordingly, a display foraging model is devised to predict the time spent by users on a screen as a characterization of their interest. Across several studies, this approach is shown to be effective in characterizing user behavior in real conditions. Finally, inverse foraging models are proposed to infer users' interest from collected data. The models are accurate in predicting interest based on both synthetic and real user data. The obtained results demonstrate that modeling can be effectively applied to improve user engagement in pervasive displays.

Description

Defence is held on 4.6.2021 12:00 – 16:00 Zoom: https://aalto.zoom.us/j/64404604587

Supervising professor

Di Francesco, Mario, Prof., Aalto University, Department of Computer Science, Finland

Thesis advisor

Oulasvirta, Antti, Prof., Aalto University, Finland

Keywords

ubiquitous computing, ubiquitous devices, visual communication

Other note

Parts

  • [Publication 1]: Jacopo Bufalino, Maria L. Montoya Freire, Juho Kannala, Mario Di Francesco. MAMBA: Adaptive and Bi-directional Data Transfer for Reliable Camera-display Communication. In IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, Cork, Ireland, pp, 1-7, September 2020.
    DOI: 10.1109/WoWMoM49955.2020.00059 View at publisher
  • [Publication 2]: Roope Palomäki, Maria L. Montoya Freire, Mario Di Francesco. Distance- Dependent Barcodes for Context-Aware Mobile Applications. In International Conference on Human Computer Interaction with Mobile Devices and Services, Oldenburg, Germany, pp,1-11, October 2020.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-202010165874
    DOI: 10.1145/3379503.3403534 View at publisher
  • [Publication 3]: Maria L. Montoya Freire, Venkata Praneeth Tatiraju, Mohit Sethi, Mario Di Francesco. Replication of web-based pervasive display applications. In ACM International Symposium on Pervasive Displays, Oulu, Finland, pp, 204-211, June 2016.
    DOI: 10.1145/2914920.2915013 View at publisher
  • [Publication 4]: Maria L. Montoya Freire, Dominic Potts, Niraj Ramesh Dayama, Antti Oulasvirta, Mario Di Francesco. Foraging-based optimization of pervasive displays. In Pervasive and Mobile Computing, Volume, 55, pp, 45-58, April 2019.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-201904022522
    DOI: 10.1016/j.pmcj.2019.02.008 View at publisher
  • [Publication 5]: Maria L. Montoya Freire, Antti Oulasvirta, Mario Di Francesco. Inverse Foraging: Inferring Users’ Interest in Pervasive Displays. Submitted, 2021

Citation

Permanent link to this item

https://urn.fi/URN:ISBN:978-952-64-0347-2

Collections

[diss] Perustieteiden korkeakoulu / SCI