Enabling Ubiquitous Augmented Reality with Crowdsourced Indoor Mapping and Localization

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
School of Electrical Engineering | Doctoral thesis (article-based) | Defence date: 2019-06-10
Degree programme
71 + app. 75
Aalto University publication series DOCTORAL DISSERTATIONS, 86/2019
With a proliferation of sensor-rich small form factor devices such as smart glasses and smartphones, augmented reality (AR) applications attracted tremendous interest from both, industry professionals and academics. AR applications enrich the real-world view, seen by a user, with additional information such as computer-generated 3D artifacts that blend seamlessly with real-world objects. Although popular AR applications, especially AR games, are already used by millions of people, enabling shared and ubiquitous AR experiences is still challenging. It is still highly challenging to provide persistent AR experience which aligns artificial objects seamlessly with designated real-world places and allows multiple users to simultaneously perceive the same objects. Furthermore, enabling truly ubiquitous AR requires AR applications to work in arbitrary environments, while users access the applications via commodity devices such as smartphones. In this dissertation, we focus on enabling technologies for ubiquitous multi-user AR applications for indoor environments. We observe that an accurate, real-time localization system is required, in order to provide ubiquitous AR experience indoors. Consequently, we investigate the applicability of computer vision-based techniques for efficient indoor mapping and study how the maps can be used to enable accurate six-degrees-of-freedom positioning, suitable for AR-based applications. Specifically, we investigate applicability of visual crowdsourcing for mapping and providing accurate and infrastructure-less indoor localization and navigation services. Furthermore, we develop mobile AR applications that use the developed indoor positioning services. We solve the challenge to enable energy-efficient and accurate real-time position and facing direction tracking, which is required to enable seamless AR experiences. Finally, we focus on deployment of the developed real-time AR-based systems on a hierarchical edge cloud environment. In particular, we focus on initial computing capacity planning that satisfies the Quality of Service requirements of the developed systems. In this dissertation we conduct empirical studies in order to answer the research questions. We develop a practical indoor mapping and localization system and a smartphone application that uses the localization system for AR-based indoor navigation. The results of this work provide basis for enabling ubiquitous AR experience within entertainment, productivity and social applications.
Supervising professor
Xiao, Yu, Prof., Aalto University, Department of Communications and Networking, Finland
augmented reality, indoor mapping, visual crowdsourcing, indoor navigation, capacity planning
Other note
  • [Publication 1]: Jiang Dong, Marius Noreikis, Yu Xiao, Antti Ylä-Jääski. ViNav: A Vision-based Indoor Navigation System for Smartphones. IEEE Transactions on Mobile Computing, accepted for publication, pages 1–14, 2019.
    DOI: 10.1109/TMC.2018.2857772 View at publisher
  • [Publication 2]: Marius Noreikis, Yu Xiao, Jiyao Hu, Yang Chen. SnapTask: Towards Efficient Visual Crowdsourcing for Indoor Mapping. In Proceedings of IEEE 38th International Conference on Distributed Computing Systems (ICDCS), Vienna, pages 578–588, July 2018.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-201812106047
    DOI: 10.1109/ICDCS.2018.00063 View at publisher
  • [Publication 3]: Marius Noreikis, Yu Xiao, Antti Ylä-Jääski. SeeNav: Seamless and Energy-Efficient Indoor Navigation using Augmented Reality. In Proceedings of the Thematic Workshops in ACM Multimedia, Mountain View, pages 186–193, October 2017.
    DOI: 10.1145/3126686.3126733 View at publisher
  • [Publication 4]: Yuki Sato, Marius Noreikis, Yu Xiao: Low-cost mapping of RFID tags using reader-equipped smartphones. In Proceedings of 15th IEEE International Conference on Mobile Ad Hoc and Sensor Systems (MASS), Chengdu, pages 299–307, October 2018.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-201901141217
    DOI: 10.1109/MASS.2018.00052 View at publisher
  • [Publication 5]: Marius Noreikis, Yu Xiao, Antti Ylä-Jääski. QoS-oriented Capacity Planning for Edge Computing. In Proceedings of IEEE International Conference on Communications (ICC), Paris, pages 1–6, May 2017.
    DOI: 10.1109/ICC.2017.7997387 View at publisher
  • [Publication 6]: Marius Noreikis, Yu Xiao, Yuming Jiang. Edge Capacity Planning for Real Time Compute-Intensive Applications. To appear in the Proceedings of IEEE International Conference on Fog Computing (ICFC), Prague, pages 1–10, June 2019.
  • [Errata file]: Errata of P 4