Enabling Ubiquitous Augmented Reality with Crowdsourced Indoor Mapping and Localization

Abstract

With a proliferation of sensor-rich small form factor devices such as smart glasses and smartphones, augmented reality (AR) applications attracted tremendous interest from both, industry professionals and academics. AR applications enrich the real-world view, seen by a user, with additional information such as computer-generated 3D artifacts that blend seamlessly with real-world objects. Although popular AR applications, especially AR games, are already used by millions of people, enabling shared and ubiquitous AR experiences is still challenging. It is still highly challenging to provide persistent AR experience which aligns artificial objects seamlessly with designated real-world places and allows multiple users to simultaneously perceive the same objects. Furthermore, enabling truly ubiquitous AR requires AR applications to work in arbitrary environments, while users access the applications via commodity devices such as smartphones. In this dissertation, we focus on enabling technologies for ubiquitous multi-user AR applications for indoor environments. We observe that an accurate, real-time localization system is required, in order to provide ubiquitous AR experience indoors. Consequently, we investigate the applicability of computer vision-based techniques for efficient indoor mapping and study how the maps can be used to enable accurate six-degrees-of-freedom positioning, suitable for AR-based applications. Specifically, we investigate applicability of visual crowdsourcing for mapping and providing accurate and infrastructure-less indoor localization and navigation services. Furthermore, we develop mobile AR applications that use the developed indoor positioning services. We solve the challenge to enable energy-efficient and accurate real-time position and facing direction tracking, which is required to enable seamless AR experiences. Finally, we focus on deployment of the developed real-time AR-based systems on a hierarchical edge cloud environment. In particular, we focus on initial computing capacity planning that satisfies the Quality of Service requirements of the developed systems. In this dissertation we conduct empirical studies in order to answer the research questions. We develop a practical indoor mapping and localization system and a smartphone application that uses the localization system for AR-based indoor navigation. The results of this work provide basis for enabling ubiquitous AR experience within entertainment, productivity and social applications.