Methods and applications of mobile audio augmented reality

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
School of Science | Doctoral thesis (article-based) | Defence date: 2016-07-29
Degree programme
118 + app. 72
Aalto University publication series DOCTORAL DISSERTATIONS, 122/2016
In augmented reality, virtual objects are presented as if they were a part of the real world. In mobile audio augmented reality, sounds presented with headphones are perceived as if they originated from the surrounding environment. This thesis investigates potential applications of mobile audio augmented reality and different methods that are needed in these applications. The two main topics studied are distance presentation and spatial audio guidance. Reverberation is known to be an important factor affecting the perceived distance of sound sources. Here, a practical method for modifying the perceived distance of virtual sound sources is investigated, where the temporal envelopes of binaural room impulse responses (BRIRs) are modified. In a listening test, speech sources were presented using these modified BRIRs. The results show that the perceived distance is controlled most effectively by modifying an early-to-late energy ratio with the first 50–100 ms of the BRIR included in the early energy. Presenting large distances in an audio augmented reality environment is difficult, since people underestimate the distances of distant sound sources and very distant sound sources cannot even be heard. In a user study, the presentation of points of interest (POIs) outdoors using auditory distance cues was compared with a voice saying the distance in meters. The results suggest that distances should be given in meters if fairly accurate distance estimates are needed without prior training. With training, however, the user study participants were able to estimate the distances of the POIs fairly accurately based on the provided auditory distance cues, performing the task faster than when the distances were presented in meters. In addition to the presentation of POIs, another type of spatial audio guidance is investigated: using spatialized music to guide pedestrians and cyclists to their destination. Two forms of guidance, route and beacon guidance, were tested in different environments. The user studies showed that music guidance is a pleasant and effective aid for navigation. Both route and beacon guidance were effective methods, but suitable for different environments and circumstances. This thesis also investigates a mobile teleconferencing scenario, where participants can move freely from one location to another. With hear-through headphones, co-located participants can hear each other naturally. To avoid transmitting the speech of the participants to other participants in the same room – as this would be perceived as an echo – acoustic co-location detection is applied. In a user study, utilization of acoustic co-location detection was shown to improve the clarity of communication. Together, the studies presented in this thesis provide methods and guidelines for the development of mobile audio augmented reality applications.
Supervising professor
Lokki, Tapio, Assoc. prof., Aalto University, Department of Computer Science, Finland
Thesis advisor
Lokki, Tapio, Assoc. prof., Aalto University, Department of Computer Science, Finland
audio augmented reality, spatial audio, auditory distance perception, guidance, navigation, communication
Other note
  • [Publication 1]: Robert Albrecht and Tapio Lokki. Adjusting the perceived distance of virtual speech sources by modifying binaural room impulse responses. In Proceedings of the 19th International Conference on Auditory Display, Lodz, Poland, pages 233–241, July 2013.
  • [Publication 2]: Robert Albrecht and Tapio Lokki. Auditory distance presentation in an urban augmented reality environment. ACM Transactions on Applied Perception, volume 12, issue 2, article no. 5, 19 pages, March 2015.
    DOI: 10.1145/2723568 View at publisher
  • [Publication 3]: Robert Albrecht, Riitta Väänänen, and Tapio Lokki. Guided by music: pedestrian and cyclist navigation with route and beacon guidance. Personal and Ubiquitous Computing, volume 20, issue 1, pages 121–145, February 2016.
    DOI: 10.1007/s00779-016-0906-z View at publisher
  • [Publication 4]: Robert Albrecht, Sampo Vesa, Jussi Virolainen, Jussi Mutanen, and Tapio Lokki. Continuous mobile communication with acoustic co-location detection. In 134th Convention of the Audio Engineering Society, Rome, Italy, paper no. 8895, 10 pages, May 2013.